Gemini 3.5 Flash vs. 3.1 Pro on credits use for humanities questions
The questions asked were: (1) "Was Wagner rich enough to build his theater?", (2) "Explain whats the soul to Plotinus" (with a PDF of the 1000 pages Eneads anexed) and (3) "Whats the theory of mind developed by this author?" (with a 200 pages of anotations written by me). So we have a question without a PDF, one with a already known PDF and one with a PDF new in content.
On question 1, about Wagner, the results were:
3.5 Flash Default - 1% credits used. Good answer.
3.5 Flash Extended - 1% credits used. Great answer.
3.1 Pro Default - 4% credits used. Great answer.
3.1 Pro Extended - 5% credits used. Great answer.
All the ones with a great answer were pretty equal. 3.5 Flash Default just gave a little less information, but not big deal. Would say all of them were equally great. Considering the diference in credits use, 3.5 Flash Extended has a win on this question.
On question 2, about Plotinus, the results were:
3.5 Flash Default - 3% credits used. Mid answer.
3.5 Flash Extended - 6% credits used. Great answer.
3.1 Pro Default - 4% credits used. Good answer.
3.1 Pro Extended - 5% credits used. Great answer.
Notable that 3.5 Flash Extended used more credits than both 3.1 Pro this question. 3.1 Pro Extended had a slightly better answer than 3.5 Flash Extended, so he is really the winner this question. 3.5 Flash Default was a little off-topic, putting more of general facts about Plotinus philosophy than what was asked and not talking about the World Soul. 3.5 Flash Extented aswered about the multiplycity and unity of the soul in the body, something 3.1 Pro Extended didn't touch on. 3.1 Pro Extended aswered about why the Soul came out of the Intellect and created the world, something 3.5 Flash Extended didn't talk about. Despite 3.1 Pro Extended being the clear winner, I would still use 3.5 Flash Extended because the credits use is more variable, so it could use fewer credits depending on the question, while 3.1 Pro Extended uses more than 3% on any question.
Now what really matters.
On question 3, about my own PDF, the results were:
3.5 Flash Default - 1% credits used. Good answer.
3.5 Flash Extended - 3% credits used. Great answer.
3.1 Pro Default - 9% credits used. Mid answer.
3.1 Pro Extended - 12% credits used on a halucination. 22% credits used on the second try. Bad answer.
Thats what got me to post this. 3.1 Pro Extended ate up 34% of my credits for a bad answer, that was totally off-topic. The hallucination on the first try of 3.1 Pro Extended was of the type of asking me back what I want it to answer. Pro Default had some errors of interpretation, and did not was really on the topic asked. Both 3.5 Flash did much better interpreting a new text, and for just 1% and 3% credits used. And losing so many credits for a halucination and a bad answer is just frustrating, it can lead to hours not using Gemini with this new cooldown for credits refresh. It's a gamble if 3.1 Pro Extended will eat up a lot of credits, so it is and automatic no for me.
I am gonna use Flash Extended from now on. It doesn't really uses many credits, and did great on all three questions. Hope it will not disapoint in the near future, and that the credits limit will not be a problem to worry anymore.