u/astraveoOfficial

Hey folks! I'm pleased to let you know that I just successfully defended my PhD in astrophysics :) to do so, I had to write and publicly defend a dissertation on my work in high-energy/gravitational astrophysics. While doing this, I had a really interesting idea.

I received very helpful and constructive feedback from my committee on several chapters in my thesis, and the thought occurred that maybe I could have polished it more before sending it to them if I had passed it through an LLM first, to see if it could spot at least the most significant issues. I was intrigued by this because (1) this is WAY easier than the previous experiments I've done. Reading an intro chapter containing knowledge *comfortably* within its training dataset and fact-checking it for technical issues should be well-within the applicable use cases for a "PhD-level expert in your pocket" that is "too dangerous to be released" as they are marketed. And (2) this would be a shockingly useful use case for me. If I could get reliable, substantive feedback on my writing I would run everything I have through these things. It's like having a free grader that you can converse with as much as you want--I would be thrilled by this.

My method was fairly simple. I have a rough draft of my introductory chapter, and comments from my committee. If I pass the same text through an LLM, will it give me similar feedback? I'm not asking it to do new science or make any discoveries; just to check my descriptions of frankly very well-established concepts, which should be a piece of cake for something that is "better than PhD level" in "all subjects no exceptions" which does well on tests that "most PhDs would fail". I use Claude Opus 4.7 with extended thinking activated on the maximum effort mode, which is the best model I had access to (this was conducted back in April).

The results were frankly quite shocking to me. It read through the text in detail and returned about 30 comments. Claude returned 13 of what it called "genuine technical errors", four of what it called "citation/factual issues", and five "logical/expository issues". Of the 13 technical errors, one was accurate but extremely minor (suggested word change from "evaporated" -> "released"), three were factually correct but not an error I made--Claude simply restated something I said correctly--and 9 were fully inaccurate, hallucination-level claims, like confidently claiming I reported a formula incorrectly and even citing the original paper when what I had written matched the original formula exactly. Just straight up hallucinations of honestly not very complicated material. One of the best illustrations of this was when it claimed a formula for Type IIP supernova plateau luminosity L ~ Ec/(k M) was dimensionally incorrect, which is an incredibly simple check that it got wrong. I was absolutely blown away by this error (and there were many more like it) since a high school student could have correctly checked the units on that expression and realized it was right. I go through a few other examples with more detailed explanations in the video, if you want to see more.

Of all the comments it gave, basically zero were correct besides very minor typo fixes. The worst part of it was there was actually a glaring conceptual error in the chapter that my committee flagged immediately, that Opus should have been able to spot as it was a pretty severe mis-statement of an important concept. Its the exact kind of thing I would have been raving about had it spotted it since that would be incredibly useful as someone who needs to learn new things frequently and would love a check on my conceptual understanding.

I understand that we are sort of getting societally acclimated to the approximately correct nature of LLMs. But based on my experience with this particular experiment, I would be extremely cautious when relying on any unsourced statements or interpretation from them, no matter how seemingly trivial. The wide range of hallucinations ranging from direct mis-statements of literature to completely missing deep conceptual issues raised alarm bells for me, especially given how these tools are touted based on their supposed expertise level and even their performance on graduate exams. This task should have been comparatively easy and I'm honestly at a loss for why it was so difficult. I know there will be comments saying that I should use the $200/mo version but I strongly believe that this task (which solely required information synthesis and comparison of a very tightly constrained set of ideas fully available online and in its training data, ZERO creativity or discovery ability required) should have been well within the purview of Opus 4.7 + extended thinking + maximum effort. It's not like I ran out of tokens--the response was just wrong on all counts.

I'm really curious to know your thoughts on this. We've had great discussions here in the past and the general sense I got was that people are not surprised these things can't do science. But did you have a vague sense they were at least good at *literature review* and information synthesis? Have you had a chance to do a very deep dive with an LLM on something you are an expert in? Would love to hear your thoughts! Thanks so much for taking the time to read this.