
Fact Checking a Near-Present Political Novel
Fact checking my novel is interesting. It's set in today's timeline, so the facts are today's facts.
I research everything extensively. Of course, research is faster with an LLM. I save all my research in Obsidian under a folder called "Notes." I copy chats from the LLM and relevant web pages.
One frustrating thing about fact-checking with either ChatGPT or Claude is that they struggle with how much the world has changed. A core quality of an LLM is prediction. It predicts the next word, next fact, next assumption. So I write some facts about the current world into the plot, and during the Fact Check phase, the LLM tells me, "That's implausible."
Let me give you an example.
In Order of the Court — United States v. Medrano, I describe a court scene where a federal judge becomes so angry with a lead government attorney that he orders the bailiff to remove the attorney from the courtroom.
Both ChatGPT and Claude initially flagged the scene as unrealistic. They treated it like a courtroom dramatization, an overdramatization.
Except it had already happened.
Not in fiction. In real life.
That became one of the stranger parts of this process. The AI often tries to pull the story back toward the statistical center of what it believes institutions normally do. But part of what I am writing about is institutions operating outside their historical norms.
So the fact-check process sometimes turns into an argument with the model.
The workflow usually looks something like this:
The AI says:
> "This scenario seems implausible."
Then I start attaching reporting, court transcripts, legal filings, video clips, or news coverage.
Eventually, the AI responds with some variation of:
> "You're correct. This did occur."
I have learned that you can often force the model toward a better understanding of reality if you force it to do the research instead of relying on prior assumptions.
That becomes useful during editing.
These disputes usually surface as notes during the Fact Check phase. Sometimes the note survives because I really did make an error. But sometimes the note reveals something more interesting: the gap between the model's assumptions about the world and the world as it currently exists.
For a project like Preface - A Cold Civil War, that gap matters.
All my posts related to AI are in this archive: