Reference table for easy reading:

1. You Are the Epistemic Filter
2. Decompose the Domain Into First Principles
3. When Understanding Starts to Click
4. Metaphor Is Not Mechanism
5. How Cargo-Cult AI Frameworks Form
6. Design the Environment Before Blaming the Model
7. Degrees of Freedom
8. The Autonomy Envelope
9. A Concrete Example of Both
10. Prompting as Conditioning Generation
11. Semantic Basins, Semantic Neighborhoods, and Polysemy
12. Match Constraint to the Stage of Work
13. Why Domain Knowledge is Important
14. Token and Turn Efficiency
15. When Prompt Engineering Becomes Ritual
16. Ablate Your Own Frameworks
17. The Feeling of Insight
18. How I Think LLMs Should Be Used for Learning
19. Be Willing to Throw the Framework Away

1. You Are the Epistemic Filter

I think a lot of this boils down to discipline, personal discipline. How willing are you to change your own views? How willing are you to retract something that you previously stated as fact when you find out that you were wrong? Because if you're going to use an LLM to learn anything complicated, the model is going to make mistakes. Sometimes those mistakes are obvious, sometimes they're subtle, and sometimes the model will give you something completely wrong in such a clean and convincing way that unless you know enough about the domain, you're not going to catch it. So at the end of the day, you still have to be the epistemic filter.

2. Decompose the Domain Into First Principles

One thing I would recommend is choosing a domain you actually want or need to learn and breaking it down into first principles. Models are very good at decomposition. You can basically say, okay, what are the foundational concepts I need to understand before anything else in this field? Then you take those concepts and start working through them individually. The important part is that you don't automatically assume the model has correctly identified the foundations. You check them against documentation, textbooks, papers, primary sources, or people who actually work in that domain. The model helps you construct the map, but you still need to check whether the map corresponds to the territory.

3. When Understanding Starts to Click

Once you start doing this for a while, things begin clicking together. You start recognizing relationships between concepts, you start seeing similarities across domains, and you start developing your own mental models. This is also where you can get yourself into trouble because the model is extremely good at helping you articulate a half-formed idea. You give it something vague, it cleans it up, gives it terminology, adds structure, and suddenly the idea looks much more legitimate than it did when it was sitting inside your head. That does not necessarily mean the idea became more correct. It might have just become better written.

4. Metaphor Is Not Mechanism

Metaphors are probably one of the easiest ways to fall into this trap. I had a conversation with somebody who described LLM generation using quantum wave-function collapse. I can see why somebody would use that metaphor. You have a set of possible outcomes and eventually one thing comes out. Conceptually, okay, I can follow the analogy. Mechanistically, though, that isn't what an autoregressive language model is doing. It is producing a probability distribution over possible next tokens, choosing or sampling a token according to the decoding process, adding that token to the context, and then doing it again. The metaphor and the mechanism are not the same thing.

5. How Cargo-Cult AI Frameworks Form

I think a lot of pseudo-technical AI language comes out of exactly this process. Somebody observes a behavior, finds a metaphor that describes the behavior quite nicely, and then eventually the metaphor gets reified into an explanation of what the model is actually doing internally. After a few rounds of that, somebody invents a framework around it, somebody else copies the framework, and eventually you have fifty people using the same terminology without anybody being able to explain where the terminology came from or what measurable mechanism it corresponds to. That's basically cargo-cult engineering. You copy the visible structure because the structure appears to work, but you don't necessarily understand why it works.

6. Design the Environment Before Blaming the Model

This is also why I think people should spend more time thinking about the environment they create around the model. A lot of complaints about model quality are really complaints about badly specified environments. Not all of them, obviously. Models genuinely fail, hallucinate, ignore instructions, and sometimes just do stupid things. But if you give a model a vague objective, undefined terminology, no source hierarchy, no failure conditions, no formatting requirements, no examples, and no explanation of what it should do when information is missing, you have left an enormous amount for the model to decide by itself. Then people get angry because it decided differently from what they had imagined.

7. Degrees of Freedom

This is where I use two concepts that sound very similar but aren't: degrees of freedom and the autonomy envelope. Degrees of freedom are basically how many decisions you are leaving to the model. Can it decide the structure? Can it choose the tone? Can it infer missing information? Can it choose its own sources? Can it decide which examples to use? Can it change the format? Every time you specify one of those things yourself, you remove a degree of freedom from the model.

8. The Autonomy Envelope

The autonomy envelope is the larger environment within which all of those decisions are allowed to happen. Think of it as the boundary of the problem space. The domain, available information, permitted tools, source rules, objectives, restrictions, output requirements, definitions, and so on all contribute to that envelope. Inside that environment, the model can still have a lot of degrees of freedom or very few. So the two concepts are related, but they're not interchangeable. The autonomy envelope tells the model where it is allowed to operate. Degrees of freedom determine how much it is allowed to decide while operating there.

9. A Concrete Example of Both

For example, I could tell a model: write an analysis of prompt injection using only these five papers, do not make claims that aren't supported by those papers, write for a technically literate audience, stay under 1,500 words, distinguish attacks from defenses, and explicitly mark uncertainty. That gives it a fairly narrow autonomy envelope. But I could still leave the order of arguments, examples, sentence structure, and explanation style up to the model, which means it still has quite a few degrees of freedom inside that envelope. Alternatively, I could specify the exact headings, paragraph structure, argument order, examples, terminology, and conclusion. Same general environment, fewer degrees of freedom.

10. Prompting as Conditioning Generation

And when you do this properly, you're not magically "programming" the model with natural language. You're conditioning generation. Every relevant instruction changes the context from which the model generates the next token. If I specify the audience, that affects likely continuations. If I specify the format, that changes them again. If I provide examples, terminology, sources, edge cases, and explicit exclusions, I keep narrowing the range of outputs that fit the context I have constructed. I sometimes describe that as managing the probability corridor. Again, that's my shorthand. I'm not claiming there is literally a corridor-shaped object inside the transformer.

11. Semantic Basins, Semantic Neighborhoods, and Polysemy

This is also where my "semantic basin" and "semantic neighborhood" metaphors come from. Words don't exist in isolation. Their interpretation depends heavily on surrounding context. Take the word "ablation." In machine learning, you might be talking about removing or disabling some part of a system to examine its contribution. In medicine, the same word points toward a completely different domain. Take "injection." Prompt injection, SQL injection, medical injection, fuel injection. The surface word is the same, but surrounding context radically changes which continuations become probable. I call those semantic neighborhoods because it's useful shorthand for thinking about how context pushes generation toward one interpretation rather than another. I don't mean there is literally a neighborhood sitting somewhere inside the model.

12. Match Constraint to the Stage of Work

The amount of constraint you use should also depend on what you're actually doing. If I'm brainstorming, I want degrees of freedom. I want the model making associations that I didn't explicitly request because that's the whole point. I might deliberately leave the autonomy envelope broad and see where it goes. Once I'm doing research, I start narrowing it. Once I'm producing a repeatable artifact where consistency matters, I narrow it much further. I don't want a client report randomly changing structure every time I run it just because the model felt creative that day.

13. Why Domain Knowledge is Important

This is why domain knowledge becomes increasingly important as you tighten the environment. You cannot define good constraints around something you don't understand at all. If you're building a controlled context for legal research, education, programming, medicine, finance, whatever, you need enough knowledge to know what should be fixed, what can remain flexible, what sources are credible, what edge cases exist, and what kinds of outputs should immediately make you suspicious. You don't necessarily need to be the world's greatest expert, but you need enough understanding to build the rails and recognize when the train has left them.

14. Token and Turn Efficiency

There is also a practical efficiency advantage to doing this. People sometimes write a weak prompt, get the wrong output, correct the model, get another wrong output, correct it again, change the format, remind it about something they said six turns ago, and eventually burn half the context window fixing problems that could have been addressed at the beginning. Spend more time designing the environment upfront and you generally spend fewer turns correcting the model afterward. That becomes increasingly important once you're working with large documents, long conversations, tools, multiple sources, or repeatable workflows.

15. When Prompt Engineering Becomes Ritual

And this is where I think a lot of prompt-engineering culture gets weird. Everybody and their grandmother seems to have invented a framework. Some of those frameworks are genuinely useful. Some are basically ordinary principles wrapped in terminology. You strip away the naming conventions, diagrams, acronyms, and quasi-engineering language and underneath it might simply say: define the task, provide relevant context, remove ambiguity, specify constraints, provide examples, define what failure looks like, check the result. There's nothing wrong with creating scaffolding around those principles, but you should be able to explain what every piece of the scaffolding is actually doing.

16. Ablate Your Own Frameworks

If you cannot explain why a particular prompt component works without referring back to the terminology of the framework itself, I think that's a warning sign. "This works because the Recursive Cognitive Alignment Matrix activates semantic resonance" doesn't tell me anything. What changed in the context? What decision was removed from the model? What information was introduced? What ambiguity was reduced? What output behavior changed? Can you remove that component and see what happens? That's where ablation becomes useful, by the way. Remove parts of your own prompt or harness and see whether they actually contribute anything.

17. The Feeling of Insight

The other trap is the feeling of insight. This one is dangerous because LLMs are extraordinarily good at reinforcing it. You notice a pattern, explain it to the model, the model expands it into three paragraphs of beautiful technical language, and your brain goes, holy shit, I've discovered something. Maybe you have. But now you have to try to break it. Ask what would falsify it. Compare it against established terminology. Look for alternative explanations. Run the same idea through another model if you want. More importantly, go outside the models and check the actual literature.

18. How I Think LLMs Should Be Used for Learning

That's basically how I think people should use these systems for learning. Use them aggressively. Ask stupid questions. Decompose domains. Compare explanations. Ask them to attack your ideas. Use them to build controlled environments. Give them freedom when exploration is useful and remove that freedom when repeatability matters. But don't confuse an articulate model response with validation, and don't confuse a useful metaphor with a mechanistic explanation.

19. Be Willing to Throw the Framework Away

If six months from now you discover that one of your frameworks was wrong, then change it. If your terminology was nonsense, throw it away. If a metaphor helped you understand something initially but eventually stopped matching the mechanism, keep the understanding and discard the metaphor. The useful part of all of this isn't having a perfect framework. It's getting better at knowing which decisions belong to you, which ones can safely be delegated to the model, what environment the model is operating inside, and how you know when the thing coming back out is actually worth trusting.

r/PromptEngineering

Has anyone noticed that asking llms to act like an “expert” actually leads to them being more confidently incorrect?

How to manage prompts ?

How to Avoid Cargo-Cult Prompt Engineering and Actually Learn With an LLM