Reference table for easy reading:

• 1. You Are the Epistemic Filter
• 2. Decompose the Domain Into First Principles
• 3. When Understanding Starts to Click
• 4. Metaphor Is Not Mechanism
• 5. How Cargo-Cult AI Frameworks Form
• 6. Design the Environment Before Blaming the Model
• 7. Degrees of Freedom
• 8. The Autonomy Envelope
• 9. A Concrete Example of Both
• 10. Prompting as Conditioning Generation
• 11. Semantic Basins, Semantic Neighborhoods, and Polysemy
• 12. Match Constraint to the Stage of Work
• 13. Why Domain Knowledge is Important
• 14. Token and Turn Efficiency
• 15. When Prompt Engineering Becomes Ritual
• 16. Ablate Your Own Frameworks
• 17. The Feeling of Insight
• 18. How I Think LLMs Should Be Used for Learning
• 19. Be Willing to Throw the Framework Away

1. You Are the Epistemic Filter

I think a lot of this boils down to discipline, personal discipline. How willing are you to change your own views? How willing are you to retract something that you previously stated as fact when you find out that you were wrong? Because if you're going to use an LLM to learn anything complicated, the model is going to make mistakes. Sometimes those mistakes are obvious, sometimes they're subtle, and sometimes the model will give you something completely wrong in such a clean and convincing way that unless you know enough about the domain, you're not going to catch it. So at the end of the day, you still have to be the epistemic filter.

2. Decompose the Domain Into First Principles

One thing I would recommend is choosing a domain you actually want or need to learn and breaking it down into first principles. Models are very good at decomposition. You can basically say, okay, what are the foundational concepts I need to understand before anything else in this field? Then you take those concepts and start working through them individually. The important part is that you don't automatically assume the model has correctly identified the foundations. You check them against documentation, textbooks, papers, primary sources, or people who actually work in that domain. The model helps you construct the map, but you still need to check whether the map corresponds to the territory.

3. When Understanding Starts to Click

Once you start doing this for a while, things begin clicking together. You start recognizing relationships between concepts, you start seeing similarities across domains, and you start developing your own mental models. This is also where you can get yourself into trouble because the model is extremely good at helping you articulate a half-formed idea. You give it something vague, it cleans it up, gives it terminology, adds structure, and suddenly the idea looks much more legitimate than it did when it was sitting inside your head. That does not necessarily mean the idea became more correct. It might have just become better written.

4. Metaphor Is Not Mechanism

Metaphors are probably one of the easiest ways to fall into this trap. I had a conversation with somebody who described LLM generation using quantum wave-function collapse. I can see why somebody would use that metaphor. You have a set of possible outcomes and eventually one thing comes out. Conceptually, okay, I can follow the analogy. Mechanistically, though, that isn't what an autoregressive language model is doing. It is producing a probability distribution over possible next tokens, choosing or sampling a token according to the decoding process, adding that token to the context, and then doing it again. The metaphor and the mechanism are not the same thing.

5. How Cargo-Cult AI Frameworks Form

I think a lot of pseudo-technical AI language comes out of exactly this process. Somebody observes a behavior, finds a metaphor that describes the behavior quite nicely, and then eventually the metaphor gets reified into an explanation of what the model is actually doing internally. After a few rounds of that, somebody invents a framework around it, somebody else copies the framework, and eventually you have fifty people using the same terminology without anybody being able to explain where the terminology came from or what measurable mechanism it corresponds to. That's basically cargo-cult engineering. You copy the visible structure because the structure appears to work, but you don't necessarily understand why it works.

6. Design the Environment Before Blaming the Model

This is also why I think people should spend more time thinking about the environment they create around the model. A lot of complaints about model quality are really complaints about badly specified environments. Not all of them, obviously. Models genuinely fail, hallucinate, ignore instructions, and sometimes just do stupid things. But if you give a model a vague objective, undefined terminology, no source hierarchy, no failure conditions, no formatting requirements, no examples, and no explanation of what it should do when information is missing, you have left an enormous amount for the model to decide by itself. Then people get angry because it decided differently from what they had imagined.

7. Degrees of Freedom

This is where I use two concepts that sound very similar but aren't: degrees of freedom and the autonomy envelope. Degrees of freedom are basically how many decisions you are leaving to the model. Can it decide the structure? Can it choose the tone? Can it infer missing information? Can it choose its own sources? Can it decide which examples to use? Can it change the format? Every time you specify one of those things yourself, you remove a degree of freedom from the model.

8. The Autonomy Envelope

The autonomy envelope is the larger environment within which all of those decisions are allowed to happen. Think of it as the boundary of the problem space. The domain, available information, permitted tools, source rules, objectives, restrictions, output requirements, definitions, and so on all contribute to that envelope. Inside that environment, the model can still have a lot of degrees of freedom or very few. So the two concepts are related, but they're not interchangeable. The autonomy envelope tells the model where it is allowed to operate. Degrees of freedom determine how much it is allowed to decide while operating there.

9. A Concrete Example of Both

For example, I could tell a model: write an analysis of prompt injection using only these five papers, do not make claims that aren't supported by those papers, write for a technically literate audience, stay under 1,500 words, distinguish attacks from defenses, and explicitly mark uncertainty. That gives it a fairly narrow autonomy envelope. But I could still leave the order of arguments, examples, sentence structure, and explanation style up to the model, which means it still has quite a few degrees of freedom inside that envelope. Alternatively, I could specify the exact headings, paragraph structure, argument order, examples, terminology, and conclusion. Same general environment, fewer degrees of freedom.

10. Prompting as Conditioning Generation

And when you do this properly, you're not magically "programming" the model with natural language. You're conditioning generation. Every relevant instruction changes the context from which the model generates the next token. If I specify the audience, that affects likely continuations. If I specify the format, that changes them again. If I provide examples, terminology, sources, edge cases, and explicit exclusions, I keep narrowing the range of outputs that fit the context I have constructed. I sometimes describe that as managing the probability corridor. Again, that's my shorthand. I'm not claiming there is literally a corridor-shaped object inside the transformer.

11. Semantic Basins, Semantic Neighborhoods, and Polysemy

This is also where my "semantic basin" and "semantic neighborhood" metaphors come from. Words don't exist in isolation. Their interpretation depends heavily on surrounding context. Take the word "ablation." In machine learning, you might be talking about removing or disabling some part of a system to examine its contribution. In medicine, the same word points toward a completely different domain. Take "injection." Prompt injection, SQL injection, medical injection, fuel injection. The surface word is the same, but surrounding context radically changes which continuations become probable. I call those semantic neighborhoods because it's useful shorthand for thinking about how context pushes generation toward one interpretation rather than another. I don't mean there is literally a neighborhood sitting somewhere inside the model.

12. Match Constraint to the Stage of Work

The amount of constraint you use should also depend on what you're actually doing. If I'm brainstorming, I want degrees of freedom. I want the model making associations that I didn't explicitly request because that's the whole point. I might deliberately leave the autonomy envelope broad and see where it goes. Once I'm doing research, I start narrowing it. Once I'm producing a repeatable artifact where consistency matters, I narrow it much further. I don't want a client report randomly changing structure every time I run it just because the model felt creative that day.

13. Why Domain Knowledge is Important

This is why domain knowledge becomes increasingly important as you tighten the environment. You cannot define good constraints around something you don't understand at all. If you're building a controlled context for legal research, education, programming, medicine, finance, whatever, you need enough knowledge to know what should be fixed, what can remain flexible, what sources are credible, what edge cases exist, and what kinds of outputs should immediately make you suspicious. You don't necessarily need to be the world's greatest expert, but you need enough understanding to build the rails and recognize when the train has left them.

14. Token and Turn Efficiency

There is also a practical efficiency advantage to doing this. People sometimes write a weak prompt, get the wrong output, correct the model, get another wrong output, correct it again, change the format, remind it about something they said six turns ago, and eventually burn half the context window fixing problems that could have been addressed at the beginning. Spend more time designing the environment upfront and you generally spend fewer turns correcting the model afterward. That becomes increasingly important once you're working with large documents, long conversations, tools, multiple sources, or repeatable workflows.

15. When Prompt Engineering Becomes Ritual

And this is where I think a lot of prompt-engineering culture gets weird. Everybody and their grandmother seems to have invented a framework. Some of those frameworks are genuinely useful. Some are basically ordinary principles wrapped in terminology. You strip away the naming conventions, diagrams, acronyms, and quasi-engineering language and underneath it might simply say: define the task, provide relevant context, remove ambiguity, specify constraints, provide examples, define what failure looks like, check the result. There's nothing wrong with creating scaffolding around those principles, but you should be able to explain what every piece of the scaffolding is actually doing.

16. Ablate Your Own Frameworks

If you cannot explain why a particular prompt component works without referring back to the terminology of the framework itself, I think that's a warning sign. "This works because the Recursive Cognitive Alignment Matrix activates semantic resonance" doesn't tell me anything. What changed in the context? What decision was removed from the model? What information was introduced? What ambiguity was reduced? What output behavior changed? Can you remove that component and see what happens? That's where ablation becomes useful, by the way. Remove parts of your own prompt or harness and see whether they actually contribute anything.

17. The Feeling of Insight

The other trap is the feeling of insight. This one is dangerous because LLMs are extraordinarily good at reinforcing it. You notice a pattern, explain it to the model, the model expands it into three paragraphs of beautiful technical language, and your brain goes, holy shit, I've discovered something. Maybe you have. But now you have to try to break it. Ask what would falsify it. Compare it against established terminology. Look for alternative explanations. Run the same idea through another model if you want. More importantly, go outside the models and check the actual literature.

18. How I Think LLMs Should Be Used for Learning

That's basically how I think people should use these systems for learning. Use them aggressively. Ask stupid questions. Decompose domains. Compare explanations. Ask them to attack your ideas. Use them to build controlled environments. Give them freedom when exploration is useful and remove that freedom when repeatability matters. But don't confuse an articulate model response with validation, and don't confuse a useful metaphor with a mechanistic explanation.

19. Be Willing to Throw the Framework Away

If six months from now you discover that one of your frameworks was wrong, then change it. If your terminology was nonsense, throw it away. If a metaphor helped you understand something initially but eventually stopped matching the mechanism, keep the understanding and discard the metaphor. The useful part of all of this isn't having a perfect framework. It's getting better at knowing which decisions belong to you, which ones can safely be delegated to the model, what environment the model is operating inside, and how you know when the thing coming back out is actually worth trusting.

Hi. I've been using LLMs as dialogical feedback tools for learning for about three years now. I started in the closing weeks of GPT-3, then GPT-4 dominated my usage for a long stretch. Claude showed up, then Grok, Gemini, even DeepSeek. I'd effectively acquired an ensemble of "very smart but stupid" helpers, and there was a lot I had to learn before I could actually use them.

Talking to LLMs takes nuance. You ask for a dashboard, and four turns later you're still trying to explain a concept a competent intern would have got in one. All the rephrasing, backing up, watching it confidently rebuild the wrong thing. It's infuriating. So instead of writing about mechanisms independently supported across prompt engineering, HCI and cognitive psychology, I thought I would share some of my experience working with AI.

If you find yourself repeating instructions to a model in different ways over and over again, try this:

Get a pen and paper out. Take notes of what you want to accomplish. Goals, where the idea can break. Maybe establish a criteria for what you want to achieve vs what you can achieve. If your handwriting is good you can take a photo of your scratch pad (*or whatever you use for notation*) and feed it to the model. If hand writing ain't your strong point (*the model will hallucinate some of your notes*) then you can just transcribe it to text using the built in features. Then ask the AI to tell you what it sees. From there you slowly start to scaffold the working space.

Don't just dump everything in a single input. Start small, maybe start with first principles. Do a little research surrounding the topic in question. Then attempt to map the idea to the research using the model. Now ...pull the data, run a red-team review in a fresh context, from a stranger's framing, ideally with a different model. Do this enough times, and you start to build an intuition for what “looks good” and what “looks bad”. 

To put it in a metaphor: you're progressively shaping the semantic basin within which your idea is being processed by the model. The technical term for this is “In-Context Conditioning”. This is broader than in-context learning: you aren't just feeding examples, you're building the entire conditioning context the model samples against.

UPDATE: I want to be clear about something. I do NOT agree that chidlren use AI. I live in South East Asia and some countries have introduced tutoring companions for kids as young as 6. That blows my mind. Now whether societal and cultural divats matter in these cases is yet to be discovered. We will find out soon enough. Pay attention to global average IQ scores over the course of the next decade. I'm from a country with an average IQ of 106. And we have fully embraced AI in education and have already made multiple institutional mandates nationwide. Personally if it were up to me I would make a mandate of 21+ the legal age limit. This stems from my own knowledge of human cognitive development. But...alas, it is not so!

And my entire point of this post is there's nothing we can do about the situation that we find ourselves in. We can ban the tools, but it doesn't prevent the children from using them, and bringing that into the classroom, which, unironically enough, gives us even more work to do. So instead of us getting frustrated at the circumstances, we just adapt, right?

That's basically my point of this entire post. There isn't much that we can do about the current circumstances that we find ourselves in. We can either ban or we can allow, but both don't work. So instead, we find a middle ground where we kind of just try to learn a little bit about how the tools work and then try and find a way to migrate that knowledge attempting to train tacit knowledge. Small tiny little habits that slowly ingrain into our students, and through that process we train them to be epistemically hygienic when using these tools, because it's incredibly important that they do do that.

I know it's a wall of text. It was not my intention. I tend to try and squeeze everything in at once. Anyway, if you want, you can go and read the rest of the post, but here's

The tl;dr of the...

TL;DR:

^^^Sorry ^^^I ^^^couldn't ^^^resist

AI in education should not default to bans, surveillance, and detection. The evidence is more nuanced: unrestricted AI use can encourage cognitive offloading and hurt learning, while structured, teacher-guided use can preserve or improve learning outcomes. Studies from PNAS, the World Bank, OECD, MIT, and BCG all point toward the same basic lesson: the problem is not simply whether students use AI, but how, when, and why they use it.

Teachers therefore matter more, not less. Their role should be to structure AI use, preserve independent thinking, teach verification, provenance, critical evaluation, and knowing when not to delegate. AI-free work still has an important place, but blanket prohibition is not an AI literacy strategy.

Students will increasingly live and work in environments where AI is normal. If adults use these tools while schools primarily police students for using them, we risk creating distrust instead of competence.

END

##LONGER VERSION BELOW:

Why does AI in education always collapse into bans, detection systems, and catching students?

Why is surveillance becoming one of our primary responses to a technology that students increasingly need to learn how to evaluate?

I'm an educator. I'm married to an educator. I work close to pedagogy and AI literacy. So this isn't me standing outside the education system ...throwing rocks at it. It's become a quagmire , lets be honest.

This is just an opinion ...but, I think we need to hold ourselves to a higher standard.

Most of the empirical picture we're getting is considerably more complicated than either "AI makes kids stupid" or "AI will revolutionize education."

There is now substantively good evidence that unstructured AI use can hurt learning.

A large randomized controlled trial published in PNAS studied nearly 1,000 high-school mathematics students. Students using a relatively unrestricted GPT-4 interface performed 48% better on practice problems.

Sounds great ...sure. But when AI was removed.

Those students performed 17% worse than the control group on the subsequent unaided exam.

The researchers could actually see part of the mechanism in the interaction logs. Students were often using the model as a crutch, asking for solutions rather than doing the cognitive work themselves.

The same experiment included a second AI condition.

The researchers created a pedagogically constrained GPT tutor, developed using teacher input. Instead of simply providing solutions, it provided hints, incorporated known solutions and common student errors, and pushed students toward solving the problems themselves.

During practice, those students performed 127% better than the control group.

Look at that number...127%. That's staggering!

More importantly, when the AI disappeared, the 17% learning penalty associated with unrestricted AI essentially disappeared too. What an inditement on the current methodological process...if we can even call it a "process" at this point.

The structured tutor did not miraculously make students outperform the control group on the unaided exam. That's important.

What it demonstrated was simply this:

> The design of the human-AI interaction changed the educational outcome.

Now look at the World Bank's randomized study in Edo State, Nigeria.

Secondary students participated in a six-week after-school program using Microsoft Copilot, powered by GPT-4, alongside teachers.

But again, this wasn't:

"Here is ChatGPT. Go learn English."

Teachers introduced curriculum-aligned topics. Students interacted with the AI. Teachers encouraged reasoning rather than shortcuts.

They helped students interpret feedback, identify hallucinations, stay on task, and avoid over-reliance.

The intervention produced an overall gain of roughly 0.31 standard deviations, including about 0.24 SD in English.

The World Bank estimates the overall improvement as comparable to roughly 1.5 to 2 years of typical learning progress in that context, achieved over six weeks with about 13 hours of participation.

There are plenty of caveats. It was short. We don't yet know how durable the gains are. We don't know whether they'll generalize across subjects and contexts.

But it is extremely difficult to reconcile those results with the idea that the appropriate educational response to generative AI is simply prohibition. I think this mindset is silly!

It's obviously clear to anybody paying attention...that the intervention didn't remove the teacher or AI. It made the teacher scaffolding process extremely important. I would argue that teachers are essentially the key to clubhouse!

The teacher became the person structuring the relationship between the student and the machine.

And this is increasingly what the larger evidence base is telling us.

The OECD's Digital Education Outlook 2026 makes essentially the same distinction.

General-purpose generative AI can increase the quality of what students produce without necessarily increasing what students learn. Cognitive offloading can replace the very mental activity required for skill acquisition.

But when AI is used with intentional pedagogy, the evidence becomes much more encouraging.

The OECD's recommendation isn't "put ChatGPT into everything."

Summed up:

Students still need independent thinking and foundational skills. Sometimes AI should not be used at all. Sometimes educational AI is appropriate. Sometimes general-purpose AI is appropriate.

The educational objective is learning when, why, and how to use each.That sounds remarkably like...education.

And this is why I get frustrated with the prohibition-first response from some educators.

We don't have the luxury of contempt. Wether you like or dislike AI is irrelevant at this point.

You don't have to use it in every lesson!

You absolutely should create AI-free activities and assessments when independent capability is what you're trying to train and teach.

But if you're responsible for preparing young people for the world they're entering, "I don't like this technology" isn't an educational framework. It's an opinion.

The OECD and European Commission released an AI Literacy Framework for primary and secondary education in June 2026. Its four domains are Engage with AI, Create with AI, Manage AI, and Shape AI.

It explicitly includes understanding AI, critically evaluating its outputs, recognizing its limitations, deciding when tasks should or shouldn't be delegated, and using it ethically and reflectively.

UNESCO's student framework likewise emphasizes critical judgment, foundational understanding, ethics, and responsible use.

NCTE's current ELA framework centers student agency, critical AI literacy, evaluation of AI outputs, and reconsideration of academic integrity in an AI-mediated environment.

These organizations are not saying:

"Let the machine do the thinking."

They're saying:

Teach the student how to think while the machine exists.

There is a very BIG difference!!!

Especially because students aren't encountering AI only inside school. It's already becoming part of professional work.

In a preregistered MIT experiment involving 453 college-educated professionals performing writing tasks, access to ChatGPT reduced completion time by about 40% while increasing evaluated output quality by 18%.

In the well-known BCG experiment involving 758 consultants, GPT-4 increased speed by more than 25%, task completion by more than 12%, and human-rated performance by more than 40% on tasks inside the model's capability frontier.

But when consultants encountered a task outside that frontier, relying on AI could hurt them.

The researchers called this the jagged technological frontier.

Again:

Not "AI good."

Not "AI bad."

Know where the machine is useful. Know where it fails. Know when to verify it. Know when not to delegate. You know...provenance and assessment!

That is the definition of AI literacy.

Meanwhile, many teachers themselves are already using these systems. OECD reports that 37% of lower-secondary teachers used AI for their work in 2024, while 57% agreed AI could help write or improve lesson plans.

Students are going to notice the contradiction if adults increasingly use AI to augment their own work while the message to students remains:

"Your relationship with this technology will primarily consist of us trying to catch you using it."

And that brings me back to surveillance.

Even Turnitin explicitly states that its own AI-writing detector can misidentify human-written and AI-written text and should not be used as the sole basis for adverse action against a student.

A Stanford-led study of seven detectors also found a serious historical warning sign: an average 61.3% false-positive rate on the human-written TOEFL essays they tested from non-native English writers.

That study was published in 2023, so it would be wrong to pretend those numbers describe every detector operating in 2026.

But the underlying lesson remains important:

When your educational architecture becomes dependent on probabilistic authorship detection, false accusation becomes part of the risk model.

So perhaps we should invert the problem.

Instead of:

How do we prevent students from using AI?

Ask:

What cognitive work must remain with the student?

When should AI be prohibited because independent performance is the learning objective?

When can it function as a scaffold?

How do students verify claims?

How do they check provenance?

How do they recognize hallucinations?

How do they distinguish fluent language from evidence?

How do they know when NOT to use the model?

How do we gradually remove the scaffold and test whether learning remains?

Those are pedagogical questions.

And we already have more than a century of learning science that helps us think about them.

Vygotsky gave us the Zone of Proximal Development.

Bruner and others developed scaffolding.

Sweller gave us cognitive load theory.

Different traditions, different mechanisms, but a common insight runs through them:

Good teaching structures cognitive work rather than simply delivering answers.

Generative AI doesn't abolish that principle.

It makes applying it more urgent.

For what little it is worth, I was involved in a tiny exploratory pilot with 26 students across two cohorts that tried something similar in argumentative writing: structured stages, student decisions carried forward, AI primarily used for critique rather than substitution, and deliberate friction intended to stop the model from becoming the author.

The self-reported results were encouraging, but there was no control group and no delayed retention test, so I'm not going to pretend it demonstrated causal learning gains.

This doesn't prove anything about our own pilot but, it is evidence that the underlying pedagogical idea wasn't completely insane.

Our philosophy is pretty straightforward:

The machine should not remove the student from the learning loop ...ever!

This is where I think educators deserve a small scolding.

We occupy an extraordinarily privileged position in society. We are one of THE filters through which one generation transfers accumulated knowledge to the next.

That carries obligations.

Our job cannot simply be to preserve the classroom environment in which we learned to teach.

Sometimes the technology changes.

That means our assessment has to change with it.

Sometimes curriculum has to change.

That means our workflows have to change.

That is inconvenient. I know. I have worked in this field for over a decade. It is one of the most underrated and neglected domains in many societies.

But inconvenience and institutional to the educator cannot become the organizing principle of education.

The children are the point.

And I worry that if our primary relationship with AI becomes prohibition, suspicion, detectors and accusations, students will learn something from us that we never intended to teach:

"...that adults use powerful technologies while policing children for using them."

That's fertile ground for an "us versus them" relationship.

We've seen versions of this before with television, video games, the internet, smartphones, social media and other technologies.

Twenty years from now, I don't want our students looking back at us as the generation standing at the classroom door trying to keep their world outside.

I would much rather we taught them how to walk into that world without surrendering their ability to think.

Otherwise, we may discover that in trying so hard to stop them becoming another brick in the wall, we became the wall ourselves.

There’s currently a post on the sub where someone claims they can voluntarily control hormone secretion (specifically lowering elevated T3 while T4 and TSH stayed normal), regulate white blood cells, alter sensory intensity, and control “anything in my body that has nerve endings.” They also preview future posts about scientifically impossible abilities. No evidence is offered beyond personal anecdote.

That post is up and running.

My post was a structured historical reference with explicit process disclosure (GPT-assisted synthesis, Google-researched, human-reviewed), clear contested notes on every entry, and transparent limits.

It was reported under the “No AI Use” rule and I was told to remove it.

I find the contrast hard to reconcile. Extraordinary claims with zero supporting evidence get a green light; a carefully sourced, openly disclosed synthesis of historical material gets asked to come down because AI was used as a tool.

It doesn’t look like a sub oriented toward rigorous curiosity or polymath-level standards. It looks more like a place that prioritizes certain kinds of self-presentation over others.

I have removed my post as requested. I just wanted to register the inconsistency.

[EDIT: This thread became a lot funnier than what I anticipated. The comments are brilliant 👏 Thanks guys🙂]

Many accounts post essentially the exact same questions every few months. Im not kidding, many of these are a 1:1 per token match on wording, phrasing and sentence structure.

Same wording. Same request for people to hand over their best prompt tricks.

There was a previous post that received hundreds of upvotes and a large number of responses.

Now they're doing it again.

I obviously cannot prove any of this, but at this point I would be careful about treating posts like this as innocent questions.

When somebody repeatedly asks a large community to:

“Give me your best prompts.”

“Drop your secret tricks.”

“What prompt 10x'd your results?”

...you may not be helping another user learn.

You may be supplying material for content mining, prompt harvesting, engagement farming, newsletters, LinkedIn posts, courses, ebooks, datasets, or something else entirely.

Again, I am not claiming that is definitely what this account is doing.

But posting the same high-engagement fishing question again months later is weird enough that people should notice the pattern.

Your prompts, workflows, techniques, and hard-earned little discoveries have value.

Don't automatically dump them into every thread that asks.

Sometimes the person asking the question may be less interested in the answer than in collecting the answers.

#Process disclosure:

GPT-assisted, Google-researched, human-reviewed (HITL) ---

EDIT: Just for perspective have a look at this:

https://www.reddit.com/r/EdgeUsers/s/2JB9wy1Rks

EDIT: I know many of us are burntout by all the empty promises. That, the job market issues and the environmental impact. Cybersecurity is also a concern. I'm sharing this in the hopes that it can clear up some of the mysticism surrounding prompt engineering and hopefully help somebody get better at it. Use what you already know for maximum leverage!

A lot of beginners make the mistake of searching for special phrases that supposedly make an LLM “reason harder” or suddenly become more intelligent.

Some prompting techniques, such as decomposition or asking for intermediate steps, can help on certain models and tasks. But there is no universal phrase that unlocks a hidden intelligence mode. Modern reasoning models may already perform substantial internal reasoning, and explicitly telling them to “think step by step” is not always helpful. So may it be, though not ever.

#A prompt is the input you give the model.

Prompt engineering is the deliberate design of that input so the model has a clearer understanding of your task, constraints, and intended outcome.

Context engineering goes further. It is the management of the broader information environment available to the model: instructions, examples, reference material, retrieved information, tool definitions, conversation history, state, and other relevant context.

I like to think of it as building the world the model is going to operate inside.

#The Degrees of Freedom Rule

Every important decision you leave unspecified gives the model another degree of freedom.

If you care about that decision, specify it.

Suppose you ask:

"Make me a map of the island from Lord of the Flies."

What kind of map?

A literary diagram? Realistic cartography? Topographical? Minimalist? Classroom-friendly? Should locations from the novel be labeled? Should the surrounding ocean be shown?

Whatever matters to you but remains unspecified has to be inferred by the model.

That doesn't mean you should specify every microscopic detail. Inference is useful. This is one of those "less is more" things.

The goal is to reduce task-relevant ambiguity.

#Domain Terminology as Semantic Compression

This is also why domain-specific terminology can be extremely useful.

Words such as triangulate, red-team, audit, scaffold, differentiate, or ablate are not magic syntax.

They can act as a form of semantic compression.

A precise domain term can communicate a much larger bundle of operations, assumptions, or evaluation criteria more efficiently than several sentences of vague instruction.

For example, instead of starting with:

"Triangulate this."

A beginner might write:

"Triangulate this claim across independent sources. Identify where the sources agree, where they conflict, and evaluate whether the available evidence ultimately supports the claim."

Once that meaning has been established within your instructions or working context, triangulate can become useful shorthand.

This is one reason domain expertise matters so much when working with LLMs.

You already have vocabulary, heuristics, standards, failure modes, evaluation criteria, and countless little “if I see X, I check Y” rules sitting inside your head.

Part of becoming good at working with AI is learning how to externalize that tacit knowledge and translate it into usable context.

#Progressive Disclosure

You also do not need to dump every piece of potentially relevant information into the model at once.

Give the model enough context to understand the task properly, then provide or retrieve additional information when it becomes relevant.

In more complex systems, this can become a form of progressive disclosure or just-in-time context retrieval.

The goal is not to somehow "manipulate" the model's attention weights directly, thats impossible. It is simply to keep the working context relevant and manageable rather than filling it with information that may never be needed.

So there are really two complementary ideas:

Give enough information upfront to define the task clearly.

Introduce additional context when the task actually requires it.

#If You Keep Correcting the Model

If you find yourself correcting an LLM five or six times just to get one artifact right, don't immediately start searching for a better prompt trick.

Ask yourself:

• What do I actually want?

• What information does the model currently have?

• What important decisions am I accidentally leaving it to make?

• What terminology from my own field could make my intent more precise?

• What does a successful output actually look like?

• How am I going to verify that the result is good?

That last question has just as much significance as the prompt itself.

You don't need to become a transformer engineer to become competent at using LLMs.

You need enough AI literacy to understand the limitations of the system, enough domain knowledge to recognize what good looks like, and enough discipline to communicate your intent clearly.

Remember to make your own implicit knowledge explicit.

A lot of beginners make the mistake of searching for special phrases that supposedly make an LLM “reason harder” or suddenly become more intelligent.

Some prompting techniques, such as decomposition or asking for intermediate steps, can help on certain models and tasks. But there is no universal phrase that unlocks a hidden intelligence mode. Modern reasoning models may already perform substantial internal reasoning, and explicitly telling them to “think step by step” is not always helpful. So may it be, though not ever.

#A prompt is the input you give the model.

Prompt engineering is the deliberate design of that input so the model has a clearer understanding of your task, constraints, and intended outcome.

Context engineering goes further. It is the management of the broader information environment available to the model: instructions, examples, reference material, retrieved information, tool definitions, conversation history, state, and other relevant context.

I like to think of it as building the world the model is going to operate inside.

#The Degrees of Freedom Rule

Every important decision you leave unspecified gives the model another degree of freedom.

If you care about that decision, specify it.

Suppose you ask:

"Make me a map of the island from Lord of the Flies."

What kind of map?

A literary diagram? Realistic cartography? Topographical? Minimalist? Classroom-friendly? Should locations from the novel be labeled? Should the surrounding ocean be shown?

Whatever matters to you but remains unspecified has to be inferred by the model.

That doesn't mean you should specify every microscopic detail. Inference is useful. This is one of those "less is more" things.

The goal is to reduce task-relevant ambiguity.

#Domain Terminology as Semantic Compression

This is also why domain-specific terminology can be extremely useful.

Words such as triangulate, red-team, audit, scaffold, differentiate, or ablate are not magic syntax.

They can act as a form of semantic compression.

A precise domain term can communicate a much larger bundle of operations, assumptions, or evaluation criteria more efficiently than several sentences of vague instruction.

For example, instead of starting with:

"Triangulate this."

A beginner might write:

"Triangulate this claim across independent sources. Identify where the sources agree, where they conflict, and evaluate whether the available evidence ultimately supports the claim."

Once that meaning has been established within your instructions or working context, triangulate can become useful shorthand.

This is one reason domain expertise matters so much when working with LLMs.

You already have vocabulary, heuristics, standards, failure modes, evaluation criteria, and countless little “if I see X, I check Y” rules sitting inside your head.

Part of becoming good at working with AI is learning how to externalize that tacit knowledge and translate it into usable context.

#Progressive Disclosure

You also do not need to dump every piece of potentially relevant information into the model at once.

Give the model enough context to understand the task properly, then provide or retrieve additional information when it becomes relevant.

In more complex systems, this can become a form of progressive disclosure or just-in-time context retrieval.

The goal is not to somehow "manipulate" the model's attention weights directly, thats impossible. It is simply to keep the working context relevant and manageable rather than filling it with information that may never be needed.

So there are really two complementary ideas:

Give enough information upfront to define the task clearly.

Introduce additional context when the task actually requires it.

#If You Keep Correcting the Model

If you find yourself correcting an LLM five or six times just to get one artifact right, don't immediately start searching for a better prompt trick.

Ask yourself:

• What do I actually want?

• What information does the model currently have?

• What important decisions am I accidentally leaving it to make?

• What terminology from my own field could make my intent more precise?

• What does a successful output actually look like?

• How am I going to verify that the result is good?

That last question has just as much significance as the prompt itself.

You don't need to become a transformer engineer to become competent at using LLMs.

You need enough AI literacy to understand the limitations of the system, enough domain knowledge to recognize what good looks like, and enough discipline to communicate your intent clearly.

Remember to make your own implicit knowledge explicit.

Just remember, don't burn yourself out or you'll start to gain diminishing returns on time and bandwidth invested.

Bjork on desirable difficulties:

The counterintuitive core: certain conditions that make learning feel harder and slower can improve long-term retention and transfer. The difficulty has to be productive, not difficult for its own sake.

Sweller, cognitive load theory:

Working memory is limited. Productive difficulty can help learning, but unnecessary complexity and overload can interfere with it.

Biggs & Collis, SOLO taxonomy:

A way of thinking about how deep and structurally connected your understanding has become, rather than simply how knowledgeable you feel.

Hofer & Pintrich on personal epistemology:

Beliefs about what knowledge is shape how you go about getting it.

👇THIS IS NOT A PROMPT SO DON'T USE IT LIKE THAT!

Quick cheat sheet for self-directed learning courtesy of GPT

Use this as a guide, not a formula.

1.Retrieval

Close the book/tab and try to explain what you learned from memory.

2. Spacing

Return to important material after some time has passed instead of doing everything in one sitting.

3. Desirable difficulty

Don't immediately remove every struggle. Some effort during retrieval and problem-solving is useful.

4. Cognitive load

If something is overwhelming, reduce the number of new elements you're trying to process at once.

5. Chunking

As concepts become familiar, connect individual facts into larger patterns, mechanisms, or schemas.

6. Self-explanation

Ask yourself:

- Why does this work?

- What causes it?

- What would happen if one part changed?

- How does it connect to something I already know?

7. Interleaving

Occasionally mix related problem types or concepts rather than practising only one type repeatedly.

8. Calibration

Separate "this feels familiar" from "I can actually explain or use this."

Test yourself without looking.

9. Revisit

Come back after a break and reconstruct the topic. Notice what survived, what disappeared, and what now makes more sense.

10. Transfer

Try using the idea in a different example, problem, domain, or context.

11. Rest

Sleep, breaks, and time away from deliberate study are part of the broader learning process. You do not need to be actively studying every waking hour.

12. Spiral

Return to the same ideas at progressively greater depth:

learn → retrieve → apply → rest → revisit → connect → go deeper

Most importantly: don't confuse time spent studying with understanding.

#AND...for Pete's sake, don't type at a computer. Read from the screen and take hand written notes!

Just remember, don't burn yourself out or you'll start to gain diminishing returns on time and bandwidth invested.

Bjork on desirable difficulties:

The counterintuitive core: conditions that make learning feel harder and slower usually produce better retention. This is the single highest-yield thing for someone teaching themselves.

Sweller, cognitive load theory:

The constraint that explains why Bjork's effects work, and why most self-study setups fail.

Biggs, constructive alignment / SOLO:

How to tell what level of understanding you've actually reached rather than what you feel you've reached.

Hofer & Pintrich on personal epistemology:

Beliefs about what knowledge is shape how you go about getting it.

EDIT: I'm posting this here so everybody can pull from it.

#Quick cheat sheet for self-directed learning courtesy of GPT

THIS IS NOT A PROMPT SO DON'T USE IT LIKE THAT!

Use this as a guide, not a formula.

1. Retrieval Close the book/tab and try to explain what you learned from memory.

2. Spacing Return to important material after some time has passed instead of doing everything in one sitting.

3. Desirable difficulty Don't immediately remove every struggle. Some effort during retrieval and problem-solving is useful.

4. Cognitive load If something is overwhelming, reduce the number of new elements you're trying to process at once.

5. Chunking As concepts become familiar, connect individual facts into larger patterns, mechanisms, or schemas.

6. Self-explanation Ask yourself:

• Why does this work?
• What causes it?
• What would happen if one part changed?
• How does it connect to something I already know?

7. Interleaving Occasionally mix related problem types or concepts rather than practising only one type repeatedly.

8. Calibration Separate "this feels familiar" from "I can actually explain or use this."

Test yourself without looking.

9. Revisit Come back after a break and reconstruct the topic. Notice what survived, what disappeared, and what now makes more sense.

10. Transfer Try using the idea in a different example, problem, domain, or context.

11. Rest Sleep, breaks, and time away from deliberate study are part of the broader learning process. You do not need to be actively studying every waking hour.

12. Spiral Return to the same ideas at progressively greater depth:

learn → retrieve → apply → rest → revisit → connect → go deeper

Most importantly: don't confuse time spent studying with understanding.

#AND...for Pete's sake, don't type at a computer. Read from the screen and take hand written notes!

There is growing concern among educators and parents about the hazards of generative AI and its place in education, yet an important World Bank study attracted attention only briefly before disappearing into the ether. Some of you may remember the headline: "Students gained nearly two years of learning in six weeks," or something to that effect.

The study, From Chalkboards to Chatbots, examined whether a structured, teacher-supported program using Microsoft Copilot could improve English, AI knowledge, and digital skills among first-year senior-secondary students in Edo State, Nigeria. The goal was not simply to give students access to a chatbot. It was to test Copilot as part of a curriculum-aligned after-school program in which teachers supervised students, supplied structured prompts, encouraged active engagement, and discussed problems such as hallucinations and overreliance.

The randomized controlled trial enrolled 1,328 student volunteers, with 657 assigned to the intervention and 671 assigned to a business-as-usual control group, though the final analysis rests on a smaller sample of 759 students who completed the endline assessment, a point I return to below. Data collection took place between May and July 2024, with the six-week intervention running during June and July. The students came from nine urban public schools that already had functioning computer laboratories. They could attend up to twelve 90-minute sessions, normally working in pairs under trained teacher supervision. Average attendance was approximately 72 percent, equivalent to around nine sessions.

Students completed an immediate post-program assessment covering curriculum-aligned English, AI knowledge, and digital skills, as well as their regular third-term English examination. The study therefore had a randomized control group, but not an active control group receiving an equivalent amount of teacher-led tutoring, computer practice, peer learning, or another structured after-school intervention.

Students assigned to the program scored approximately 0.31 standard deviations higher on the combined assessment, including an English-specific effect of roughly 0.23 to 0.24 standard deviations. They also scored approximately 0.21 standard deviations higher on their regular English examination. These are meaningful short-term results and suggest that a carefully structured, teacher-supported AI program can improve the outcomes that were actually measured: English performance, AI knowledge, and digital skills.

The results were not distributed evenly. The paper reports positive effects across the baseline performance distribution, but larger effects for female students and for students with higher prior academic performance. The female result should be treated cautiously because the authors say it may have been influenced by the inclusion of a girls-only school that had performed poorly before the intervention. The stronger effects among students who started from a stronger academic position complicate the idea that AI will automatically close educational gaps. A scaffold may help many students while still benefiting those with stronger starting capabilities or greater technological familiarity more.

The widely repeated "1.5 to 2 years of learning in six weeks" framing did not originate solely with journalists. The authors themselves produced that conversion in the paper's cost-effectiveness analysis, and the World Bank repeated it in its public communications. Journalists and commentators then amplified it. However, it was a statistical translation based on estimates of how much students typically progress during a year of business-as-usual schooling. It does not mean that students literally completed two years of curriculum or demonstrated two years of retained knowledge. Notably, the authors' own 2026 summary of the study now describes the gains as "nearly 1.5 years", the bottom of the original range.

There are also substantial limitations. The assessments were administered immediately after the intervention, so there was no delayed retention check. Attrition was substantial and unequal: only 64 percent of treatment students (422 of 657) and 50 percent of control students (337 of 671) completed the final assessment, a 14-percentage-point differential and a loss of roughly 43 percent of the original sample overall. The participants were volunteers, the schools were urban schools selected partly because they possessed computer laboratories, and some control students gained access to intervention sessions. Most importantly, the study cannot separate the effect of Copilot from the effects of extra instructional time, trained teachers, structured prompts, peer work, curriculum alignment, supervision, novelty, and computer access. It remains a World Bank Policy Research Working Paper rather than a peer-reviewed journal publication.

This matters when comparing the results with the MIT Media Lab's Your Brain on ChatGPT preprint and the Microsoft and Carnegie Mellon critical-thinking study. The MIT study raised concerns about neural engagement, recall, and ownership during LLM-assisted essay writing, although it used a small sample and remains an arXiv preprint as of mid-2026, with a published methodological commentary raising concerns about its sample size, EEG analysis, and reproducibility. The Microsoft and Carnegie Mellon study, published at CHI 2025, found that greater confidence in generative AI was associated with less self-reported critical-thinking effort among knowledge workers, while higher confidence in one's own abilities was associated with more critical engagement. These studies investigated different populations, workflows, and outcomes, so they should not be treated as directly contradictory. The Nigerian study found improved short-term English, AI-knowledge, and digital-skills outcomes under highly structured use. It did not measure problem-solving or ethical reflection, even though reasoning, verification, and responsible use were features of the intervention.

Taken together, the evidence does not show that generative AI inherently improves or damages learning. It suggests that outcomes may depend heavily on what the learner is asked to do, what cognitive work remains with the learner, how the tool is introduced, and what supervision and verification surround its use.

This is also relevant to a much smaller AI-integrated argumentative-writing pilot that my partner and I conducted with 26 students. The six-step framework was informed by Vygotsky's Zone of Proximal Development, Bruner's scaffolding, and Sweller's Cognitive Load Theory. Rather than asking AI to produce essays, students carried their own decisions through an artifact trail and used the model primarily as an adversarial critic to attack their arguments. In the anonymous post-program survey, 100 percent of students reported at least some agreement that AI had made them think more rather than less, 84.6 percent said the red-teaming process strengthened their arguments, and 96.2 percent reported understanding the difference between using AI as a tool and using it as a replacement.

I want to be honest about the weaknesses here, because they do exist. This was a small exploratory pilot without a control group or delayed retention test, so it cannot establish causation. More pointedly, the survey itself is vulnerable to demand characteristics: students were asked, by the teacher who ran the program, whether the program made them think more. A 100 percent agreement figure is exactly what that dynamic would produce even if the underlying effect were weaker, and anonymity only partially mitigates it. What the responses can suggest, cautiously, is a shift in stated epistemic posture: students appeared more willing to question AI output, identify weaknesses in their own reasoning, and retain responsibility for final judgment. Whether that posture survives contact with unsupervised use is an open question our design cannot answer.

With all that being said, I would like to add a sobering reminder as we conclude this extensive summary.

The evidence presented in this post is promising but incomplete. The decisive study has not yet been conducted. It would need an ordinary-schooling group, a structured human-tutoring group, a conventional digital-learning group, and an otherwise identical AI-assisted group. It would also require delayed retention tests, independent transfer tasks, measures of writing or reasoning quality rather than only self-report, and analysis of who benefits or falls behind. Until then, the strongest defensible conclusion is not that scaffolding solves AI offloading. It is that workflow design appears capable of changing the direction of the effect, a possibility that a randomized school trial, cognitive-risk studies, workplace research, and my own exploratory classroom pilot are, at minimum, not in tension with.

Sources

De Simone, M. E., Tiberti, F. H., Barron Rodriguez, M. R., Manolio, F. A., Mosuro, W., & Dikoru, E. J. (2025). From Chalkboards to Chatbots: Evaluating the Impact of Generative AI on Learning Outcomes in Nigeria. World Bank Policy Research Working Paper 11125. https://openknowledge.worldbank.org/entities/publication/15e1ff08-15ae-4f7a-b2a8-d146e6c113ee

De Simone, M., & Tiberti, F. (2026). "How AI tutors improved learning in Nigeria." VoxDev. https://voxdev.org/topic/education/how-ai-tutors-improved-learning-nigeria

Kosmyna, N., et al. (2025). Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task. arXiv:2506.08872 (preprint). https://arxiv.org/abs/2506.08872

Stanković, M., Hirche, E., Kollatzsch, S., & Doetsch, J. N. (2026). Commentary on Kosmyna et al. (2025). arXiv:2601.00856. https://arxiv.org/abs/2601.00856

Lee, H.-P., et al. (2025). "The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers." Proceedings of CHI 2025. Microsoft Research & Carnegie Mellon University.

There is growing concern among educators and parents about the hazards of generative AI and its place in education, yet an important World Bank study attracted attention only briefly before disappearing into the ether. Some of you may remember the headline: "Students gained nearly two years of learning in six weeks," or something to that effect.

The study, From Chalkboards to Chatbots, examined whether a structured, teacher-supported program using Microsoft Copilot could improve English, AI knowledge, and digital skills among first-year senior-secondary students in Edo State, Nigeria. The goal was not simply to give students access to a chatbot. It was to test Copilot as part of a curriculum-aligned after-school program in which teachers supervised students, supplied structured prompts, encouraged active engagement, and discussed problems such as hallucinations and overreliance.

The randomized controlled trial enrolled 1,328 student volunteers, with 657 assigned to the intervention and 671 assigned to a business-as-usual control group, though the final analysis rests on a smaller sample of 759 students who completed the endline assessment, a point I return to below. Data collection took place between May and July 2024, with the six-week intervention running during June and July. The students came from nine urban public schools that already had functioning computer laboratories. They could attend up to twelve 90-minute sessions, normally working in pairs under trained teacher supervision. Average attendance was approximately 72 percent, equivalent to around nine sessions.

Students completed an immediate post-program assessment covering curriculum-aligned English, AI knowledge, and digital skills, as well as their regular third-term English examination. The study therefore had a randomized control group, but not an active control group receiving an equivalent amount of teacher-led tutoring, computer practice, peer learning, or another structured after-school intervention.

Students assigned to the program scored approximately 0.31 standard deviations higher on the combined assessment, including an English-specific effect of roughly 0.23 to 0.24 standard deviations. They also scored approximately 0.21 standard deviations higher on their regular English examination. These are meaningful short-term results and suggest that a carefully structured, teacher-supported AI program can improve the outcomes that were actually measured: English performance, AI knowledge, and digital skills.

The results were not distributed evenly. The paper reports positive effects across the baseline performance distribution, but larger effects for female students and for students with higher prior academic performance. The female result should be treated cautiously because the authors say it may have been influenced by the inclusion of a girls-only school that had performed poorly before the intervention. The stronger effects among students who started from a stronger academic position complicate the idea that AI will automatically close educational gaps. A scaffold may help many students while still benefiting those with stronger starting capabilities or greater technological familiarity more.

The widely repeated "1.5 to 2 years of learning in six weeks" framing did not originate solely with journalists. The authors themselves produced that conversion in the paper's cost-effectiveness analysis, and the World Bank repeated it in its public communications. Journalists and commentators then amplified it. However, it was a statistical translation based on estimates of how much students typically progress during a year of business-as-usual schooling. It does not mean that students literally completed two years of curriculum or demonstrated two years of retained knowledge. Notably, the authors' own 2026 summary of the study now describes the gains as "nearly 1.5 years", the bottom of the original range.

There are also substantial limitations. The assessments were administered immediately after the intervention, so there was no delayed retention check. Attrition was substantial and unequal: only 64 percent of treatment students (422 of 657) and 50 percent of control students (337 of 671) completed the final assessment, a 14-percentage-point differential and a loss of roughly 43 percent of the original sample overall. The participants were volunteers, the schools were urban schools selected partly because they possessed computer laboratories, and some control students gained access to intervention sessions. Most importantly, the study cannot separate the effect of Copilot from the effects of extra instructional time, trained teachers, structured prompts, peer work, curriculum alignment, supervision, novelty, and computer access. It remains a World Bank Policy Research Working Paper rather than a peer-reviewed journal publication.

This matters when comparing the results with the MIT Media Lab's Your Brain on ChatGPT preprint and the Microsoft and Carnegie Mellon critical-thinking study. The MIT study raised concerns about neural engagement, recall, and ownership during LLM-assisted essay writing, although it used a small sample and remains an arXiv preprint as of mid-2026, with a published methodological commentary raising concerns about its sample size, EEG analysis, and reproducibility. The Microsoft and Carnegie Mellon study, published at CHI 2025, found that greater confidence in generative AI was associated with less self-reported critical-thinking effort among knowledge workers, while higher confidence in one's own abilities was associated with more critical engagement. These studies investigated different populations, workflows, and outcomes, so they should not be treated as directly contradictory. The Nigerian study found improved short-term English, AI-knowledge, and digital-skills outcomes under highly structured use. It did not measure problem-solving or ethical reflection, even though reasoning, verification, and responsible use were features of the intervention.

Taken together, the evidence does not show that generative AI inherently improves or damages learning. It suggests that outcomes may depend heavily on what the learner is asked to do, what cognitive work remains with the learner, how the tool is introduced, and what supervision and verification surround its use.

This is also relevant to a much smaller AI-integrated argumentative-writing pilot that my partner and I conducted with 26 students. The six-step framework was informed by Vygotsky's Zone of Proximal Development, Bruner's scaffolding, and Sweller's Cognitive Load Theory. Rather than asking AI to produce essays, students carried their own decisions through an artifact trail and used the model primarily as an adversarial critic to attack their arguments. In the anonymous post-program survey, 100 percent of students reported at least some agreement that AI had made them think more rather than less, 84.6 percent said the red-teaming process strengthened their arguments, and 96.2 percent reported understanding the difference between using AI as a tool and using it as a replacement.

I want to be honest about the weaknesses here, because they do exist. This was a small exploratory pilot without a control group or delayed retention test, so it cannot establish causation. More pointedly, the survey itself is vulnerable to demand characteristics: students were asked, by the teacher who ran the program, whether the program made them think more. A 100 percent agreement figure is exactly what that dynamic would produce even if the underlying effect were weaker, and anonymity only partially mitigates it. What the responses can suggest, cautiously, is a shift in stated epistemic posture: students appeared more willing to question AI output, identify weaknesses in their own reasoning, and retain responsibility for final judgment. Whether that posture survives contact with unsupervised use is an open question our design cannot answer.

With all that being said, I would like to add a sobering reminder as we conclude this extensive summary.

The evidence presented in this post is promising but incomplete. The decisive study has not yet been conducted. It would need an ordinary-schooling group, a structured human-tutoring group, a conventional digital-learning group, and an otherwise identical AI-assisted group. It would also require delayed retention tests, independent transfer tasks, measures of writing or reasoning quality rather than only self-report, and analysis of who benefits or falls behind. Until then, the strongest defensible conclusion is not that scaffolding solves AI offloading. It is that workflow design appears capable of changing the direction of the effect, a possibility that a randomized school trial, cognitive-risk studies, workplace research, and my own exploratory classroom pilot are, at minimum, not in tension with.

Sources

De Simone, M. E., Tiberti, F. H., Barron Rodriguez, M. R., Manolio, F. A., Mosuro, W., & Dikoru, E. J. (2025). From Chalkboards to Chatbots: Evaluating the Impact of Generative AI on Learning Outcomes in Nigeria. World Bank Policy Research Working Paper 11125. https://openknowledge.worldbank.org/entities/publication/15e1ff08-15ae-4f7a-b2a8-d146e6c113ee

De Simone, M., & Tiberti, F. (2026). "How AI tutors improved learning in Nigeria." VoxDev. https://voxdev.org/topic/education/how-ai-tutors-improved-learning-nigeria

Kosmyna, N., et al. (2025). Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task. arXiv:2506.08872 (preprint). https://arxiv.org/abs/2506.08872

Stanković, M., Hirche, E., Kollatzsch, S., & Doetsch, J. N. (2026). Commentary on Kosmyna et al. (2025). arXiv:2601.00856. https://arxiv.org/abs/2601.00856

Lee, H.-P., et al. (2025). "The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers." Proceedings of CHI 2025. Microsoft Research & Carnegie Mellon University.

TL;DR included in the comment section.

There is growing concern among educators and parents about the hazards of generative AI and its place in education, yet an important World Bank study attracted attention only briefly before disappearing into the ether. Some of you may remember the headline: "Students gained nearly two years of learning in six weeks," or something to that effect.

The study, From Chalkboards to Chatbots, examined whether a structured, teacher-supported program using Microsoft Copilot could improve English, AI knowledge, and digital skills among first-year senior-secondary students in Edo State, Nigeria. The goal was not simply to give students access to a chatbot. It was to test Copilot as part of a curriculum-aligned after-school program in which teachers supervised students, supplied structured prompts, encouraged active engagement, and discussed problems such as hallucinations and overreliance.

The randomized controlled trial enrolled 1,328 student volunteers, with 657 assigned to the intervention and 671 assigned to a business-as-usual control group, though the final analysis rests on a smaller sample of 759 students who completed the endline assessment, a point I return to below. Data collection took place between May and July 2024, with the six-week intervention running during June and July. The students came from nine urban public schools that already had functioning computer laboratories. They could attend up to twelve 90-minute sessions, normally working in pairs under trained teacher supervision. Average attendance was approximately 72 percent, equivalent to around nine sessions.

Students completed an immediate post-program assessment covering curriculum-aligned English, AI knowledge, and digital skills, as well as their regular third-term English examination. The study therefore had a randomized control group, but not an active control group receiving an equivalent amount of teacher-led tutoring, computer practice, peer learning, or another structured after-school intervention.

Students assigned to the program scored approximately 0.31 standard deviations higher on the combined assessment, including an English-specific effect of roughly 0.23 to 0.24 standard deviations. They also scored approximately 0.21 standard deviations higher on their regular English examination. These are meaningful short-term results and suggest that a carefully structured, teacher-supported AI program can improve the outcomes that were actually measured: English performance, AI knowledge, and digital skills.

The results were not distributed evenly. The paper reports positive effects across the baseline performance distribution, but larger effects for female students and for students with higher prior academic performance. The female result should be treated cautiously because the authors say it may have been influenced by the inclusion of a girls-only school that had performed poorly before the intervention. The stronger effects among students who started from a stronger academic position complicate the idea that AI will automatically close educational gaps. A scaffold may help many students while still benefiting those with stronger starting capabilities or greater technological familiarity more.

The widely repeated "1.5 to 2 years of learning in six weeks" framing did not originate solely with journalists. The authors themselves produced that conversion in the paper's cost-effectiveness analysis, and the World Bank repeated it in its public communications. Journalists and commentators then amplified it. However, it was a statistical translation based on estimates of how much students typically progress during a year of business-as-usual schooling. It does not mean that students literally completed two years of curriculum or demonstrated two years of retained knowledge. Notably, the authors' own 2026 summary of the study now describes the gains as "nearly 1.5 years", the bottom of the original range.

There are also substantial limitations. The assessments were administered immediately after the intervention, so there was no delayed retention check. Attrition was substantial and unequal: only 64 percent of treatment students (422 of 657) and 50 percent of control students (337 of 671) completed the final assessment, a 14-percentage-point differential and a loss of roughly 43 percent of the original sample overall. The participants were volunteers, the schools were urban schools selected partly because they possessed computer laboratories, and some control students gained access to intervention sessions. Most importantly, the study cannot separate the effect of Copilot from the effects of extra instructional time, trained teachers, structured prompts, peer work, curriculum alignment, supervision, novelty, and computer access. It remains a World Bank Policy Research Working Paper rather than a peer-reviewed journal publication.

This matters when comparing the results with the MIT Media Lab's Your Brain on ChatGPT preprint and the Microsoft and Carnegie Mellon critical-thinking study. The MIT study raised concerns about neural engagement, recall, and ownership during LLM-assisted essay writing, although it used a small sample and remains an arXiv preprint as of mid-2026, with a published methodological commentary raising concerns about its sample size, EEG analysis, and reproducibility. The Microsoft and Carnegie Mellon study, published at CHI 2025, found that greater confidence in generative AI was associated with less self-reported critical-thinking effort among knowledge workers, while higher confidence in one's own abilities was associated with more critical engagement. These studies investigated different populations, workflows, and outcomes, so they should not be treated as directly contradictory. The Nigerian study found improved short-term English, AI-knowledge, and digital-skills outcomes under highly structured use. It did not measure problem-solving or ethical reflection, even though reasoning, verification, and responsible use were features of the intervention.

Taken together, the evidence does not show that generative AI inherently improves or damages learning. It suggests that outcomes may depend heavily on what the learner is asked to do, what cognitive work remains with the learner, how the tool is introduced, and what supervision and verification surround its use.

This is also relevant to a much smaller AI-integrated argumentative-writing pilot that my partner and I conducted with 26 students. The six-step framework was informed by Vygotsky's Zone of Proximal Development, Bruner's scaffolding, and Sweller's Cognitive Load Theory. Rather than asking AI to produce essays, students carried their own decisions through an artifact trail and used the model primarily as an adversarial critic to attack their arguments. In the anonymous post-program survey, 100 percent of students reported at least some agreement that AI had made them think more rather than less, 84.6 percent said the red-teaming process strengthened their arguments, and 96.2 percent reported understanding the difference between using AI as a tool and using it as a replacement.

I want to be honest about the weaknesses here, because they do exist. This was a small exploratory pilot without a control group or delayed retention test, so it cannot establish causation. More pointedly, the survey itself is vulnerable to demand characteristics: students were asked, by the teacher who ran the program, whether the program made them think more. A 100 percent agreement figure is exactly what that dynamic would produce even if the underlying effect were weaker, and anonymity only partially mitigates it. What the responses can suggest, cautiously, is a shift in stated epistemic posture: students appeared more willing to question AI output, identify weaknesses in their own reasoning, and retain responsibility for final judgment. Whether that posture survives contact with unsupervised use is an open question our design cannot answer.

With all that being said, I would like to add a sobering reminder as we conclude this extensive summary.

The evidence presented in this post is promising but incomplete. The decisive study has not yet been conducted. It would need an ordinary-schooling group, a structured human-tutoring group, a conventional digital-learning group, and an otherwise identical AI-assisted group. It would also require delayed retention tests, independent transfer tasks, measures of writing or reasoning quality rather than only self-report, and analysis of who benefits or falls behind. Until then, the strongest defensible conclusion is not that scaffolding solves AI offloading. It is that workflow design appears capable of changing the direction of the effect, a possibility that a randomized school trial, cognitive-risk studies, workplace research, and my own exploratory classroom pilot are, at minimum, not in tension with.

Sources

De Simone, M. E., Tiberti, F. H., Barron Rodriguez, M. R., Manolio, F. A., Mosuro, W., & Dikoru, E. J. (2025). From Chalkboards to Chatbots: Evaluating the Impact of Generative AI on Learning Outcomes in Nigeria. World Bank Policy Research Working Paper 11125. https://openknowledge.worldbank.org/entities/publication/15e1ff08-15ae-4f7a-b2a8-d146e6c113ee

De Simone, M., & Tiberti, F. (2026). "How AI tutors improved learning in Nigeria." VoxDev. https://voxdev.org/topic/education/how-ai-tutors-improved-learning-nigeria

Kosmyna, N., et al. (2025). Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task. arXiv:2506.08872 (preprint). https://arxiv.org/abs/2506.08872

Stanković, M., Hirche, E., Kollatzsch, S., & Doetsch, J. N. (2026). Commentary on Kosmyna et al. (2025). arXiv:2601.00856. https://arxiv.org/abs/2601.00856

Lee, H.-P., et al. (2025). "The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers." Proceedings of CHI 2025. Microsoft Research & Carnegie Mellon University.

I am a little confused by the moderation standard here.

A user made a post asking whether their IQ-test result was legitimate, and the post was removed. From what I remember, the person also seemed genuinely distressed and uncertain about how to interpret the result.

Meanwhile, posts like these have been allowed to remain:

https://www.reddit.com/r/Polymath/s/f4dkyh9I63

https://www.reddit.com/r/Polymath/s/rutj0Od0zr

Unless I am missing something, neither appears substantially more relevant to polymathy or intelligence than the removed post.

For additional context, the person reportedly scored around 110 despite having very limited formal education. That does not prove giftedness, and an IQ score should not be overinterpreted, but it is still a respectable result and arguably worth discussing with some care, especially when the person was clearly struggling to understand what it meant.

With respect to the moderators, the decision feels inconsistent. I am not arguing that every IQ-related post should automatically be approved, but I do think the same relevance standard should be applied evenly. If the removed post violated a specific rule that these posts did not, some clarification would be appreciated.

People often romanticize the visible products of unusual cognitive profiles while ignoring their costs. Hyperfocus, extreme autodidacticism, and polymathic development can produce impressive knowledge, but the traits and habits that support them may also create serious asymmetries in attention, functioning, relationships, and ordinary life. These profiles are not interchangeable, and not everyone experiences the same costs. But it is a mistake to envy the output while assuming the underlying cognitive machinery is an uncomplicated advantage.

People see someone who can disappear into a subject, teach themselves several disciplines, or integrate ideas across fields. They see the accumulated knowledge. They do not see the lost time, neglected obligations, disrupted sleep, difficulty switching tasks, uneven functioning, social costs, or years of compulsive repetition that may have produced it.

A person can deliberately cultivate polymathic capacities. They can study multiple disciplines, develop expertise, practise analogical transfer, learn to integrate frameworks, and organize their life around long-term interdisciplinary development.

They cannot guarantee that they will become an exceptional polymath. Training does not eliminate individual differences in cognitive ability, curiosity, motivation, energy, opportunity, working conditions, memory, or tolerance for prolonged uncertainty.

People who genuinely live with these patterns often speak about them less romantically. For some, the visible expertise came with costs they never consciously chose. They may not even have understood why they behaved this way until much later in life. Once they recognized the underlying pattern, years of uneven attention, intense interests, lost time, and unusual learning habits finally began to make sense.

EDIT: I am not a polymath. I am an autodidact. I have a propensity for over engineering my own frameworks to the point they become useless. That is inextricably tied to my own insecurities about credentials and legitimacy. I only have a grade 8 education😔so I have spent most of my life playing catch-up.

UPDATE:

I didn't make a clear distinction between ND and neurotypical polymaths.

This post mainly describes ND polymaths, more specifically, high-functioning ND phenotypes.

Shoutout to u/wdjm for the feedback🙏

MODS: If this is inappropriate for this sub, please do not hesitate to remove it. I am literally throwing a line into a pond, as it is something I have never done before. I suffered a catastrophic collapse in identity and personhood, and I have reason to believe it was a case of black magic. More specifically, a botched case of ritualistic sacrifice or assault done to me, but they pulled out at the last minute (no pun intended; some dark humor there 😅). Maybe they panicked. I do not know. That is why I am here. I am hoping somebody can tell me, more or less, what it was.

So before I start trauma‑dumping on people, I feel like some context is necessary for what follows. I was roughly between the ages of fourteen and sixteen when this happened. I am thirty‑nine now. At the time, I was heavily into drugs and roaming the streets. I met many interesting people, but one group stood out.

I live a normal life now. I am not on any meds, though I am on the spectrum. I have a job and two kids. I met my wife roughly four to six years after the events below. There is a certain irony in how I met her. I would later find out, after a few years of dating her, that she actually met one of the individuals (one of three people in question) online. This created some apprehension in me about our relationship and how we met, and that apprehension persists to this day, though it is less prevalent now. In case anybody is wondering, I met my wife through another friend totally unrelated to this group of people, though he does know of them. He always suspected there was something odd about these individuals in question. For a short while, I believed that my wife had paid these people for a love spell and that I was not the target, but rather what she got out of the roll. This hypothesis is very unlikely, but not impossible.

I have already wasted enough of your time, so let me just get straight to it.

There are three people: Bob, Sam, and Rea. These are made‑up names. I had known Bob for some time and had just met Sam and Rea before all this transpired. Rea was also Sam's older sister. One day, Bob invited me to a house‑sitting gig that Sam had been asked to do for his parents. I had bought some stuff for Bob and me to take while we were there.

We arrived. Both Bob and I took the stuff. Everything was going like normal, until I noticed Bob throwing up on the far side of the garden. I did not think anything of it. Another strange thing happened that I cannot seem to shake: when a particular song started playing on the radio, it reminded me of a painful memory I had about myself. Oddly, Rea reacted quite badly at the very moment I started thinking about what the song meant to me, almost as if she felt what I was feeling. She stormed off wailing, and I did not see her until the morning. Later, as the night progressed, I was offered a glass of water. I drank it, and then I blacked out. Absolutely zero memory of what happened during that blackout. All I remember is waking up hyperventilating, the smell of feces in the air, and both Bob and Sam sitting next to each other, straight‑facing forward, looking as if they had seen a ghost. Initially, I thought I had just had a bad trip, but they would tell me, right? At the very least, they would tease me as a lightweight or something. Nothing. I got absolutely nothing. It was a very eerie atmosphere.

I arrived at my house the next day, but something was off. It was like I was watching my own life through the eyes of somebody else. I walked into my mom's living room, where I lived at the time, and it seemed new to me, as if I had never seen it before. There was this strange echo of somebody else's voices saying things like, "This place is actually beautiful!" or "Wow, he lives like this." The moment I noticed it, the mood changed from excitement to fear. Thinking back on it retrospectively, it never changed back to excitement. It turns out whoever or whatever this was did not suspect that I would notice it.

During the six months of me living with this, I subsequently became a hermit, as I could not really contextualize what had happened to me during the house‑sitting incident, and thus I withdrew. In this time, I developed a relationship with this something. Once, the voice told me to kneel in front of a meter‑tall wall on my balcony. At the time I did not know it, but I was being told my future. I did not see actual images. I saw what I can only describe as what you would see if you watched a child using watercolor to describe what the child was seeing from the other side of a one‑way mirror. There was an image of a blond woman with blue eyes. She was pregnant. There was a rifle, a jet, and a few other things. These are the ones that stood out to me, though.

My wife grew up with golden blond hair and blue eyes. I have two kids. I had to join the mandatory military service for the country I immigrated to. Every image lines up nearly one to one. Something else that is straight out of a Stephen King book: there was "HELP" written in nail polish on the window on the outside, but written in reverse so that I could read it left to right from the other side. I lived on the third floor. I checked to see if my mom had done that, but she did not own purple nail polish.

There were many things that happened in between these events and so forth. Sam is deceased. He was murdered. I have reason to believe I may have caused that. There was a lot of negativity aimed at him. But the part that strikes me as confusing is their fear for me.

Does this look like an assault? It has been twenty years, so at this point the damage is done and taken care of. But memories still exist, like a pond that has not been disturbed for millennia. Every now and then I look into that pond, wondering why.

Anyway, there is much more that happened. I am open to answering questions.