Behind the Chatbot
TL;DR: Built an interactive tool that teaches students how LLMs actually work by having them talk to one. It's grounded in information literacy pedagogy but hasn't gotten much traction in library-world yet — curious what an AI education audience thinks of the mechanics and approach.
Hi all — I'm a community college librarian, and I built an interactive AI literacy activity called Behind the Chatbot. Students have a guided 1-on-1 conversation that walks them through how LLMs actually work: next-word prediction, training data, tokenization, embeddings, attention and context windows, temperature, RLHF, reward hacking, algorithmic bias, and hallucination — one concept at a time, each building on the last.
I built it specifically because I think understanding how an AI's output gets made is part of learning to evaluate it — the same way understanding how a news article or a study gets produced changes how critically you can read it. That framing comes out of information literacy pedagogy, but honestly, it's been a hard sell in library-world so far — a lot of librarians are (understandably) wary of AI on labor, environmental, and epistemic grounds, so tools like this haven't gotten a ton of traction or attention in that space yet. I'm hoping an audience more focused on AI education specifically might have a different, useful perspective on it.
What I'd love feedback on:
- The mechanics it covers — anything missing, unnecessary, or in the wrong order?
- The lecture-style content itself — anywhere it felt unclear, too dense, or too hand-wavy?
- Where you got stuck or confused going through it
- Ideas for follow-up activities that could build on this (I'm especially interested in whether a prompt-engineering module makes sense as a next step for multi-session settings)
Try the live tool here: https://claude.ai/public/artifacts/b238f07b-6f32-468f-a7c4-6688ac529b45
The tool and lesson slides are posted on the ACRL Sandbox (that's all that's up right now, more to come): https://sandbox.acrl.org/resources/behind-chatbot-interactive-ai-mechanics
I'm presenting this as a poster at the Illinois Library Association conference in October and would love feedback before then. Happy to answer questions in the comments.