u/elsecrafter

I’m building a text RPG with an LLM narrator agent, which was having ~15-second cold starts in some cases.

I recently had the chance to dive deep into the root causes, and it was not what I was expecting. I thought the issue may be related to the model providers I was calling (e.g., perhaps a lack of caching on their end).

However, it turned out to be two fairly unexpected issues:
- Firstly, I was using tiktoken to estimate the number of tokens in the input to my agents. Only after putting a timer around almost everything in my code did I realize that loading the encoding (e.g., o200k_base) was taking over 5 seconds.
- Then, I realized that my AWS Lambda function only had 256 MB of memory. I thought this was sufficient, since it was never getting close to that limit. However, it turns out that Lambda CPU scales with memory and the lack of CPU power was significantly slowing down my initial ai client call. Increasing the memory lowered the narrator agent cold start by several seconds.

Anyway, the main lesson I took away is that latency analysis tends to reveal surprising results (at least for me), so I find it can be useful to really time everything before theorizing too confidently. I’ll include the link to my web app in a comment below, so let me know if the latency is feeling reasonable, or if there are any other issues. Since it’s still in an experimental / development phase, it’s currently free and doesn’t require sign up to get started.

I’ve been working on an interactive sandbox that allows you to role-play in worlds you build. I’ve found that it’s surprisingly difficult to get it right. For example, trying to keep model costs down while still having enough context and keeping the world state consistent is an ongoing challenge that I’m not sure anyone has fully solved.

That said, I’m curious if this is something that people would find useful and what would be the most important challenges for me to focus on. I’ll comment the link for anyone to try if interested. I’m still ironing out the bugs and looking for feedback — it’s currently free and doesn’t require sign up to get started.

Fundamentally, most action resolution mechanics seem to involve estimating the success probability of an action (e.g., comparing skill/ability level vs difficultly level) and rolling dice. In some games like Ironsworn, you roll multiple dice to get a ternary outcome space (strong hit, weak hit, miss), which I find can be more engaging than a binary outcome space in many cases.

I’ve been experimentally working on a text-based rpg web app and am trying to land on a great action resolution mechanic, looking to ttrpgs for inspiration. I’m curious what your favorite action resolution mechanic is. For example, have you found something that feels particularly realistic? Or something especially creative?

In the latest release, Elsecraft lets you build worlds for role-playing stories.

I’ve played a number of rpg video games like skyrim and the witcher. Often, my favorite aspect is to just forget about the main quest and install a bunch of mods to create my own side stories, so I feel like I might enjoy the freedom of solo ttrpgs.

What’s a good place to start?