MVU Game Maker preview - Auto Image gen/Multi LLM call/Deterministic D20
The following post was not written by AI, so it will have grammatical and spelling errors, and the post will be long, you have been warned.
# 💡 Current Status
It's been a while since I give an update on the MVU Game Maker. At the beginning, I was making the MVU Game Maker v1.5, which feature the Japanese School genre. The game should mimic the well known NSFW game from Illgames - Koikatsu. Of course I can't make it fully interactive like the real game, but it should have the same feeling in a visual novel style. It turns out, the game is way bigger than I anticipated. The game alone with all the logic in place cost 85k token/reply excluding events and image generation. I guess it will pass 100k token per reply, which is way too expensive.
That was almost 3 months ago, and I was back to drawing board after the MVU Game Maker v1.5 School Genre was ALREADY implemented. I need a way to selectively send what is needed to AI model, not the complete engine. Long story short, I implemented School Genre v1, v2, v3 and then finally v4...actually got that right.
The screenshot you saw in the post, it's the in game GUI which feature a galgame style interface. The image is auto generated based on story, and backend is comfyui with Anima or Krea 2 image model. I believe you will need at the minimum of 8GB ram of video card to get the image generation to work. That also means for all local model users with low end video card, you will have to choose if you want image generation , or running a model to write your story. The game works in plain sillytavern mode without image generation.
I ran through so many different iteration of School genre implementation because I found a fatal issue during development that plague the whole user base of English speaking Sillytavern users. For whatever reason, we have many Sillytavern like engine, and we have many more presets, but there are no content creator to create character cards. It's almost zero if you look at this reddit channel. Unlike other language Sillytavern community, they made thoudsands of character cards. And they are NOT just a few lorebook entries with a few alt greetings. All of those have game mechanics in the game.
# 💡 What is Game Mechanics?
AI model is trained to execute instruction. You throw a lorebook with some characters and background to AI model would not buy you any game mechanics, because there is no instruction to tell the AI model what , when and how to do. For example, You have a lorebook entry that is a warrior called Hero with 50 Str, 30 CON, 15 INT. The AI Model do NOT know how to translate these stats into damage. When you add weapon and armor on top of these stats, AI just randomly make up numbers when you hit a mob. So, reply 5 and reply 40 can have completely different damage with the same stats. Slice of life/dating game will also have the same problem. Bonding 30, Affection 40, Jealousy 35 have no meaning to AI model. It will randomly generate the behavior of the girl and her behavior and personality could be completely different from turn to turn. Does affection 45 allow you to kiss? AI doesn't know. Final fantasy have game mechanics, Dragon Quest have game mechanics. You know exactly how much you will damage with the weapon you are using to the same level mob EVERY single time. You know fire based magic works way better on human mob in Final fantasy. This is game mechanics.
That also point to another issue with tracking extension. There are many tracking extension out there, including preset also is doing that now. It will embed the state of every single character in the main text every single turn. It takes a lot of token when your party is big, and you can't really track too many stuff without messing up your chat. Moreover, tracking extension without game mechanics are next to useless for AI model. How much damage should I hit with 45 str? How does 58 Trust, 43 Comfort, and 60 Affection translate to her behavior? AI model just make it up, and it could be different every single time and they are wildly different.
Local model will have significant challenge as well because most users use local model that is good at prose, they usually suck at reasoning. I saw numerous report people have difficulty just tracking simple stuff in a preset with a local model. I guess we are getting to a point that you NEED API model to do anything other than prosing. A 27B model can stuff only so much information it can carry, there is a limit on what a local model can do on a regular video card. The worst reply on a GLM 5.2 is the best reply you can get on a Gemma 4.
And this is only a problem for English speaking Sillytavern community. As you can see this, this , this , this and this, which is created by Korean/Chinese Sillytavern community. Every single one of them have a proper GUI, stats tracking capability with game mechanics built into the game card, and they all play differently. Even as simple as character creation, it is way better than anything you can see in any English based card. You are REQUIRED to find out the game mechanics so that you can survive in the game card, just like how you do on a console game. That's why 99%+ of Chinese Sillytavern users use Gemini/Claude/GLM as their main AI model, because all these requirements need a model smart enough to follow instruction, not writing prose. Local model will just fail to run.
Another problem is that you can't make game content with a plain lorebook. Zero, none.
# 💡 What is Game Contents?
If you want to make a Konosuba game and want to play as the main cast to experience the same event and fight those boss in the same order, you can't do that with a plain lorebook. There is no way to track the state nor a way to trigger event. So, smaller AI model will become dry after 100 turns with 800 words reply or so because you will see it write the same thing over and over again. Even with big model, there is still NO content, AI just write in a way that the pattern "most likely" what will happen, which you have already seen that in another character card. It will not trigger event the way you want, let alone you want to fight the same boss in Konosuba. You can't rely on AI model to create game content, because AI model required you to give instruction. What, when, and how to create content. Otherwise, it will just do "Given everything before this point, what text is most likely/appropriate to come next?". This is how AI model works.
That's why so many people care about prose, only because there is nothing else you can tweak on. Characters card from other community have image, music, game contents, Character design, trigger events, very different game mechanics so that you do NOT need to focus on the prose only. You can focus on the content, like you will focus what content a game will offer in a console game. Game mechanics would not restrict how AI prose, it just provides a structure for AI to prose so that it will be consistent across the whole story.
If you manage to read this far, this is the message I want to deliver. We need game character card creator, to build events and game mechanics so that people could explore what the content creator offer. I am trying to build something to fill the gap.
# 🤖 MVU Game Engine v2.0
- **Logic engine** > non-emotional related calculation, anything that can be handle by calculation locally would be coded in javascript which save a LOT of token by not sending the instruction to AI model.
- **Multi LLM call** > you can have multiple LLM call before (PRE) the main call and multiple LLM call after (POST) the main call. You can create different lorebook entry + logic in pure javascript before or after main call to guide how main call to shape. That means, for example, you can create a "town NPC simulation" in a PRE call and main call will know NPC out of your main story is doing something.
- **Extension** > The Logic engine is installed in an extension that is still in development at about 90% done, which it can add game mechanics to a game character card. User can build RPG logic engine, love dating logic engine..etc. Activate it on a character card and it will make a card with game mechanics + stats tracking done in backend. Only required stats will be sent to AI model to process, not the complete roster of every single NPC you track. For regular user, you will just download whatever Logic Engine that match the genre of the character card you are playing and activate it.
- **Plugin engine** > which allow content creator to create addon functionality to the main Logic engine that is missing. (eg. a weather plugin for RPG logic engine)
- **Event engine** > which allow content creator to create events using a GUI in mvu status menu builder. No coding required. Based on condition to trigger event to hint main llm call to prose an event happen. (eg. When mainchar.level > 30 and quest.step2 = completed) > "A dragon roaming in a dungeon far east". Based on this design, you can pretty much code a complete story of Naruto. Basically you create a checkpoint with condition. When you hit a checkpoint and complete it, it will give you the next checkpoint. Between checkpoint A to B, you can still do whatever you want as you can now, but you have a objective to push the story forward. And this is open for anyone to make new events for a story using a provided GUI in status menu builder. Event engine is about SHARING.
- **Status Menu engine** > Content creator can create different status menu (GUI) for different logic engine. User can switch to different status menu in the extension.
- **DNA Harvest** > Auto harvest on character's DNA in a story so that auto image generation will draw the same guy with the same outlook everytime.
As you can see, all these can be as simple as creating events for a game card by non-coder, to a dev with Claude to create a complete game with their own coded logic engine. Since it's all modular design, engine can be plugged into a regular character card. **It's about SHARING and something in common that people can talk about the game, talk about tricks to beat the game mechanics, not a complete random generated story.**
The reason I didn't create yet another SillyTavern-like system because.... Sillytavern can just do the same thing. What is missing is the game mechanics and game contents and events, which no system can fill the gap. It's a human content creation thing. We need game character creator to make game card, this is the missing part. During design phase, I also looked into the possibility of using preset. And eventually using extension as the media and not a preset because I found that preset have a big limitation on what you can do with scripting. Even as simple as a true crypto D20 dice roll, it can't do it. No array, no collection and no looping can be a real challenge to pull off the Japanese school genre with event driven trigger and a lot of calculation behind the scene. And preset will take a lot of token to keep track of state because there is just no easy and stable way to do delta changes on stat manipulation, you have to submit the complete snapshot of state for every single character you track every single turn ,which is expensive. On top of that, multi language support is mandatory like all other extension i released.
I end up draw the line like this for the design of MVU Game Engine v2.0. Console = Sillytavern , Controller = preset, Console Game = character card, memory card = memory extension. I will just use preset to do prosing and turn off anything regarding logic inside the preset, and keep all the logic inside the game card, just like game play and logic is inside a console game cartridge. Sillytavern is the main console machine to compute the coding behind the scene. This is the final design of MVU Game Engine v2.0, and the first consumer is the Japanese School Genre...Logic Engine backport of previous RPG and Dating genre follows after. The token usage is around 35k token per reply with events and auto image generation. I would believe RPG genre Logic engine would be even less than this because all the logic are formula based and everything can be moved to javascript. Events doesn't take much token anyway, because logic engine only feed event to AI model when requirement is met. It won't feed 100+ events to the AI model every single turn.
No, I don't have the time line yet when I am going to release. The design become extremely complex when I have to take into account that the extension shouldn't need to upgrade to adapt to another genre Logic Engine. The reason I am posting this is to give an update...and call to arms. We need game content creator. The new extension **"MVU-Helper"** just provide a framework, make it accessible to community user to SHARE contents and talk about the game contents...not some random generated garbage. AI will be used to prose what the content creator want to do, not the *only* member to design how the content should flow.
