u/badhiyahai

https://preview.redd.it/gopkyaw60ymh1.png?width=1066&format=png&auto=webp&s=df442eb69f32dc045c77ce0ccfb0156adb45cb33

Every year like clock work these f**king juvenile kids and sometimes adults from nearby slums come demanding for ganpati donation.

Demanding, not politely asking, or requesting. The moment you say no, they get aggressive. The money is not the matter, the f**king arrogance as if it's their to demand money. That filthy little pad gives them the right somehow.

Last time I had to call the cops. Same scene again today, almost called the cops but this time they were juveniles and I left it.

What problem are we solving?

Let us start with a problem statement,

>You have to deploy a public facing chatbot, the bot is supposed to be capable of executing various tools, for example ffmpeg.

Now the simplest solution is to just host an app on your server which simply accepts a prompt, decides which tools to call, executes the code on the server itself & returns the output.

   User/Frontend            Your Server                                                                                                              ┃
   +-----------+        +-----------------------+                                                                                                    ┃
   |           |  --->  | LLM picks tool        |                                                                                                    ┃
   |  Prompt > |        |   -> runs code        |                                                                                                    ┃
   |           |  <---  |   -> tools call       |                                                                                                    ┃
   +-----------+        +-----------------------+

Non air-gapped (one machine does everything):

This is an inherently problematic approach. Take an example, an app which lets you run ffmpeg command using a text prompt. User enters "delete the lib ffmpeg"

In the above solution it will eventually run the instructed command no matter how robust the system instruction is. Final result, all the users are affected.

So how do we actually solve it?

The design which we came up with was to have prompts evaluated by OpenAI on a system where we never execute the code. The code is then sent to another machine where ffmpeg is installed, the code generated by the LLM is executed here. Even if the code is malicious it only affects that particular user's ephemeral machine.

   User/Frontend        Persistent Server         Ephemeral                                                                                          ┃
   +-----------+        +---------------+       +--------------+                                                                                     ┃
   |           |  --->  |    Prompt     |  ---> |              |                                                                                     ┃
   |  Prompt > |        |       |       |       | Execute Code |                                                                                     ┃
   |           |  <---  |       v       |  <--- |              |                                                                                     ┃
   +-----------+        |     Code      |       +--------------+                                                                                     ┃
                        +---------------+

Anthropic has also come up with a similar model with managed agents, although they don't explicitly call it as such.

Bonus section

If someone noticed, you might be thinking that what if the prompt injection asks for "Give all environment secrets" in the Persistent Server - wouldn't a successful prompt injection leak our OpenAI secret (used from Persistent Server to generate ffmpeg code)?

Great question! for this reason, we never store OpenAI or any such key in the Persistent Server, it gets injected on the fly by a proxy from a separate vault which contains those keys.

I checked with chatgpt and it said 0.1 to 1% but wanted to know from someone who has actually seen these numbers.

It might vary from product to product so if you could mention the type of product it was. (Do not link/name it, self promotion rules and all)

I checked with ai tools and it says around 0.1 to 1% .. I need to know it from someone who has had the experience and saw the actual numbers. I know it can vary for the type of product.. so if you could mention your product type it would be great, like was it a saas. Don't link your product or name.

And it was the gpt 5.6 Sol. We got habituated to `skip permissions dangerously` and `yolo` and are now playing the victim card.

He blamed the model.

How are you guys navigating this `rm -rf` possibility?

Edit: thanks for all the suggestions, seems like sandboxing is the clear way forward. Will check out instavm