I gave my open-source agent shell access and the ability to rewrite its own skills. Here's the governance kernel that keeps it from doing something catastrophic.
The most-upvoted posts here lately are some version of "your AI agent is already compromised and you don't even know it." Fair. An agent that runs shell commands, calls tools, and rewrites its own skills is one bad token away from rm -rf, leaking a key, or corrupting your repo.
Chimera (open-source, Apache-2.0) does exactly those scary things — so "what stops it from doing something catastrophic?" had to be answered before anything else. The actual design (all in the repo, not a promise):
- Every action passes a trust kernel -> allow / warn / review / block.
- Deterministic lexical rules catch fixed-signature threats: rm -rf /, mkfs / dd to a device, fork bombs, chmod -R 777 /, curl ... | bash (-> human review), sudo rm (-> warn), and secret patterns (sk-..., AKIA..., PRIVATE KEY blocks -> warn).
- An optional semantic judge handles the intent-dependent cases a regex can't.
- Invariant: a benign action is NEVER hard-blocked. No rule + no judge = allow. A safety layer that blocks normal work is worse than useless.
The kernel gets cheaper over time. Repeated judge verdicts are distilled into cheap lexical rules, so the expensive semantic check runs less as it learns the threat surface.
Self-modification is gated, not free. The agent can't just rewrite arbitrary files. Skill/schedule changes pass a static validator (a constrained edit surface) and are rejected pre-execution if invalid — then it's verify-or-revert: test the change, keep it only if it passes, roll back otherwise.
Everything is audited (append-only log), so you can see what it did and why.
Honest limits:
- Defense-in-depth, not a force field. Prompt-injection that talks the model into a "benign-looking" harmful action is the hard, unsolved part — lexical rules catch signatures, the judge catches some intent, neither is perfect.
- It's alpha. 540+ tests, strict typing/linting on every change, but not battle-hardened in production yet.
If you build agents, I genuinely want to be attacked: reply with the command or self-edit you think slips past this and I'll show you where it's caught (or fix it if it isn't). Repo link + the exact rule set/kernel are in a comment below (this sub's rule 3). Apache-2.0.