Moving prompts out of three services finally made rollbacks easy
Our agent prompt had grown across multiple services. Each service owned a reasonable fragment at first. Over time they accumulated different defaults, tool descriptions, safety language and model parameters. Staging tested one combination. Production could render another depending on which service handled the request. Debugging prompt behavior became archaeology with deployment manifests.
We moved the shared logic into a prompt registry with immutable versions. A candidate prompt now gets one explicit ID, runs against a fixed dataset and moves through staging and production through environment promotion. The services reference the chosen version and attach that prompt ID to trace metadata.
We've been using Braintrust for prompt management and evaluation for the registry, experiment comparison, and production trace. Now when a new instruction increased toolcall failures, we could compare it against the previous version and roll back the environment pointer without rebuilding.
There is still normal operational work. Access control matters. Prompt changes need review. Cached versions need clear invalidation behavior. But the rollback is now a small, observable configuration change instead of a coordinated deploy.
Has anyone found a clean way to keep prompt ownership flexible while making version promotion as disciplined as application releases?