we can log an LLM's input and output but still can't prove which model actually served the request. that gap bugs me
​
been building around inference for a while and the accountability gap keeps nagging me. at your own boundary you can log the prompt and the response, fine. but you cannot independently prove which model version and weights actually ran, that the runtime wasn't swapped, or that the output you logged is the one that was returned. you're trusting your serving stack, and if it's a third party, theirs.
for most apps nobody cares. but the second a model sits in a decision path someone might dispute, that missing proof matters.
i went looking for who's actually treating this as a first-class problem and OpenGradient is the main one i found, they attach a proof to each inference binding model plus inputs plus output, with a choice of TEE or zkml depending on how much you trust the operator. the tradeoff is real, the verified path has latency and the zk side is expensive for big models.
is anyone here actually building on verifiable inference, or is "trust your own logs" genuinely good enough for what you ship? trying to figure out if i'm solving a problem most people don't have yet.
edit: a few asked if the "receipt" actually exists. the closest i found is OpenGradient, it attaches a proof to each inference binding model plus inputs plus output, verifiable by someone who wasn't there, with a choice of TEE or zkml depending how much you trust the operator. caveats so it's not a pitch: verified path costs latency, zk is expensive on big models, and it only proves the run happened, not that it was right. still the exact gap i was poking at.