u/Aromatic_Charge822

There was a survey out of the UK last week, ACI Worldwide asked 2000 adults about AI shopping assistants. The numbers are brutal. 60% said one mistake and they stop using the tool forever. Only 19% trust AI to make routine buying decisions. 70% said if the AI bought something without asking first they would walk. And 44% said they would not trust an AI shopping assistant no matter how much money it saved them.

These numbers are about a specific use case, AI shopping, but the pattern is the same across every consumer AI product. One bad experience and the trust is gone. Not temporarily lost, gone. And the AI industry is not built to handle this.

The problem is not the obvious mistake. If an AI shopping assistant tells you a toaster costs three dollars, you laugh and move on. The problem is the mistake that looks right. The assistant that confidently tells you this is the best deal, compares three products with plausible numbers, reads like a competent human wrote it, and it is wrong. You buy the thing, you find out later you overpaid, and you never trust the assistant again. This is the failure mode that burns trust permanently, and it is the one the industry is optimized to produce.

There is a term for this now, pseudo correctness. An answer that passes every check the system can run on itself, reads as competent, stays internally consistent, and is still wrong. Stumbled on it in a writeup about the apodex release, they named it and built their whole verification architecture around catching it. The insight is that asking the model to check its own work harder does not help, because the same blind spot that produced the error is doing the checking. You need a separate system that did not produce the answer to verify it.

The trust crisis is not just about shopping assistants. It is about every product where AI is the interface and the user cannot verify the output themselves. Medical advice, legal guidance, financial planning, news summaries. The pattern is the same. User tries it, gets a confident wrong answer, acts on it, gets burned, never comes back. The industry is burning through its user base one mistake at a time and the churn is invisible because the user growth numbers are still going up.

The way out is not to make the model hallucinate less. That is a moving target and the model is always improving and the next version will still be confidently wrong sometimes. The way out is to build verification into the product itself. Separate the thing that generates the answer from the thing that checks it. Show the user the evidence. Tell them where the sources disagree. Make the confidence transparent instead of hiding it behind a polished paragraph.

A few companies are already moving in this direction, some research platforms are putting independent verification at the architecture level. But most consumer AI products are still just a text box with a beautiful output. The trust crisis is coming and the ones that survive it will be the ones that treat verification as a product feature, not a training problem.

I work in logistics planning. A couple weeks back my team was evaluating a freight forwarding platform for a sales review, and I had my research agent summarize the provider's EU compliance features.

The summary it gave back looked solid. One feature jumped out, something like a "dynamic customs filing router" that supposedly automated tax declarations on the fly. I pasted it into the deck without checking.

During the review, our VP stops on that slide. He had just left that shipping company the year before. He said the feature was never built, it was a brainstorm idea from 2023 that got killed because of customs rules.

I wanted to melt into the floor. I spent the rest of the day sending correction emails and reworking the deck.

The part that still bugs me is that the agent wasn't even guessing wrong in an obvious way. It was wrong in a very specific, very believable way. And when I asked it to verify, it just doubled down because it was reading its own previous output. Same model, same prompt window, same blind spots.

I have been rebuilding the workflow since then. The rule I landed on is simple: the thing that writes the answer cannot also be the thing that checks it. If you want a real check, it has to be a separate run, ideally with separate context, separate search, and no memory of the first answer. A friend mentioned a new agent verification tool called Apodex that is built around this split, but honestly you could do the same thing yourself with two different model calls and some web search.

The embarrassment alone is enough to make me verify everything manually from now on.

Got burned twice this week so I am collecting war stories.

First one was a model swearing a specific tax form existed in my country with a very specific form number and a filing deadline. Spent an afternoon chasing it. The form does not exist, the number belongs to something unrelated, and the deadline was invented whole. It said it with the same tone it uses to tell me the sky is blue.

Second was a research summary that cited three papers. Two were real but said the opposite of what was claimed, one was just made up, looked more legit than the real ones honestly. Cleaner abstract, more confident conclusion, perfect formatting. If I had been in a hurry I would have cited it in something I sent to a client.

What unsettles me is not that they lie. It is that the lies are structurally indistinguishable from the truth from the inside of the model. There is no hesitation, no hedging, no tell. The confident wrong answer and the confident right answer come out in the exact same voice. Asking it to double check just produces a second confident answer in the same voice. The only thing that has actually caught these for me is running the claim through a separate pass that does not share the model's reasoning. You cannot let the thing that lied also be the thing that confirms it did not lie.

So what is yours. The one where you almost believed it, or the one where you did believe it for too long.

One thing that took me way too long to internalize while learning ML: telling a model to "double check your work" or "verify your answer" does basically nothing on the mistakes you actually care about. I kept adding those lines to my prompts and assuming the model was now being careful. It was not. Figuring out why was more useful than any prompt tweak I made.

The cleanest way I found to think about it: imagine you wrote a math proof and made a mistake on step 3. Now imagine the only person allowed to grade that proof is you, reading it again. You are the same brain that already thought step 3 was fine the first time. There is no reason the second read catches what the first read missed. The error and the inability to see the error come from the same place.

That is essentially what self check does on an LLM. The checker is the same model that wrote the answer, so it walks in with the same blind spot already baked in. It re reads its own output, finds it all internally consistent, and comes back more confident than before. I would read those transcripts and the model would literally restate its reasoning and conclude "this is correct" without ever touching the questionable claim.

The part that made it click: the mistakes that actually bite are not random typos, they are structural. A citation that does not exist but is formatted more cleanly than the real ones. A fact that is wrong in a way that looks right. From inside the model there is no detectable difference between the confident wrong answer and the confident right answer, they come out of the same machinery in the same voice. So a process that only re reads that output has no signal to work with.

What actually catches these is a different process looking at the claim with fresh sources, not the same model re reading. Some of the newer research systems are explicitly built with the checker as a separate team from the writer, apodex is the one I see referenced most for that split, but the principle is older than any specific tool. You cannot discover what you cannot verify, and you cannot verify yourself.

The takeaway I keep coming back to: self check is not a weaker version of verification, it is a different category of thing entirely. If the goal is catching the structural errors, the verifier has to be independent of the generator, full stop. Everything else is the model agreeing with itself louder.

Curious whether others here ran into the same wall, and if you found the independence has to be architectural (separate model, separate context) or whether a disciplined prompt level separation on one model gets you most of the way. I have only really seen it work the first way but my sample is small.

Reading through the apodex 1.0 report what I want to discuss is not the leaderboard, it is one training and inference idea that I cannot decide is novel or just well packaged. They describe a generate verify revise loop. The model writes a candidate. A grader, which is the same model handed only the problem statement and that candidate, with the reference solution and any rubric deliberately withheld, scores it on a small scale and writes a short critique of where it is weakest. A new attempt is then conditioned on the previous attempt plus that critique. Repeat for a fixed number of rounds, submit the highest scored one. Base is a Qwen3.5 checkpoint, and they report this helps most on tasks like proofs where one bad step invalidates everything.

My first reaction was that this is best of n with extra steps. You sample candidates, you score them, you keep the best, and a learned scorer standing in for a reward model is not new. But the part that is at least structurally different is that the attempts are not independent. In best of n the samples are iid given the prompt. Here attempt k is explicitly conditioned on the written critique of attempt k minus one, so it is sequential refinement rather than parallel sampling. Whether that buys you anything over a good reward model plus beam or plus iterative correction is the actual question, and the report does not give me a clean ablation that isolates the conditioning from the extra compute.

The next thing I keep snagging on is the independence claim. The grader shares weights with the generator, so on any problem the model is systematically wrong about, the grade should be wrong in a correlated way and the loop should be uninformative or actively misleading. Yet they report real gains on the hard sets, roughly a doubling on a proof benchmark suite and a larger jump on the hardest proof subset, with no oracle in the loop. If that holds, the lift has to be coming from something other than the grader having independent signal. My best guess is the critique format forces a different decomposition of the problem on each pass, so you are getting diversity that ordinary resampling at temperature does not, and the scoring is mostly doing selection. That is a more modest claim than no answer key needed, and I would want it stated that way.

Two things would settle it for me. A compute matched best of n baseline on the same checkpoint, same total tokens, where the only difference is whether attempts are conditioned on the prior critique. And an analysis of how often the self grade is actually correct on problems the model gets wrong, because if the grader cannot tell good from bad exactly when it matters, the whole thing reduces to expensive resampling with a confident sorter on top. If someone has already pulled those numbers out of the report or run the matched baseline themselves, I would rather read that than keep speculating. The implementation and eval scripts are in their harness repo if anyone wants to look at the loop directly rather than the blog summary.

This is a weird one and not a model rec in the usual sense, so bear with me. My problem on long campaigns has never really been prose quality, the usual mid size models are fine at that. It is continuity. Fifty messages deep the model forgets an NPC died, swaps a character's eye color, or has someone reference an event that the lorebook says has not happened yet. I swipe, I fix it by hand, I edit the context, and it creeps back in because the model writing the next reply is too busy being in the scene to audit the scene.

What I have been messing with the last week or so is keeping a second, tiny model running purely as a continuity checker, not as the RP model at all. After a reply lands, a small script feeds the checker the character card, the relevant lorebook entries, and the last chunk of the chat, and asks one thing, does anything in this latest reply contradict the established facts, list it, do not rewrite anything. It does not generate story, it does not get a vote on the writing, it just flags. The reason a separate model helps is the same reason a fresh pair of eyes catches your typos, it did not write the reply so it is not invested in the reply being right, where the main model re reading its own output just nods along.

For the checker I have been using one of the small open apodex weights, the little SFT ones in the 0.8B to 4B range that went up recently. Full disclosure on what they are, they are not roleplay tuned at all, they come from a research agent family built around the idea of a separate part of the system checking the answer instead of the model trusting its own pass, which is exactly the property I wanted for this job even though the intended use is nothing like RP. They are useless for actually writing a scene, do not try, but for the narrow does X contradict Y question they punch above their size and they are small enough to sit next to a real RP model without eating all your VRAM. I run the 4B for this, the 0.8B is faster but misses subtler stuff.

It is duct tape, not a feature, and it adds a beat of latency after each reply so I only leave it on for the long serious campaigns where continuity actually matters. For quick one offs it is pointless. But it has caught real contradictions before they snowballed into me having to retcon three messages, which on a fifty plus message thread is worth the hassle. If anyone has a cleaner setup for this than my hacky script I would take it, the checker idea is sound but my plumbing is held together with hope.

Been on Swagbucks for years, mostly grinding surveys out of habit, and I finally bothered to compare what I make from surveys against what I make from the shopping cashback side. The result was not what I expected.

Surveys, once I actually tracked it, were grim. Between the screeners that boot you four minutes in and the ones that pay 40 SB for fifteen minutes of questions, my real rate was maybe two to three dollars an hour on a good day. I had been telling myself it was better than that for a long time.

The shopping cashback on the other hand takes basically zero extra time, because I am buying the stuff regardless. The absolute amounts are smaller and lumpy, some months nothing at all, but per minute of actual effort it is not even close. I just had it filed in my head as the boring part of the app and ignored it.

For online orders I run a couple of coupon extensions alongside the SB button, Coupert being one of them. The low first cashout threshold meant I could pull a payout early instead of watching a balance crawl forever. Small detail but it mattered to me.

Anyway the surprise was that the part of the app I ignored for years was quietly the efficient one, and the surveys I actually grind are the real time sink. Am I going to stop doing surveys while half watching TV? No. But at least now I know they are the hobby and not the income.

Ich poste das, weil mir vor knapp zwei Monaten genau diese Erfahrung gefehlt hat. Alles was ich online gefunden habe, war entweder rechtliche Theorie oder Werbung. Das Gespräch selber findet man nirgendwo nüchtern beschrieben. Kurz zur Lage: Frankfurt, 3. Stock, Altbau mit normaler Stahlreling am Balkon. Vermieter ist eine kleinere Hausverwaltung, kein anonymer Großkonzern, aber auch nicht der nette Eigentümer aus dem Nachbarhaus. Ich miete seit knapp drei Jahren, war noch nie ein Problem, aber Balkonsolar war neu für die Verwaltung und das hat sich im Erstkontakt deutlich gemerkt.

Was ich vorher vorbereitet habe und was davon wirklich relevant war. Fotos vom Balkon und Reling mit Maßband sichtbar, das war eines der wenigen Dinge, das wirklich Wirkung hatte, weil die Verwaltung die Balkonbreite nicht kannte und ich sie hatte. Eine handschriftliche Skizze auf einem A4 Blatt, wo die Module sitzen, wo das Gerät steht, wohin das Kabel läuft. Hat die Verwaltung freundlicher gemacht als jeder Hochglanz Plan, weil es nicht nach Profi Renovierung aussah. Und das Datenblatt vom Gerät ausgedruckt, bei mir war es ein Jackery SolarVault 3 Pro. Was wirklich zählt: außentauglich, Steckdosenanschluss im normalen Balkonbetrieb, kein Bohren und kein Umbau an der Hausinstallation. Das musste ich der Verwaltung Schritt für Schritt erklären, weil sie zuerst dachten, ich rede von einer richtigen PV Anlage mit Wechselrichter im Keller. Diese Verwechslung war das größte Hindernis im Gespräch, nicht irgendeine Detailfrage.

Was gut ankam. Erstens die Zusage, dass ich keine Löcher in die Fassade oder Reling bohre, Klemmhalterungen mit Gummiunterlage, das war für die Verwaltung der wichtigste Punkt überhaupt. Zweitens eine zweite Sicherung pro Modul, Stahlseil zur Reling, so dass selbst bei einer brechenden Klemme nichts vom Balkon fällt. Das hat die Verwaltung nicht aktiv gefordert, aber der Hinweis hat das Gespräch sofort entspannter gemacht. Offensichtlich hatten sie Sorge, dass etwas auf den Gehweg fallen könnte, und ich habe diese Sorge weggenommen, bevor sie ausgesprochen wurde. Drittens die schriftliche Zusage, dass ich bei Auszug alles ohne sichtbare Spuren rückbaue.

Was keine Rolle spielte: die lange Erklärung zur Solar Package 1 Regelung und 800 W Einspeisung hat die Verwaltung einfach nicht interessiert. Für sie war relevant, ob ich ihre Substanz beschädige, nicht ob mein Setup gesetzeskonform ist. Energieersparnis Rechnungen waren komplett uninteressant. Was unerwartet schwierig war: die Frage nach Kondenswasser an der Hauswand. Die Verwaltung wollte wissen, ob das Gerät am Boden Feuchtigkeit unter dem Balkonbelag staut, weil das eine bestehende Sorge war. Ich konnte spontan keine gute Antwort geben. Mein Setup steht inzwischen erhöht auf zwei Steinplatten mit Luft drunter, genau aus dem Grund. Die Frage war für die Verwaltung der zweitwichtigste Punkt nach den Klemmen.

Sechs Wochen nach der Montage: keine Beschwerden, keine Folgegespräche, keine Probleme. Ich habe der Verwaltung einmal von selbst eine kurze Mail mit zwei Fotos vom installierten Stand geschickt, einfach um den Spannungsbogen abzuschließen. Das fanden sie gut. Was ich anderen Mietern raten würde, ohne Anwaltston: geh nicht mit der Frage rein, ob du Solarpanels haben darfst. Geh mit einem fertig durchdachten Plan rein, der zeigt, dass du an der Substanz nichts machst und dass du beim Auszug alles spurlos zurückbaust. Die juristische Situation in Deutschland ist mittlerweile ziemlich mieterfreundlich für Plug in Solar, aber ein Vermieter Gespräch ist kein Gerichtssaal, da geht es um Vertrauen und konkrete Sorgen, nicht um Paragrafen. Wer mit dem Gesetz im ersten Satz kommt, macht sich das Leben schwerer als nötig.

Frage in die Runde: wer hat im letzten Jahr ähnliche Gespräche geführt und welcher Punkt war für eure Verwaltung der entscheidende? Mich würde besonders interessieren, ob bei Großverwaltern vs kleinen Eigentümern andere Argumente zählen. Bei mir war ganz klar die rückbaufreundliche Montage entscheidend, aber ich vermute bei einem privaten Vermieter wäre die Optik wichtiger gewesen.

[ Removed by Reddit on account of violating the content policy. ]