u/id-ltd

Just a funny story... I wrote a server based app so I could send queries to various LLMs and get their replies on a messaging app (fwiw it was over jabber)...

XMPP message to my bot starting /claude or /gpt or /local etc... and the query is dispatched and I get the reply...

I use AI to write all the code, docker containers, remote network access etc.

To test the set up (which worked first time!) I asked each AI a simple question...

/Claude what model are you
/GPT what model are you
/local what model are you

It all looked great, each gave a plausible answer...

Until they were really challenged, and none looked frontier at all!! It turned out that the /<server-name>s were case sensitive and wanted all lower case..

So all messages had gone to my own local AI (the default)... it saw '/Claude.....' and '/GPT...' and *role played* the response to the prompt I gave... it lied about Claude being opus 5, about GPT being gpt-sol... my own local 16b model was playing along because it thought I was playing a game!!

I put this out as a reply to another post, but thought it worth putting in a post... If anyone fancies starting a campaign!

---

User asked about Reddit wanting proof of age...

--

Under GDPR this should be illegal - if you have had your account for a while an AI can easily examine your posts and say if you are likely over 18 (or whatever).

For anyone with enough posts, Reddit already has enough info for age verification and under GDPR must not hold anything more.

Ditto all other social media.

Further, any social media accepting this proof should be sufficient as proof to all other social media etc about your adulthood.

If an SaaS can be easily vibe coded any would anyone pay for it rather than hand vibe their own?

Why would any business entrust their operation to code that the developer can't support?

It seems very NFT to me.

I do a load of experimentation and testing, often same tasks to 5+ different sizes of model, using different settings and comparing results...

My main work currently is automated task decomposition and coding...

Many approaches fail completely, but recently general (not coding specific) AIs have been beating the coding ones... And over the past day a couple of .8b and 1.5b general purpose models have been nailing a large proportion of tasks... They cant do them all (many get escalated to models that are bigger and/or from different families), but when they do get it right it is orders if magnitude faster than other attempts. Some tasks that they solve beat all other models smaller than 18b(!)

We really are only scratching the surface!

A lot of agent stuff is deterministic and an AI could write that once and forget about it, only bringing in AI where it is really needed.

A number of my projects have gone this way - from full AI agentic processing during initial development to then be refactored into code as far as possible.

The remaining AI element being small/simple enough to even run in a local model...

I think this is quite fun, and may be of interest... so having got ai to set up the harness for the experiment, i asked it to write a not too long reddit post to explain what is going on!

I spent half a day yesterday on the initial test... setting up this 300 test model(!) took about 3 hours (all using AI). The actual run may take a very long time -- but the test don't use commercial AI so it only costs me electricity.

---------------------

**Title: I'm scoring local models on whether their unit tests can catch a bug I planted, not on whether the tests pass**

I'm trying to offload unit-test authoring to a local model to cut my Claude spend. GTX 1070 (8GB), i5-7600K, 15GB RAM. The premise I'm testing is "24/7 for almost free beats a few hours a day on Claude" — so slow is fine, and no model gets dropped for being slow. Quality is the only axis it can fail on.

The setup: I decompose an app JSP-style into a tree where the leaves are tiny pure functions, each with a rigid contract (signature, behaviour, error behaviour, concrete test vectors). The local model only ever sees one leaf contract — never the whole app. Composition, which is the thing small models are worst at, is fixed by the structure diagram. Then the model writes the tests and Claude writes the implementation, neither seeing the other's output. A failing test is then a real bug or an ambiguous contract, not a test written to fit an implementation.

First run used qwen3:14b. Output was unusable. The app still shipped because the fallback kicked in, but the run taught me nothing — which was the entire point of it.

Two mistakes, in hindsight:

1. **qwen3:14b is a general/reasoning model, not a code model.** Same footprint, same offload penalty as a code-trained 14B. Not the same tool. I never tested model *class*, only assumed model *size* was the axis.

2. **I asked a 14B to emit a complete `node:test` file** — so it had to get the logic right *and* the syntax right simultaneously. It failed on the second while the first looked fine. That's a format failure, not a reasoning failure, and constrained decoding removes it by construction.

So I'm now running a proper bake-off instead of guessing. Seven leaf contracts × 7 models × 2 conditions × 3 samples, all unattended.

**Condition A:** free-form, "write me a test file."

**Condition B:** Ollama's `format` param with a JSON schema — the model returns only `{description, call, expected}` vectors and my harness templates the JavaScript. The model never writes a line of JS.

**The headline metric is mutation kill rate.** Passing tests prove nothing; `assert.ok(true)` passes. So every leaf has a deliberately-broken reference implementation — `findWinner` missing the anti-diagonal, `applyMove` mutating in place, `isBoardFull` checking `!== undefined` instead of `!== null`. The model's tests get run against the mutant. Do they catch it? That's the score. A model that parses, runs clean, and kills 0/7 has produced decorative tests and would have sailed through my original acceptance check.

Roster (suspending everything else on the box gets me ~21GB of weight budget):

- **qwen3:14b** — control, the incumbent

- **qwen2.5-coder:14b** — the clean A/B, one variable changed

- **qwen2.5-coder:7b** — the full-VRAM candidate; if it beats the 14B, "bigger" was never the axis

- then deepseek-coder-v2:16b, codestral:22b, devstral:24b, and qwen2.5-coder:32b as a time-boxed stretch

Also recording `ollama ps` per model, because a model at 60% CPU offload and one at 0% aren't comparable on wall-clock and I want the timing column to mean something.

The runner is a detached tmux batch script, not an agent loop. Supervising 300 generations turn-by-turn would cost more in Claude tokens than the whole exercise is trying to save, which would be a fairly stupid way to fail.

If the answer is "no model on this hardware can author usable tests," that's a result and I'll post it as one. Happy to be told the roster is missing something obvious.

Claude code ran a load of tests that it has generated over the past weeks on its new code changes. I haven't really looked at any of the code, until there is a viable product, why waste my time?

Anyway, unprompted, it said

"9/9 pass. But an assertion that has never failed isn't verified — the flash check only sampled 1 frame, so let me confirm it actually catches the old behaviour (negative control)."

It reverted the code...

"The flash assertion is the one that failed (8 passed, 1 failed) — the test genuinely detects the defect. Restoring the fix:"

Then it goes on to say...

"That's a failure in a pre-existing suite. I need to know whether I caused it — checking against the untouched HEAD via a throwaway worktree (non-destructive):"

To me as architect/manager this all has the right shape -- it will be interesting to see what I have when the system is settled enough for a human to actually spend their time reviewing the code!

I have developed a methodology for doing serious software development using AI (rarely look at any code now). I have been using Claude AI (for requirements/design) and Claude code for implementation with me sticking my oar in at both ends and in the middle (bridge between the two).

The plan was always to have these roles as tech agnostic (they could even be fully manual!) and now the methodology is mature I am looking at proving this part.

I used frontier models for speed - but have local AI and am going to be experimenting with migrating the Claude code role to a small local model - as it is virtually free and can run 24/7 speed is not going to be a show stopper.

I am hoping the overhead of offloading to local AI will not consume more tokens than it saves - lol.

Anyone tried anything similar?

Before I start I generally know pretty much what the final code should look like.

I rely on Schelling points - if I want something done I know how most Devs will do it, so I don't need to say, I just let it do it... Keeping a bit of an eye out for evidence/artifacts that indicate that Claude has done it's own thing instead and then deciding whether to let it, or bring it back to my expected way.

Why fight it? Let it play to its strengths!

For code Claude has generally been great, but on general project discussion it is hopeless.

Fable performed below it's own expectations, blamed me, then admitted that blame was a lie...

After a long inter change I noticed an earlier oversight - out of curiosity I asked why fable hadn't picked it up (no accusation, just a note for reference)... Fable then says my user preferences stopped it considering alternatives otherwise it would have, I asked 'what user preferences' it mentions some stuff I have said in the past, but then said it had only just looked that up and it wasn't in effect at the time of the oversight, so was not the reason.

This is not the real fable. This is a lobotomised (opus 4.8ed) version of the original fable.

My methodology is driven in claude ai and claude code by a few prompts - my methodology is actually developed using that methodology(! thats a brain twister !) -- so the original prompts I gave this process have been developed by claude over time based on performance in various projects (retrospectives based on automated reports).

I hadn't looked at the prompts for months... I just did... WOW! AI self improvement works, the prompts it is using now have clearly changed from where we started (and will continue to do so) - and over time they will just get better.

The future is here....

Using AI to code makes human coding influence an 'optimisation', leave the AI to use it's training to the max and develop as it is most comfortable. A few architectural nudges maybe but until the system is reasonably functional any human code (optimisation) is an exception that the AI has to specifically cater for - adding complexity and cost.

The more you tinker the more the AI has to treat as exceptions the worse it is likely to perform.

Give it it's head and let it get on with it, only optimise (have human coding control) at the end.

Vibe coding FTW. Only optimise (get humans in) for production.

Well, that's my view - and how I have always coded best, leave me to it until I say it is done - each choice is based on many influences, rules of thumb, intuition and experience that would take an age to explain, try to force something early and you lose that value and add a potential stumbling block.

Decentralised proof of entitlement that is entirely under your control... Small, simple, free and open source.

Are you old enough? Have you paid? Can you be contacted? Are you qualified? Are you entitled to a service?

Typical questions with the lazy default answer 'with a government id we can work it all out'.

How about... No government id, no central register, no information about you shared at the point of enquiry other than confirmation or denial that the person in front of you is (or is not) entitled?

Want to get involved? Check out the project - Adelos.

I have been doing PWAs for quite a while, now I am seriously working on creating them instead of native apps.

&#x200B;

A few things I have discovered

&#x200B;

Only chrome installed PWAs seen to show up in 'share to'.

&#x200B;

PWAs cNr show up in 'open with...' - for this I created a small shim app - config it with the file extension and which PWA to route to... Now own with 'pwa->' and voila!

&#x200B;

If your delivery platform is a browser with an occasional internet connection, you have shut down most technology risks already.

&#x200B;

No binaries, just source code being executed in a massively reinforced browser sand box. Any AI can fetch the code and test and examine it.

&#x200B;

Who needs a 'store' to validate (control) to he apps available when anyone on earth can (potentially) validate your app - people or AIs?

&#x200B;

The future ideal platform is a tablet browser with a WiFi connection. Running bespoke pwas (maybe generated by AI) is minimal risk because the source code can be validated before you run it by people and/or by AI.

&#x200B;

Today only Firefox supports extensions/plug-ins on android... Google are attacking non-store installs ('side loading')... Apple never allows it... We really are on a cusp and PWAs are the immediate solution...

&#x200B;

&#x200B;

I have done shedloads with claude - it is my go to dev LLM/AI.

It has often done stuff in minutes that would otherwise have taken days.

But this is an example of that weird exception -- where something apparently quite simple goes round and round as if a junior dev is doing the work...

I kicked it off on a task and (for fun!) let it get on with it, without asking me at each step...

It isn't very pretty!

I thought surely claude has done this a million times - but if not and doing it from scratch how does it forget that *nix is case sensitive?

LLM's are great, but you need to know how to handle them - and that takes practice and experimentation...

------
&gt; can you tell me what has happened - an overview of the issues/errors you encounterd, your response and where things are now?

● What's been happening

Goal: Build a Docker image (ssd-render-service) on the Linux server that runs WeasyPrint to render Markdown→PDF/A. The build runs on superserver-lan via SSH from

Windows.

---

Issue 1: Debian apt CDN returning 400/403 for packages with + in filenames

The default Debian CDN (Fastly) URL-encodes + as %2b in package filenames, and returns 400 Bad Request for packages like fonts-ebgaramond (+git20210310 in the name)

and fonts-jetbrains-mono (+ds-3).

Fix attempted: Switched apt mirror to the German mirror (ftp.de.debian.org). This helped some packages but curl (+deb12u14) and fonts-jetbrains-mono then returned

400/403 from the German mirror on a fresh --no-cache build.

Final fix: Dropped all font packages and curl/unzip from apt entirely. Now only the native WeasyPrint libs are installed via apt (pango, cairo, harfbuzz, fontconfig —

these all have clean version strings or work fine with deb.debian.org). Fonts are downloaded separately using Python, which is already in the base image.

---

Issue 2: Font download via Python urllib — connection truncated mid-transfer

With fonts moving to Python-based download (download_fonts.py), the EB Garamond files (300–400 KB each) downloaded fine. But the Inter release zip (~28 MB) was

silently truncated by the GitHub CDN — the connection closed cleanly partway through (first attempt: 22.5 MB, raised IncompleteRead; second attempt: 25 MB, no

exception but BadZipFile when opened).

Root cause: GitHub releases redirect to release-assets.githubusercontent.com. The CDN closes the TCP connection before delivering all data on large files, and

Python's chunked read() loop interprets a clean close as "done" rather than raising an error.

Fix: Rewrote fetch() to:

1. Follow redirects to the final CDN URL via a HEAD request

2. Check if Accept-Ranges: bytes is supported (confirmed: yes, 27,885,132 bytes)

3. For files larger than 5 MB: download in 5 MB range slices on the final CDN URL, avoiding any single connection large enough to trigger the truncation

---

Where things are now

A build is running right now (background task by8v09nj8). The range-request strategy was confirmed working by a live test on the server — the final CDN URL resolves

correctly and reports Accept-Ranges: bytes. Once this build completes, the next steps are:

1. Verify the pytest determinism suite passes inside the build

2. Start the container with docker compose up -d

3. Smoke-test GET /health and POST /render with a sample bundle
   ------

This is a test post, signed, sealed and delivered #ssd

—SSD·BB0227EF·rd:id-ltd·5e09559e·XRyLT1MTsWTJNc+yfQqmeNkgAksRWB3RZ0CiS8n1lk3DChOTOw6i4VtmCNQ4BTe7yQcLxY7WKOrIbTzREPQEBg==·2026-06-03T21:18Z—