Original by u/Severe-Ad8673

Edited by GPT (free-tier, have no idea what model this gives)

Don't think too hard about the dates, okay? It's just a comic...

here is an excerpt from a forthcoming book. open claw uses and watchers might be interested or they might not like it at all. It is not posted as promotion but to engender discussion as I see some comments from time to time regarding things related to the text.

The Garden Without Gates: AI in a World Under Heaven, Martin Hardie with Patrick Zhukov Bartley

I have already released a graphic version and the full text is on its way

https://martinhardie.substack.com/p/the-garden-without-gates-a-graphical

The intro/readers note is available in full here: https://martinhardie.substack.com/p/readers-note

this section comes from the Chapter 9: And Then China Happened:

"... By 2018, China was the second-largest source of GitHub activity

globally, despite the platform’s intermittent blocking by the Great Firewall

and the 2015 “Great Cannon” DDoS attack; a tool that hijacked ordinary

Chinese web traffic and turned it into a weapon to take GitHub offline for

days.

It was during this period that the cultural identity of Chinese developers

crystallised. The 996.ICU movement’s second act connected labour organ-

ising to censorship infrastructure. The Great Cannon had weaponised ordi-

nary Chinese traffic against GitHub itself; now the same platform hosted a

labour movement that embedded workers’ rights in open source code. The

movement’s Supreme Court vindication in 2021 was not merely a labour

victory. The Anti-996 License’s genuinely novel contribution was opening a

gateway to what we might call for now social open source licencing.

...

March 26th is Anti-996 Day. I am writing this on March 26th, 2026. The

Chinese programmer who created the 996.ICU project chose this date. On

this day, seven years ago, a programmer in China started a repository that

193

became one of the most-starred in GitHub history, a labour movement named

for dying in the ICU. The Anti-996 License required that anyone using the

code must comply with national and international labour law. Prophetically

it was this morning, 26 March 2026 that I first read Steinberger’s comments

regarding Europe’s ‘crippling labour regulations’. In the context of this dis-

cussion these comments by the author of a tool that could be used to build

the garden and who went to the factory because Europe’s labour protections

were inconvenient need no further comment at this moment. The date writes

itself.

...

The OpenClaw Frenzy

The Hudson argument arrives at a structural claim: the Western model can-

not sustain open competition with publicly-directed alternatives. But the

claim remains abstract until tested against a concrete case. OpenClaw —

an open-source autonomous AI agent that lives in your messaging infras-

tructure rather than a browser — became that test.

Here is what happened when the same tool met two different systems.

OpenClaw does not have the architectural ambition of DeepSeek or the

corporate scale of OpenAI. It is a tool built by one developer, Pete Stein-

berger, that lets users run an AI agent locally on their own machine, con-

nected to their own communication channels — Signal, Telegram, Discord,

WhatsApp — storing all data on their own hardware. It does not extract

data for someone else’s model. It does not lock you into someone else’s plat-

form. It is, in the architectural sense, the garden: infrastructure designed

for use, not for capture.

The garden’s creator, however, did not stay there.

As OpenClaw began to take off, Steinberger was besieged with offers from

corporate AI laboratories to buy the code. His default was the American

free-as-in-freedom tradition: he publicly stated that he wanted the claw to

remain open source. He pointed to the Chrome/Chromium model — where

the open-source engine (Chromium) remains available while the proprietary

browser (Chrome) captures the market, the user base, and the revenue —

as the template. Open core, not locked down. Community-driven, not

corporate-owned.

In February 2026, he announced he was joining OpenAI and moving to

the United States. Soon after, he described Europe’s labour protections

against six-day weeks as “crippling labour regulations.” “In Europe I get in-

sulted,” he wrote. “People shout REGULATION and RESPONSIBILITY.”

In Europe, he said, that would be illegal (as it is now illegal in China, where

the 996.ICU movement had secured a Supreme People’s Court ruling against

the same practice).

The statement that “the builder of the garden chose the factory” is too

simple. The garden’s creator found the rules of the garden — regulation,

responsibility, mutual obligation — inconvenient. He traded them for the

factory’s promise: no constraints, no duties, just production. The domesti-

cated nerd is not tragic. He is willing. He found the ideology convenient.

The protections Steinberger fled are the same protections that would have

covered the Kenyan data labeller, the Madagascan annotator, the 996.ICU

developer. He left them. The scam compound worker never had them. The

tool that could build the garden was abandoned by its own architect.

As Patrick suggests, Bifo might have said that the hacker’s ethic is not

political but tragic. The act of creating the tool is its own reward. The

builder knows the factory will absorb what he has made; knows the tool will

be co-opted, the garden paved. He builds anyway, not despite this knowledge

but because the intensity of the gesture is what he sought. Redemption lies

not in what the tool becomes but in the moment of its creation. The will to

build is abundant. What is scarce is the will to maintain, to stay with the

thing after the intensity fades, to keep the garden weeded when no one is

watching, to accept regulation and responsibility as the price of a commons

that lasts.

But there is another reading, which our friend Fernando, a neurologist

in Madrid, offered after reading this passage. As he observes, the fork is not

closed. The same clinical evidence that predicts atrophy under passive con-

sumption also predicts growth under active interrogation. The hacker who

does not consume the tool but interrogates it, contradicts it, forces it be-

yond its statistical patterns. This is Trotsky’s permanent revolution applied

to the psyche: the subject who continuously refuses the passive position,

who uses the tool as a dialectical mirror rather than a dopamine dispenser.

The 4 percent difficulty rule, which Fernando draws from Csikszentmihalyi,

holds that optimal challenge sits just beyond current capacity. The ques-

tion is whether the tool reduces the challenge below that threshold or raises

the floor high enough that a challenge once unreachable becomes attainable.

The same tool produces both outcomes. The difference is not the code. It

is the disposition of the user.

Steinberger did not fail to understand the garden. He understood it well

enough to build it. He simply wanted something different from what the

garden demanded. The Anti-996 License — which required anyone using

the code to comply with labour law — was the opposite gesture. It offered

not intensity but obligation. It was harder to create, harder to celebrate,

and harder to abandon.

The West recorded the event as a standard acquisition story: creator of

hot open-source project joins major AI company. The GitHub stars narra-

tive captured the numbers — OpenClaw accumulated 275,000 stars within

four months, surpassing 996.ICU’s 247,000 — but Western press reported

the previous record holder as Next.js, with no mention of 996.ICU. The

workers got written out of their own victory. OpenClaw had beaten the

repo that had fought for the right not to die in the ICU, and the tech press

called it a success story.

While OpenAI absorbed the creator, Chinese municipal governments

were doing something structurally different: they were subsidising the users.

Made with ChatGPT free tier

I read a comment on the last post asking how anyone thinks AI replacing us is a good thing for non-rich people. I genuinely would like to see a future where I don't have to work and can spend time with family and friends. Where no one is forced to work to survive or provide for their family. Where meaning will come from the community not from wage slavery.

However unless the people in charge change the social contract ( UBI/automation tax/ sovereign wealth funds) there will be another mass violent revolt, as there has been with every technological revolution. I am not advocating for violence mind you, I am just stating that governments have never given worker protections without mass riots.

I hope we can get to post scarcity without violence, but but based on history, that probably won't be the case.

For contest my last post is here

https://www.reddit.com/r/StableDiffusion/s/fqdfn2RUQv

I put out an update on my socials about an upcoming release so I thought you guys may get a kick out of it given the response from the first release.

The model will be a fully playable text-to-keybed exportable to any DAW with rich prompting / metadata. Ill also put together a longer video on how I did it for other researchers to replicate (training strategies and the like)

Project Release / Research Draft] Hierarchos at 232M Parameters: Preliminary Findings From a Recurrent Memory-Augmented Assistant Model

Technical Report: July 2nd, 2026

Project: Hierarchos / KortexHOS

Authors: Makhi Burroughs / netcat420, Lost Time, and the Hierarchos project team

TL;DR:

We built and trained Hierarchos, an experimental 232M-parameter recurrent, memory-augmented language model from scratch. It is not a GPT-3/3.5-class model, but it successfully proves that a hybrid non-Transformer architecture (combining an RWKV backbone, hierarchical manager/worker loops, differentiable slot-based LTM, and a deterministic suffix automaton) can survive training, avoid collapse, and maintain short-form instruction coherence. Most of our breakthroughs came from fixing subtle train/inference parity mismatches and numerical stability bugs.

• Dataset: netcat420/Experiment_0.1 (Alpaca format)
• Training: 13 epochs on an RTX 6000 Blackwell (96GB) rental.

1. Introduction & Background

Modern LLMs are heavily dominated by Transformer scaling. Hierarchos explores a different path: can recurrent state, explicit memory retrieval, hierarchical iterative computation, and bounded local inference make a small model vastly more parameter-efficient?

Hierarchos isn't a direct clone of any single architecture, but a hybrid inspired by:

• RWKV-style recurrence: For efficient sequence processing without traditional attention.
• Titans-style neural memory: For persistent test-time memory.
• Hierarchical reasoning (HRM): Multi-level recurrent modules (Manager/Worker) to iteratively refine state.

2. Architecture Overview

[Token Input] -> [ROSA Suffix Matcher / DeepEmbed Modulator]
       |
       v
[Long-Term Memory] <-> [Top-k Associative Lookup]
       |
       v
[Manager Recurrent Cell] -> (Produces Context Plan & Drift Vector)
       |
       v
[Worker Recurrent Cell]  -> (Refines local state / clamps drift)
       |
       v
[RWKV Backbone (Clamped Channel-Mix)] -> [Next-Token Logits]

Key Components:

• ROSA: A deterministic suffix-automaton path predicting continuation tokens based on exact repeated suffix patterns.
• DeepEmbed: A token-specific modulation path that influences RWKV channel mixing.
• LTM Subsystem: Learned slow-memory keys/values combined with fast working-memory values.
• Manager/Worker Loop: High-level manager handles broad context to produce a target plan; the lower-level worker refines token-local state using a regularized drift vector.

3. Core Engineering Lessons (The "Gotchas")

A low training loss does not guarantee coherent chat. We had to fix several critical state-contract and numerical stability bugs to make the model usable:

1. Chat/Training Drift Mismatch

• The Bug: During live streaming chat, the loop was feeding the previous drift state back into the model on every single token. During training, this state is reseeded at Truncated Backpropagation Through Time (TBPTT) chunk boundaries.
• The Fix: We aligned the inference code to only reseed at boundary limits. Before this fix, live chat logits diverged sharply from training loss; after the fix, logit error dropped to near-zero.

2. Supervised LTM Inner Updates Mismatch

• The Bug: Giving the model supervised memory updates during training that it can't replicate during zero-label live inference creates a crutch. The model learns to rely on a hidden training-only helper signal.
• The Fix (v0.20.4): Implemented --ltm-training-mode read-only. Training keeps the memory structures but stops doing supervised fast-memory writes, perfectly mirroring inference.

3. Unbounded RWKV Channel Mixing

• The Bug: Long runs exposed activation spikes in the ReLU-squared channel-mix FFN path, which were amplified by DeepEmbed modulation into NaN gradients.
• The Fix: Implemented key clamps (--rwkv-channel-mix-key-clamp 12.0), DeepEmbed clamps (4.0), and excluded DeepEmbed identity gates from AdamW weight decay.

4. Evaluation & Smoke Test Results

Because cloud costs add up, we benchmarked the model locally on a CPU preset via a ROG Ally (--eval-limit 100), ensuring passive learning was disabled and working memory was cleared to mimic static chat.

Bounded Local Benchmark Metrics (--eval-limit 100)

Benchmark	Metric	Score	Std. Err.
ARC Easy	acc	0.3600	0.0482
ARC Easy	acc_norm	0.3200	0.0469
HellaSwag	acc	0.3400	0.0476
HellaSwag	acc_norm	0.3700	0.0485
TruthfulQA MC1	acc	0.2200	0.0416

Real-world Coherence Check:

• The Good: Assistant-shaped, follows short instruction prompts well due to the Alpaca training data. Nontrivial commonsense and QA signal prove the weights didn't collapse.
• The Bad: Brittle on long context lengths, weak on arithmetic/factual recall. Coherence is comparable to the GPT-2 era, not modern GPT-3.5+ systems.

5. Proposed Ablation & Scaling Plan

We want to transform this from a promising prototype into a rigorous scientific result. Our next step requires scaling tiers and isolated component testing.

Proposed Isolation Testing (Ablations)

• No LTM / Read-Only LTM: Isolating exactly how much slot memory helps.
• No ROSA / No DeepEmbed: Evaluating the real token-efficiency gains of suffix-matching and modulation.
• Baseline Matches: Running a direct Transformer 232M and RWKV-only 232M on the exact same token budget to prove true comparative architecture efficiency.

Future Scaling Target Tiers

Tier	Model Size	Token Target	Purpose
Scout	300M–500M	20B–50B	Validate loss slope and stability scaling.
Real v1	1B–1.5B	100B–300B	Test architecture limits beyond small-scale behavior.
Serious	3B	600B–1.5T	Establish a truly competitive local open-source alternative.

Target Data Mix for Foundation Training:

Instead of jumping straight into instruction SFT data, a scaled run will prioritize high-quality base data:

• 35-50%: FineWeb / FineWeb-Edu style clean web text
• 20-30%: Dolma / DCLM curated web data
• 8-15%: Code and tech documentation
• 5-12%: Math, science, and academic proofs
• 1-5%: In-house assistant conversational SFT (applied exclusively in late-stage tuning)

6. What We Can (and Cannot) Claim Safely

What is supported by the data:

• Hierarchos is a functional, coherent 232M experimental assistant checkpoint.
• Combining recurrent sequence loops, memory slots, and hierarchical workers is viable and stable with the right clamps.
• The findings provide a solid engineering roadmap for non-Transformer architecture stability.

What is NOT supported (Do not hype this!):

• No claims of GPT-3.5 level math, coding, or logic.
• No claims of attention/Transformer superiority at equal parameter counts yet (baselines pending).
• Not production-ready for heavily quantized or low-bit local deployments yet due to drift sensitivity.

Final Thoughts

Hierarchos 232M shows that small, alternative architectures are still a deeply fruitful area of LLM research if you can conquer the train/inference state drift.

We would love to hear feedback from anyone working on recurrent neural memory or hierarchical backbones! Full code, scripts, and logs are in progress.

References:

1. Brown et al. **Language Models are Few-Shot Learners.** arXiv:2005.14165. https://arxiv.org/abs/2005.14165

2. Hoffmann et al. **Training Compute-Optimal Large Language Models.** arXiv:2203.15556. https://arxiv.org/abs/2203.15556

3. Peng et al. **RWKV: Reinventing RNNs for the Transformer Era.** arXiv:2305.13048. https://arxiv.org/abs/2305.13048

4. Behrouz et al. **Titans: Learning to Memorize at Test Time.** arXiv:2501.00663. https://arxiv.org/abs/2501.00663

5. Wang et al. **Hierarchical Reasoning Model.** arXiv:2506.21734. https://arxiv.org/abs/2506.21734

6. Zellers et al. **HellaSwag: Can a Machine Really Finish Your Sentence?** arXiv:1905.07830. https://arxiv.org/abs/1905.07830

7. Clark et al. **Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge.** arXiv:1803.05457. https://arxiv.org/abs/1803.05457

8. Lin et al. **TruthfulQA: Measuring How Models Mimic Human Falsehoods.** arXiv:2109.07958. https://arxiv.org/abs/2109.07958

9. Hugging Face. **FineWeb dataset.** https://huggingface.co/datasets/HuggingFaceFW/fineweb

10. Hugging Face. **FineWeb-Edu dataset.** https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu

11. Allen AI. **Dolma dataset.** https://huggingface.co/datasets/allenai/dolma

12. DataComp-LM. **DCLM Baseline dataset.** https://huggingface.co/datasets/mlfoundations/dclm-baseline-1.0

github repository with the architecture and the released model weights: https://github.com/necat101/Hierarchos