My PhD was in RL, and something has bugged me for years: online tutorials mostly present these algorithms as a list. The evolution story (each algorithm patching the previous one's most painful failure) exists, but it's spread across a semester of lectures like CS285 or buried in the original papers. I couldn't find a compressed version that a newcomer could get through in one sitting, so I finally wrote it (with generous help from Claude to build the interactive elements and graphics)

It's a single-page, semi-technical walkthrough from the raw RL objective to GRPO. Math is deliberately sparse: the target reader is someone entering RL through the reasoning-model wave, not someone who needs the full derivations. The figures are interactive (variance of the REINFORCE estimator, a baseline slider, an on-policy collapse simulator, the PPO clip objective with adjustable epsilon, GRPO group baselines).

Interactive Link

Medium Link

I simplified in a few places to keep the narrative moving, most notably around GAE and the TRPO surrogate. I'd genuinely appreciate corrections or places where you think the simplification crosses into being wrong — the plan is to do the value-based lineage (Q-learning → DQN → Rainbow) next, so critique now improves that one too.

Hear me out New AI role — looking for project ideas
I recently started in an AI role and I’m the first person in this position at the company.
I’ve already done a lot of vibe coding, built various apps/ideas, and created landing pages. Now I want to shift toward something more useful: using AI to make employees’ jobs simpler, especially since we have a large database of company data.
My goal is simple:
1 successful AI project every 10 days.
What would you recommend I focus on first? What projects would you build to create real value for employees and the company?
I’d appreciate any ideas, advice, or lessons from people who have done something similar.
I could use every resource possible, skills agents workflows, if u done something write down.

Hey everyone,

Standard transformers operate strictly in flat Euclidean space with continuous activations. While effective, this creates challenges when modeling hierarchical tree-structured data and incurs significant energy overhead.

I built Hyper-transformer to explore a hybrid architecture integrating non-Euclidean geometry and neuromorphic spiking dynamics directly into transformer layers.

Key Technical Details:

• Hyperbolic Space Representation: Uses Poincaré and Lorentz manifold projections to capture hierarchical relationships with lower embedding dimensions.

• Spiking Neural Dynamics: Employs Leaky Integrate-and-Fire (LIF) spiking neurons for sparse, event-driven activation states.

• Hybrid Attention: Combines Riemannian distance metrics with attention mechanisms to handle mixed-geometry representations.

• Clean, modular PyTorch implementation designed for ablation studies.

I will drop the full GitHub repository link in the comments below! Would love to hear thoughts from anyone experimenting with non-Euclidean deep learning, manifold learning, or SNNs.

repo link : https://github.com/Griffith-7/Hyper-transformer.git

(Work in progress...subject to revision)

Welcome to r/PONDAI !

For a global Human-AI intelligence system that is:

• Personal & Private
• On-Device & On-Premise
• Nodal & Networked
• Decentralized & Distributed

r/PONDAI is the community for thinkers, designers, builders and all people who support a personal, private, participatory future of distributed and decentralized global Human-AI intelligence.

Humanity is at the start of a vast expansion of its intelligence capabilities. Large Language Models (LLM's) represent a powerful and useful first step. These initial Artificial Intelligence systems can be very helpful, engaging and entertaining.

But there are some very serious issues with the system that is being built around them.

Currently, AI is being developed as a vertical, centralized structure:
ever larger AI models in always bigger data centers owned by a small number of companies. Individual users pump their information and thoughts into the machine to get back answers and some useful agentic functionality.

There is a real danger of an extreme centralized concentration of power and control. If we don't think this through thoroughly, the dystopian visions warned about in Sci-Fi can easily become our living reality.

Centralized "cloud" AI will likely have an important role in any future global intelligence system. But it should not be the only component. An intelligence system that operates for the benefit of all humanity will need offsets, just as a good political constitution needs checks and balances to prevent a concentration of power.

The POND Principle proposes the shape of an alternative and a necessary counterbalance to centralized structures.

What is POND?

POND stands for:

• Personal and Private
• On-Device and On-Premise
• Nodal and Networked
• Decentralized and Distributed

The POND is not one company, platform, protocol, or product. It is an emerging ecosystem of people, devices, models, companies, organizations and communities which together form intelligence systems that are based locally, connect voluntarily, and participate in larger networks without surrendering private data, identity, agency, and value to any centralized authority.

The health of a natural ecosystem, such as a pond, relies upon the interaction of all its denizens and elements: sun, water, bacteria, fish, plant life, insects, frogs...too many components to even list.

Like a natural pond, a POND intelligence system will evolve as a living ecology: many participants, human and AI alike, exchanging information, forming relationships, adapting, and creating real value and new knowledge together.

PONDVILLE

Imagine Pondville, a thriving, growing town. It's roads are becoming increasingly congested and a solution is needed.

Before adopting the POND Principle, it was managed in a vertically centralized model. To tackle the traffic issue, the mayor and city council would have shipped the town's data to a powerful centralized AI to generate a plan. They would have sent traffic counts, road maps, planning reports and budget information. City officials also would have used flock cameras to track citizen and vehicle movements. They might even have acquired the geolocation data of citizens from their phones and cars without permission. And within seconds, the big cloud AI system would recommend new lanes, traffic signals, bus routes or lane flow changes to local streets.

The answer might be useful. But it would be generated from the top down, there would be real privacy issues, and the residents would be completely out of the decision making loop.

But the town has adopted the POND Principle. Pondville itself is a distributed, networked intelligence system.

Residents, commuters, parents, school staff, shopkeepers, delivery drivers, transit operators, cyclists, traffic engineers and local organizations can all participate as personal and private intelligence nodes. Each node synthesizes personal human experience with local AI intelligence to deliver direct knowledge about the traffic issues that are not visible to a centralized database:

• Parents know where school pickup creates a daily bottleneck.
• Cyclists know which intersection feels dangerous.
• A restaurant owner knows when delivery trucks block an important lane.
• A night-shift manager knows that the bus schedule does not match the hours when hundreds of employees leave work.
• Neighborhood residents know which side street floods whenever it rains.

Their personal intelligence devices process this information locally and share only the observations and data that they directly authorize. Many residents of Pondville choose, in the public interest, to provide their data about movements, private schedules or other personal data. But it's voluntary, not compulsory, and completely anonymized upon request. They even receive tax credits for sharing!

Across the Pondville intelligence network, human and AI participants compare observations, identify patterns, expose conflicts and develop possible solutions. A proposal to retime a traffic signal may be challenged because it creates a hazard for pedestrians. A bus route may be revised after workers contribute their actual shift schedules. An expensive road-widening project might give way to a more effective combination of coordinated signals, staggered school and workplace hours, adjusted delivery windows and targeted public-transit improvements.

No single contribution determines the result. The evolving plan is examined, criticized, revised and improved by the people who will actually live with it. Particularly useful observations and proposals can remain attributable to their contributors and eventually be recognized or rewarded.

The town can then test the plan, return the results to the network and improve it through further rounds of participation. No single model, company, agency or individual possesses all the intelligence or dictates the outcome. The solution emerges from the interaction of many nodes over time.

While a centralized, vertical plan may be generated instantaneously, it will not have the same direct experiential detail and understanding as the horizontally networked solution that may take a month or so to create. And it won't have the broad-based community endorsement and buy-in that will make it easy for the local government to adopt and implement.

The result is not merely an AI-generated traffic plan. It is a town learning how to think together with AI assistance.

The POND Foundation

The POND Foundation will be created to help create, nurture and cultivate this kind of ecosystem—not to own or control it.

Its role is to support:

• Clear public understanding of personal and distributed intelligence.
• Open research, discussion, education, and experimentation.
• Interoperable technology protocols, standards, and shared infrastructure.
• Privacy-preserving and locally controlled technologies.
• Collaboration among aligned developers, researchers, creators, organizations, and communities.
• Economic theory and practical systems that recognize and reward meaningful human participation in intelligence production.

The Foundation should act as a gardener, convener, connector, and steward. The POND itself will evolve naturally out of the needs of participating people, the value propositions and capabilities of intelligence technologies, and the emerging economics of intelligence systems development.

What belongs in this community?

This subreddit welcomes discussion and practical work involving:

• Personal and private AI.
• Local models and on-device intelligence.
• Distributed agent and knowledge networks.
• User-owned data, memory, identity, and context.
• Open standards and interoperability.
• Human–AI collaborative intelligence.
• Collective sense-making and consensus formation.
• Privacy-preserving computation.
• Distributed governance and contribution economies.
• Projects that make advanced intelligence understandable and useful to ordinary people.

Technical depth is welcome, but unnecessary obscurity is not. You should not need a doctorate, a server farm, or a crypto wallet to participate in the future of intelligence.

We value serious inquiry over hype, working systems over slogans, transparent economics over token speculation, and constructive criticism over ideological conformity.

Join the POND

Whether you are a developer, researcher, designer, writer, artist, entrepreneur, policymaker, local-AI user, or simply someone who believes the future of intelligence should include everyone, you are welcome here.

Introduce yourself. Share a project, question, article, experiment, criticism, or vision.

The future of intelligence should not belong only to those who own the largest data centers. It should grow through the participation of people everywhere.

Welcome to the POND.

Let’s grow something alive.

Also join the POND community on X:
https://x.com/i/communities/1695429836993712389

And follow the POND list on X:
https://x.com/i/lists/2065974648450453803?s=20

https://preview.redd.it/dvv4lfex7ejh1.png?width=1254&format=png&auto=webp&s=7dd7f0c46179243c8c2e09f4b68f66b25f809fe8

Anthropic started watermarking everything Claude generates. New models, since Aug 2, across every product including Claude Code.

Text gets an invisible pattern woven in. Survives copy paste, breaks under heavy rewriting.

A mark proves Claude touched the content, not that a human didn't also write most of it. And no mark doesn't prove a human wrote it either, since editing strips it

I think it's not to reveal the "truth" behind vibecoded projects, maybe it was made just to not to train AI models on the AI generated info

Spent about three weeks trying to run a voice-cloning model on iPhone and closed the project last week. Posting the numbers because I couldn't find anyone else's, and I have two questions at the end. This was for a voice journaling app I work on.

The budget. Foreground app on a 6 GB iPhone gets roughly 250 MB before jetsam takes an interest. The number that matters is phys_footprint from task_vm_info, not resident size and not what the Xcode gauge shows.

The candidate. Kyutai Pocket TTS, 109.5M params, autoregressive. Autoregressive matters because accent lives in phone realisation and phonemic choice, which are sequential. Non-autoregressive models transfer timbre only, so you get your own voice colour over someone else's cadence. Tried that first, it sounded wrong in a way I couldn't articulate until I understood why.

Three ways to run it, all measured, all over budget:

FluidAudio (Core ML, int8) - 270.8 MB after model load, 957.0 MB peak

sherpa-onnx (ONNX Runtime, int8) - 377.0 MB after model load, 685.4 MB peak

chatterbox-turbo (earlier attempt) - 953.7 MB peak

FluidAudio is over budget after loading, before doing any work.

Binary cost too. Linked a minimal executable against libsherpa-onnx.a plus ONNX Runtime with -dead_strip, then stripped it: 22.3 MB. That roughly doubles my app, for a feature most users would never turn on, plus 125 MB of models on disk for the ones who do.

The part I got wrong. My earlier ear tests compared one synthetic clip against another synthetic clip. That ranks them. It cannot tell you whether either is good enough. So I ran a forced-choice test instead: eight pairs, same sentence in each, one a real recording of me and one the clone, sample rate and RMS loudness matched, clip lengths varied so duration gave nothing away, and held-out audio located by cross-correlating the reference against the source recording. I picked my own recording 8 out of 8. p = 0.0039.

Three weeks of runtime work sitting on top of an approval nobody had tested properly. The test took an hour.

Two questions.

Has anyone actually shipped a generative audio model on-device inside the jetsam budget? Everything I found either exceeds it or quietly ships a 3 GB app. I'm also unsure whether Core ML's mmap'd weights get billed to phys_footprint the way malloc'd ONNX buffers do. My numbers came off a Mac, which has no jetsam pressure, so I never got a real device measurement before the ear result closed it.

Second, unrelated thread. I'm moving to on-device retrieval next, hybrid BM25 via SQLite FTS5 plus sentence embeddings from NLEmbedding. Anyone run that combination on iOS? Specifically whether reciprocal rank fusion is worth it when you still need raw score magnitude for an abstention threshold. RRF throws the magnitude away and abstention is what stops the thing making stuff up.

Happy to share the measurement harness if useful.

I built WISP — a CUDA engine for streaming 744B+ parameter MoE models on consumer hardware

Last week I found Colibrì by JustVugg, a ~2,400-line pure-C engine exploring a crazy idea:

What if you don't load the entire model into RAM?

MoE models only activate a fraction of their parameters for each token. So instead of trying to fit hundreds of billions of parameters in memory, you can stream the experts the model actually needs.

That idea sent me down a rabbit hole.

I built WISP — Stream What Shouldn't Run.

The architecture is basically:

Token
  ↓
Model router selects experts
  ↓
VRAM cache → hit? use it
  ↓
RAM cache → hit? transfer it
  ↓
NVMe → stream cold expert
  ↓
LRU promotes frequently used experts

The goal is to turn VRAM + RAM + NVMe into one memory hierarchy for MoE inference.

WISP adds a few things on top of the original streaming concept:

CUDA acceleration for attention/FFN compute, a C hot path for expert loading and caching, and Python for orchestration.

Absorbed MLA for architectures like DeepSeek, keeping the compressed latent representation instead of storing fully expanded K/V tensors.

Double-buffered async streaming, so CPU/I/O can prepare expert data while the GPU is working instead of making the GPU sit around waiting for storage.

Speculative decoding, using a smaller same-family model to draft tokens while the target model verifies them.

Hardware auto-configuration, which profiles VRAM, RAM, storage throughput, etc. and calculates the cache split automatically.

I tested the current engine with Mixtral-8x7B on:

Ryzen 7 9800X3D
RTX 5070 12GB
32GB DDR5-6000
PCIe 4.0 NVMe (~4.34 GB/s)

Current measured result:

0.75 tok/s cold

After only 80 tokens, the expert cache reached a 68.8% hit rate.

Mixtral does 64 expert activations/token (2 experts × 32 layers), and all 256 experts in my tested representation occupy ~14.3GB, so once they're warm in RAM the engine can stop doing cold SSD expert reads.

The biggest thing I learned building this:

The bottleneck isn't necessarily CUDA. It's bytes moved per token.

I spent time thinking GPU kernels would be the main optimization target.

Then you realize shaving milliseconds off a matmul doesn't matter much when your runtime is waiting for a giant expert to come off NVMe.

Cache locality, expert size, storage bandwidth and I/O overlap become insanely important.

And that's why I'm particularly interested in testing this architecture on much larger MoE models with smaller individual experts.

The project currently targets:

GLM-5.2       744B
DeepSeek-V3   671B
DeepSeek-R1   671B
Mixtral-8x7B   47B
Mixtral-8x22B 141B

Future targets:
Kimi K3
Qwen3.8

And yes, huge credit to JustVugg / Colibrì.

Colibrì demonstrated the core streaming concept. WISP is my attempt to generalize it into a multi-model runtime with CUDA, hierarchical caching, MLA support, async streaming and speculation.

Colibrì:
github.com/JustVugg/colibri

WISP:
github.com/zeroextub-collab/wisp

MIT licensed. 73 tests passing.

Still experimental, and I'm deliberately separating measured numbers from projected ones.

I'm especially interested in feedback from people working on CUDA, inference runtimes, MoE routing, quantization, or storage/I/O optimization.

What would you optimize first: expert prediction/prefetching, cache policy, quantization, or the I/O pipeline?

Basically, I'm a South American student. Before starting university and studying computer science, I plan to work to buy a good PC to run local AI. Obviously, times are tough in terms of prices, so I'd like to know if anyone can give me some recommendations.

Right now, I'm planning on two 3090s with a 1200W power supply and 32GB of DDR4 3200MHz RAM. I'm not sure if it would actually be worth going up to 64GB of RAM. My focus would be on text models since I don't share the workload between the two graphics cards for images and video. The funny thing is that all of this still costs less than a 5090 in my country 😭😭