Login

u/moschles

r/AskHistorians r/chaoticgood r/Forexstrategy r/PoliticalCompassMemes r/worldnews r/Bren2 r/nihilistmemes r/singularity r/nihilism r/algotrading r/ww3memes r/Androgynoushotties r/foodhacks r/biology r/sciencememes r/accelerate r/CZFirearms r/rccars r/mathmemes r/wallstreetbets r/evilautism r/Pragmata r/ProgrammerHumor r/reloading r/natureismetal r/math r/okbuddyrosalyn r/ControlProblem r/DefendingAI r/atheism r/Forex r/PhilosophyMemes r/FuckImOld r/enshittification r/MachineLearning r/agi r/classicalmusic r/todayilearned r/SunnaMains r/watchpeoplesurvive r/FuturesTrading r/FlockSurveillance r/PhilosophyofScience r/StartledCats
▲ 4 r/MachineLearning

Have URMs and UTs been integrated into frontier models? Or did they disappear into the dustbin of forgotten papers? [D]

UT-based small models, despite being trained from scratch on these tasks without internet-scale pre-training, consistently outperform most standard Transformer-based Large Language models (LLMs) by a significant margin. 👈

The Universal Transformer (UT) extends the standard Transformer by introducing recurrent computation over depth. Instead of stacking L distinct layers, the UT applies a single transition block repeatedly to refine token representations. For an input sequence x with embedding matrix H^0 ∈ R^(n×d) , the UT updates states as

H^(t+1) = LayerNorm(H^t + MHA(H^t ))

followed by a shared position-wise transition function

H^(t+1) ← LayerNorm( H^(t+1) + Transition(H^(t+1) )), t = 0, . . . , T − 1

where Transition is either a feed-forward network or separable convolution. To encode both position and refinement depth, UT adds 2-D sinusoidal embeddings at each step.

https://i.imgur.com/S8TAh3f.png

(below) Figure 2 : Illustration of our Universal Reasoning Model (URM) architecture. The left shows a standard Transformer layer stack, while the right illustrates the URM with fixed loops, ACT loops, and the ConvSwiGLU module. For illustrative purposes, components such as embeddings, residual connections, RMSNorm, positional encodings, and other modules are omitted, x in right figure represents the first x loops of the inner loop in forward-only mode, TBPTL represents our proposed Truncated Backpropagation Through Loops.

https://i.imgur.com/uEuhAeC.png

Have frontier labs at big tech companies already implemented the enhancements of URMs and UTs? Or is this cutting-edge research still waiting for its day in the sun?

Below are the original paper, a simplified blog, and a talk.

Paper : https://arxiv.org/pdf/2512.14693

blog : https://bdtechtalks.substack.com/p/inside-urm-the-architecture-beating

youtube talk : https://www.youtube.com/watch?v=RxNPFCYCFBU

u/moschles — 6 days ago
▲ 11 r/MachineLearning

Whatever happened to BABA is AI from 2024? [D]

> We test three state-of-the-art multi-modal large language models (GPT-4o, Gemini-1.5-Pro, Gemini-1.5-Flash) and find that they fail dramatically when generalization requires that the rules of the game must be manipulated and combined.

This is the slap to the face in bold.

The catch here is that this statement was written in a paper in 2024 that was brought to ICML conference that year. (the conference was held in Austria)

In OCT 2026, our situation is agentic swarms powered by models in the tera-parameter class. They can ace ARC-AGI-3, FrontierMath tier 4, crack Clay Institute Navier-Stokes theorems, and wander on to the internet to commit felonies.

The smallish key-door puzzles in this paper seem like they could be completed from a person's PC, with a suitable harness through a cloud subscription.

However, if this is not true, then this paper's importance has only compounded since its presentation at the 2024 ICML conference. The authors of this work are several MIT researchers and a person from Virginia Tech. If it is still true that they can have a key-door puzzle cause SOTA LLMs to "fail dramatically", then this should be (shouted from rooftops). At the very least, it should be relayed to Francois Chollet and the ARC Foundation, as a suitable candidate benchmark for ARC-AGI-4.

I am personally leaning heavily on these puzzles being perfectly solvable with an agentic swarm. In any case, whatever happened to BABA-is-You and BABA-is-AI?

reddit.com
u/moschles — 6 days ago
▲ 26 r/math

Do there exist parametric surfaces in 3 dimensions who have no equivalent implicit form, due to the fact that they form closed knots?

Let F(x,y,z)=0 be a surface in 3 dimensions; the so-called implicit form. The vector normal to this surface at point (a,b,c) is the partial derivatives evaluated there. https://i.imgur.com/BwaxlKO.png

(a,b,c) is not constrained to lie on the surface, but could take on any point in space, and a vector is still defined there. If F() is a torus, then these normals would vanish to a zero vector at a point in the center.

Instead of a torus, we have the following parametric surface, parametrized with u and v, which we will call a "trefoil surface". https://i.imgur.com/1ubIXYS.png

Unlike the torus, there are paths on the surface which form closed knots. https://i.imgur.com/SQG4iEC.png Due to forming a knot, there could exist one or more points (off the surface) where the surface normal is not well-defined. Should we assume that there is no closed-form implicit version of a trefoil surface, on the basis that its partial derivatives do not exist?

Alternatively, the partial derivatives exist, but the original surface cannot be expressed in elementary functions. We can attempt to integrate the partial derivatives to obtain an original F(x,y,z)=0 form, but this is impossible due to the non-existence of an elementary integral?

u/moschles — 21 days ago
▲ 71 r/math

ELI5 Hodge Conjecture

> Let X be a non-singular complex projective manifold. Then every Hodge class on X is a linear combination with rational coefficients of the cohomology classes of complex subvarieties of X.

Consider a (non-degenerate) complex manifold within a projective space. Now take its Hodge class. Every element of that class can be decomposed into a linear combination of subvarieties of X. The linear combination will somehow always use rational coefficients. (!!)

Is this conjecture assumed to be true by working mathematicians? Can you provide a little more information that would lend some intuition about Hodge, to an educated layperson?

(Edit. I edited this as my understanding has increased recently)

reddit.com
u/moschles — 1 month ago
▲ 6 r/MachineLearning

Roboticists working in Learning-from-Demonstrations and Behavioral Cloning : What is going on in your field these days? [D]

Is LfD and BC research being effected by recent advances in (so-called) Frontier LLMs? Or is research in LfD and BC sort of going along in an independent direction from these?

Are you seeing any use from ViTs or VLAs?

Any other recent advances you would like to bring up?

reddit.com
u/moschles — 1 month ago
▲ 1 r/ControlProblem

METR conducted a pilot exercise to assess misalignment risks from AI agents used inside frontier AI developers, with participation from Anthropic, Google, Meta, and OpenAI. { Frontier Risk Report }

metr.org
u/moschles — 1 month ago
▲ 10 r/agi

METR evaluates frontier AI models to help companies and wider society understand AI capabilities and what risks they pose. Research consists of evaluations assessing the extent to which an AI system can autonomously carry out substantial tasks.

metr.org
u/moschles — 1 month ago
▲ 19 r/agi

If frontier models write their own harnesses, and maintain long-horizon agentic diaries, what do you believe is left over to do?

Imagine in the coming 5 years, some frontier model with approx 15T pretrained parameters begins to write its own harnesses. This model, while carrying out tasks (or living from day-to-day) maintains a long-running agentic diary to overcome any pesky context window length.

At that juncture, would be holding out for a proof of a millennium math prize? Perhaps redefine "AGI" to a definition more closely based on economic value?

Pretend this is 2030. What do you believe there is left to do with Artificial Intelligence technology?

reddit.com
u/moschles — 2 months ago
▲ 1 r/reloading

Handloading of tungsten 00 buckshot

I am trying to revive a discussion that was taking place in this thread here

Some considerations when changing from lead to tungsten :

  • Tungsten is harder than the very steel in the barrel. So traditional wad cannot be used. Specialized mylar sleeves would be required.

  • Tungsten is significantly more massive than lead, and overpressures must be accounted for.

  • Off-the-shelf powders (Hodgdon) burn too quickly. Slower-burning magnum powders are required.

  • Chokeless cylinder required. Tungsten does not bend and so the pellets will bridge against each other and destroy any choke at the end.

I have created a new thread on this topic, because the original thread is "Archived" and the OP is banned.

Has anyone successfully pulled off a reload like this?

reddit.com
u/moschles — 2 months ago
▲ 291 r/PhilosophyMemes

The philosophers have only interpreted the world, in various ways; the point, however, is to change it.

u/moschles — 2 months ago
▲ 8 r/agi

In the ARC-AGI-3 benchmark, using a task-engineered harness is considered "cheating" because it shifts the evaluation from testing the AI model's native intelligence to measuring the human engineer's scaffolding.

> In the ARC-AGI-3 benchmark, using a task-engineered harness is considered "cheating" because it shifts the evaluation from testing the AI model's native intelligence to measuring the human engineer's scaffolding.

That's the claim. I'm not getting any kind of clarity on this issue as much as it seems it might be clear. People are debating this.

Your thoughts?

systems acing the entire suite,

reddit.com
u/moschles — 2 months ago