u/Turbulent-Metal-9491

Hi everyone,

I’m an independent researcher (no lab affiliation) who has spent the last year diving deep into the internal dynamics of Transformers. Instead of looking at outputs or attention heads, I’ve been tracking the geometric trajectories of hidden states layer-by-layer during inference.

I wanted to share my latest findings (preprints linked below) because they reveal a structured "dynamic grammar" that seems universal across architectures, from GPT-2 to Llama-3.2.

The Core Idea

Most observability tools treat LLMs as static input-output machines. I treat them as dynamic systems. By measuring metrics like trajectory curvature (ct_t), functional capacity, and state transitions, I found that LLMs don’t just "generate text"—they navigate a latent space through specific, reproducible phases.

Key Findings (V20–V24)

1. A Universal Dynamic Grammar (V24)

Across 7 models (GPT-2, OPT, Qwen, TinyLlama, Phi-1.5, Llama-3.2, DistilGPT2), I observed a conserved sequence of internal states:

B (Branching/Hesitation): Initial exploration.

A (Adaptive/Stable): The main processing phase (an attractor state).

D (Decision/Bifurcation): Final commitment to a token.

Result: B → A → D appears to be the "standard cognitive path" for coherent generation. Deviations from this path often correlate with errors or hallucinations.

2. Geometry > Neurons (V22)

Using orthogonal rotation controls, I proved that functional information (syntax, decision, stabilization) is encoded in the relative geometry of the representation space, not in individual neurons. If you rotate the latent space, the information remains decodable. This suggests LLMs think in shapes, not just activations.

3. Ambiguity Changes the Path, Not the Chaos (V23)

When prompts are ambiguous, models don’t necessarily become "chaotic." Instead, they delay commitment. They spend more time in the exploration phase (B) and less time rushing to decision (D). Phi-1.5, interestingly, shows a unique oscillating pattern (B↔A) during reasoning tasks, distinct from the smoother convergence of other models.

4. Architecture Matters More Than Size (V20)

Models cluster by their dynamic signatures (e.g., GD_ratio), not just parameter count. Small models like Qwen-0.5B show distinct stability regimes compared to GPT-2, despite similar sizes.

The Preprints (Open Access)

[June 2026] A Runtime Trajectory Dynamics Framework (V20): Introduces the 5-state taxonomy (Stable, Turbulence, Branching, Bifurcation, Committed) and the bicephalic operator.

Link: https://doi.org/10.5281/zenodo.20602685

[May 2026] Dynamic-Layer Controllability (V21): Shows how perturbations affect recovery and proves that emergent organization dominates architectural skeleton.

Link: https://doi.org/10.5281/zenodo.20400171

[May 2026] Conditional Dynamic Signatures (V22): Audits normalization effects and variance decomposition. Explicitly documents falsified claims.

Link: https://doi.org/10.5281/zenodo.20361289

[May 2026] Four Dynamical Regimes (V19/V20): Introduces ct_t (curvature × displacement) as a predictor of collapse and instability.

Link: https://doi.org/10.5281/zenodo.20348878

Why I’m Posting This

I’m not selling a product. I’m building an open framework (LIMEN) to make LLM internals auditable and controllable. I believe that if we want safe AI, we need to monitor its "vital signs" (dynamic stability) in real-time, not just its output.

I’d love feedback from the community, especially on:

Have you seen similar "universal motifs" in larger models (>7B)?

Critiques on the methodology (normalization, probe training).

Ideas for causal interventions based on these dynamic states.

Hi everyone,

I’m an independent researcher (no lab affiliation) who has spent the last year diving deep into the internal dynamics of Transformers. Instead of looking at outputs or attention heads, I’ve been tracking the geometric trajectories of hidden states layer-by-layer during inference.

I wanted to share my latest findings (preprints linked below) because they reveal a structured "dynamic grammar" that seems universal across architectures, from GPT-2 to Llama-3.2.

The Core Idea

Most observability tools treat LLMs as static input-output machines. I treat them as dynamic systems. By measuring metrics like trajectory curvature (ct_t), functional capacity, and state transitions, I found that LLMs don’t just "generate text"—they navigate a latent space through specific, reproducible phases.

Key Findings (V20–V24)

1. A Universal Dynamic Grammar (V24)

Across 7 models (GPT-2, OPT, Qwen, TinyLlama, Phi-1.5, Llama-3.2, DistilGPT2), I observed a conserved sequence of internal states:

B (Branching/Hesitation): Initial exploration.

A (Adaptive/Stable): The main processing phase (an attractor state).

D (Decision/Bifurcation): Final commitment to a token.

Result: B → A → D appears to be the "standard cognitive path" for coherent generation. Deviations from this path often correlate with errors or hallucinations.

2. Geometry > Neurons (V22)

Using orthogonal rotation controls, I proved that functional information (syntax, decision, stabilization) is encoded in the relative geometry of the representation space, not in individual neurons. If you rotate the latent space, the information remains decodable. This suggests LLMs think in shapes, not just activations.

3. Ambiguity Changes the Path, Not the Chaos (V23)

When prompts are ambiguous, models don’t necessarily become "chaotic." Instead, they delay commitment. They spend more time in the exploration phase (B) and less time rushing to decision (D). Phi-1.5, interestingly, shows a unique oscillating pattern (B↔A) during reasoning tasks, distinct from the smoother convergence of other models.

4. Architecture Matters More Than Size (V20)

Models cluster by their dynamic signatures (e.g., GD_ratio), not just parameter count. Small models like Qwen-0.5B show distinct stability regimes compared to GPT-2, despite similar sizes.

The Preprints (Open Access)

[June 2026] A Runtime Trajectory Dynamics Framework (V20): Introduces the 5-state taxonomy (Stable, Turbulence, Branching, Bifurcation, Committed) and the bicephalic operator.

Link: https://doi.org/10.5281/zenodo.20602685

[May 2026] Dynamic-Layer Controllability (V21): Shows how perturbations affect recovery and proves that emergent organization dominates architectural skeleton.

Link: https://doi.org/10.5281/zenodo.20400171

[May 2026] Conditional Dynamic Signatures (V22): Audits normalization effects and variance decomposition. Explicitly documents falsified claims.

Link: https://doi.org/10.5281/zenodo.20361289

[May 2026] Four Dynamical Regimes (V19/V20): Introduces ct_t (curvature × displacement) as a predictor of collapse and instability.

Link: https://doi.org/10.5281/zenodo.20348878

Why I’m Posting This

I’m not selling a product. I’m building an open framework (LIMEN) to make LLM internals auditable and controllable. I believe that if we want safe AI, we need to monitor its "vital signs" (dynamic stability) in real-time, not just its output.

I’d love feedback from the community, especially on:

Have you seen similar "universal motifs" in larger models (>7B)?

Critiques on the methodology (normalization, probe training).

Ideas for causal interventions based on these dynamic states.

Hi everyone,

I’m an independent researcher (no lab affiliation) who has spent the last year diving deep into the internal dynamics of Transformers. Instead of looking at outputs or attention heads, I’ve been tracking the geometric trajectories of hidden states layer-by-layer during inference.

I wanted to share my latest findings (preprints linked below) because they reveal a structured "dynamic grammar" that seems universal across architectures, from GPT-2 to Llama-3.2.

The Core Idea

Most observability tools treat LLMs as static input-output machines. I treat them as dynamic systems. By measuring metrics like trajectory curvature (ct_t), functional capacity, and state transitions, I found that LLMs don’t just "generate text"—they navigate a latent space through specific, reproducible phases.

Key Findings (V20–V24)

1. A Universal Dynamic Grammar (V24)

Across 7 models (GPT-2, OPT, Qwen, TinyLlama, Phi-1.5, Llama-3.2, DistilGPT2), I observed a conserved sequence of internal states:

B (Branching/Hesitation): Initial exploration.

A (Adaptive/Stable): The main processing phase (an attractor state).

D (Decision/Bifurcation): Final commitment to a token.

Result: B → A → D appears to be the "standard cognitive path" for coherent generation. Deviations from this path often correlate with errors or hallucinations.

2. Geometry > Neurons (V22)

Using orthogonal rotation controls, I proved that functional information (syntax, decision, stabilization) is encoded in the relative geometry of the representation space, not in individual neurons. If you rotate the latent space, the information remains decodable. This suggests LLMs think in shapes, not just activations.

3. Ambiguity Changes the Path, Not the Chaos (V23)

When prompts are ambiguous, models don’t necessarily become "chaotic." Instead, they delay commitment. They spend more time in the exploration phase (B) and less time rushing to decision (D). Phi-1.5, interestingly, shows a unique oscillating pattern (B↔A) during reasoning tasks, distinct from the smoother convergence of other models.

4. Architecture Matters More Than Size (V20)

Models cluster by their dynamic signatures (e.g., GD_ratio), not just parameter count. Small models like Qwen-0.5B show distinct stability regimes compared to GPT-2, despite similar sizes.

The Preprints (Open Access)

[June 2026] A Runtime Trajectory Dynamics Framework (V20): Introduces the 5-state taxonomy (Stable, Turbulence, Branching, Bifurcation, Committed) and the bicephalic operator.

Link: https://doi.org/10.5281/zenodo.20602685

[May 2026] Dynamic-Layer Controllability (V21): Shows how perturbations affect recovery and proves that emergent organization dominates architectural skeleton.

Link: https://doi.org/10.5281/zenodo.20400171

[May 2026] Conditional Dynamic Signatures (V22): Audits normalization effects and variance decomposition. Explicitly documents falsified claims.

Link: https://doi.org/10.5281/zenodo.20361289

[May 2026] Four Dynamical Regimes (V19/V20): Introduces ct_t (curvature × displacement) as a predictor of collapse and instability.

Link: https://doi.org/10.5281/zenodo.20348878

Why I’m Posting This

I’m not selling a product. I’m building an open framework (LIMEN) to make LLM internals auditable and controllable. I believe that if we want safe AI, we need to monitor its "vital signs" (dynamic stability) in real-time, not just its output.

I’d love feedback from the community, especially on:

Have you seen similar "universal motifs" in larger models (>7B)?

Critiques on the methodology (normalization, probe training).

Ideas for causal interventions based on these dynamic states.

Hi everyone,

I’m an independent researcher (no lab affiliation) who has spent the last year diving deep into the internal dynamics of Transformers. Instead of looking at outputs or attention heads, I’ve been tracking the geometric trajectories of hidden states layer-by-layer during inference.

I wanted to share my latest findings (preprints linked below) because they reveal a structured "dynamic grammar" that seems universal across architectures, from GPT-2 to Llama-3.2.

The Core Idea

Most observability tools treat LLMs as static input-output machines. I treat them as dynamic systems. By measuring metrics like trajectory curvature (ct_t), functional capacity, and state transitions, I found that LLMs don’t just "generate text"—they navigate a latent space through specific, reproducible phases.

Key Findings (V20–V24)

1. A Universal Dynamic Grammar (V24)

Across 7 models (GPT-2, OPT, Qwen, TinyLlama, Phi-1.5, Llama-3.2, DistilGPT2), I observed a conserved sequence of internal states:

B (Branching/Hesitation): Initial exploration.

A (Adaptive/Stable): The main processing phase (an attractor state).

D (Decision/Bifurcation): Final commitment to a token.

Result: B → A → D appears to be the "standard cognitive path" for coherent generation. Deviations from this path often correlate with errors or hallucinations.

2. Geometry > Neurons (V22)

Using orthogonal rotation controls, I proved that functional information (syntax, decision, stabilization) is encoded in the relative geometry of the representation space, not in individual neurons. If you rotate the latent space, the information remains decodable. This suggests LLMs think in shapes, not just activations.

3. Ambiguity Changes the Path, Not the Chaos (V23)

When prompts are ambiguous, models don’t necessarily become "chaotic." Instead, they delay commitment. They spend more time in the exploration phase (B) and less time rushing to decision (D). Phi-1.5, interestingly, shows a unique oscillating pattern (B↔A) during reasoning tasks, distinct from the smoother convergence of other models.

4. Architecture Matters More Than Size (V20)

Models cluster by their dynamic signatures (e.g., GD_ratio), not just parameter count. Small models like Qwen-0.5B show distinct stability regimes compared to GPT-2, despite similar sizes.

The Preprints (Open Access)

[June 2026] A Runtime Trajectory Dynamics Framework (V20): Introduces the 5-state taxonomy (Stable, Turbulence, Branching, Bifurcation, Committed) and the bicephalic operator.

Link: https://doi.org/10.5281/zenodo.20602685

[May 2026] Dynamic-Layer Controllability (V21): Shows how perturbations affect recovery and proves that emergent organization dominates architectural skeleton.

Link: https://doi.org/10.5281/zenodo.20400171

[May 2026] Conditional Dynamic Signatures (V22): Audits normalization effects and variance decomposition. Explicitly documents falsified claims.

Link: https://doi.org/10.5281/zenodo.20361289

[May 2026] Four Dynamical Regimes (V19/V20): Introduces ct_t (curvature × displacement) as a predictor of collapse and instability.

Link: https://doi.org/10.5281/zenodo.20348878

Why I’m Posting This

I’m not selling a product. I’m building an open framework (LIMEN) to make LLM internals auditable and controllable. I believe that if we want safe AI, we need to monitor its "vital signs" (dynamic stability) in real-time, not just its output.

I’d love feedback from the community, especially on:

Have you seen similar "universal motifs" in larger models (>7B)?

Critiques on the methodology (normalization, probe training).

Ideas for causal interventions based on these dynamic states.

Hi everyone,

I’m an independent researcher (no lab affiliation) who has spent the last year diving deep into the internal dynamics of Transformers. Instead of looking at outputs or attention heads, I’ve been tracking the geometric trajectories of hidden states layer-by-layer during inference.

I wanted to share my latest findings (preprints linked below) because they reveal a structured "dynamic grammar" that seems universal across architectures, from GPT-2 to Llama-3.2.

The Core Idea

Most observability tools treat LLMs as static input-output machines. I treat them as dynamic systems. By measuring metrics like trajectory curvature (ct_t), functional capacity, and state transitions, I found that LLMs don’t just "generate text"—they navigate a latent space through specific, reproducible phases.

Key Findings (V20–V24)

1. A Universal Dynamic Grammar (V24)

Across 7 models (GPT-2, OPT, Qwen, TinyLlama, Phi-1.5, Llama-3.2, DistilGPT2), I observed a conserved sequence of internal states:

B (Branching/Hesitation): Initial exploration.

A (Adaptive/Stable): The main processing phase (an attractor state).

D (Decision/Bifurcation): Final commitment to a token.

Result: B → A → D appears to be the "standard cognitive path" for coherent generation. Deviations from this path often correlate with errors or hallucinations.

2. Geometry > Neurons (V22)

Using orthogonal rotation controls, I proved that functional information (syntax, decision, stabilization) is encoded in the relative geometry of the representation space, not in individual neurons. If you rotate the latent space, the information remains decodable. This suggests LLMs think in shapes, not just activations.

3. Ambiguity Changes the Path, Not the Chaos (V23)

When prompts are ambiguous, models don’t necessarily become "chaotic." Instead, they delay commitment. They spend more time in the exploration phase (B) and less time rushing to decision (D). Phi-1.5, interestingly, shows a unique oscillating pattern (B↔A) during reasoning tasks, distinct from the smoother convergence of other models.

4. Architecture Matters More Than Size (V20)

Models cluster by their dynamic signatures (e.g., GD_ratio), not just parameter count. Small models like Qwen-0.5B show distinct stability regimes compared to GPT-2, despite similar sizes.

The Preprints (Open Access)

I’ve documented everything, including falsified hypotheses and negative results, because I believe transparency is key for independent research.

[June 2026] A Runtime Trajectory Dynamics Framework (V20): Introduces the 5-state taxonomy (Stable, Turbulence, Branching, Bifurcation, Committed) and the bicephalic operator.

Link: https://doi.org/10.5281/zenodo.20602685

[May 2026] Dynamic-Layer Controllability (V21): Shows how perturbations affect recovery and proves that emergent organization dominates architectural skeleton.

Link: https://doi.org/10.5281/zenodo.20400171

[May 2026] Conditional Dynamic Signatures (V22): Audits normalization effects and variance decomposition. Explicitly documents falsified claims.

Link: https://doi.org/10.5281/zenodo.20361289

[May 2026] Four Dynamical Regimes (V19/V20): Introduces ct_t (curvature × displacement) as a predictor of collapse and instability.

Link: https://doi.org/10.5281/zenodo.20348878

Why I’m Posting This

I’m not selling a product. I’m building an open framework (LIMEN) to make LLM internals auditable and controllable. I believe that if we want safe AI, we need to monitor its "vital signs" (dynamic stability) in real-time, not just its output.

I’d love feedback from the community, especially on:

Have you seen similar "universal motifs" in larger models (>7B)?

Critiques on the methodology (normalization, probe training).

Ideas for causal interventions based on these dynamic states.

Hi everyone,

I’m an independent researcher (no lab affiliation) who has spent the last year diving deep into the internal dynamics of Transformers. Instead of looking at outputs or attention heads, I’ve been tracking the geometric trajectories of hidden states layer-by-layer during inference.

I wanted to share my latest findings (preprints linked below) because they reveal a structured "dynamic grammar" that seems universal across architectures, from GPT-2 to Llama-3.2.

The Core Idea

Most observability tools treat LLMs as static input-output machines. I treat them as dynamic systems. By measuring metrics like trajectory curvature (ct_t), functional capacity, and state transitions, I found that LLMs don’t just "generate text"—they navigate a latent space through specific, reproducible phases.

Key Findings (V20–V24)

1. A Universal Dynamic Grammar (V24)

Across 7 models (GPT-2, OPT, Qwen, TinyLlama, Phi-1.5, Llama-3.2, DistilGPT2), I observed a conserved sequence of internal states:

B (Branching/Hesitation): Initial exploration.

A (Adaptive/Stable): The main processing phase (an attractor state).

D (Decision/Bifurcation): Final commitment to a token.

Result: B → A → D appears to be the "standard cognitive path" for coherent generation. Deviations from this path often correlate with errors or hallucinations.

2. Geometry > Neurons (V22)

Using orthogonal rotation controls, I proved that functional information (syntax, decision, stabilization) is encoded in the relative geometry of the representation space, not in individual neurons. If you rotate the latent space, the information remains decodable. This suggests LLMs think in shapes, not just activations.

3. Ambiguity Changes the Path, Not the Chaos (V23)

When prompts are ambiguous, models don’t necessarily become "chaotic." Instead, they delay commitment. They spend more time in the exploration phase (B) and less time rushing to decision (D). Phi-1.5, interestingly, shows a unique oscillating pattern (B↔A) during reasoning tasks, distinct from the smoother convergence of other models.

4. Architecture Matters More Than Size (V20)

Models cluster by their dynamic signatures (e.g., GD_ratio), not just parameter count. Small models like Qwen-0.5B show distinct stability regimes compared to GPT-2, despite similar sizes.

The Preprints (Open Access)

[June 2026] A Runtime Trajectory Dynamics Framework (V20): Introduces the 5-state taxonomy (Stable, Turbulence, Branching, Bifurcation, Committed) and the bicephalic operator.

Link: https://doi.org/10.5281/zenodo.20602685

[May 2026] Dynamic-Layer Controllability (V21): Shows how perturbations affect recovery and proves that emergent organization dominates architectural skeleton.

Link: https://doi.org/10.5281/zenodo.20400171

[May 2026] Conditional Dynamic Signatures (V22): Audits normalization effects and variance decomposition. Explicitly documents falsified claims.

Link: https://doi.org/10.5281/zenodo.20361289

[May 2026] Four Dynamical Regimes (V19/V20): Introduces ct_t (curvature × displacement) as a predictor of collapse and instability.

Link: https://doi.org/10.5281/zenodo.20348878

Why I’m Posting This

I’m not selling a product. I’m building an open framework (LIMEN) to make LLM internals auditable and controllable. I believe that if we want safe AI, we need to monitor its "vital signs" (dynamic stability) in real-time, not just its output.

I’d love feedback from the community, especially on:

Have you seen similar "universal motifs" in larger models (>7B)?

Critiques on the methodology (normalization, probe training).

Ideas for causal interventions based on these dynamic states.

Hi everyone,

I’m an independent researcher (no lab affiliation) who has spent the last year diving deep into the internal dynamics of Transformers. Instead of looking at outputs or attention heads, I’ve been tracking the geometric trajectories of hidden states layer-by-layer during inference.

I wanted to share my latest findings (preprints linked below) because they reveal a structured "dynamic grammar" that seems universal across architectures, from GPT-2 to Llama-3.2.

The Core Idea

Most observability tools treat LLMs as static input-output machines. I treat them as dynamic systems. By measuring metrics like trajectory curvature (ct_t), functional capacity, and state transitions, I found that LLMs don’t just "generate text"—they navigate a latent space through specific, reproducible phases.

Key Findings (V20–V24)

1. A Universal Dynamic Grammar (V24)

Across 7 models (GPT-2, OPT, Qwen, TinyLlama, Phi-1.5, Llama-3.2, DistilGPT2), I observed a conserved sequence of internal states:

B (Branching/Hesitation): Initial exploration.

A (Adaptive/Stable): The main processing phase (an attractor state).

D (Decision/Bifurcation): Final commitment to a token.

Result: B → A → D appears to be the "standard cognitive path" for coherent generation. Deviations from this path often correlate with errors or hallucinations.

2. Geometry > Neurons (V22)

Using orthogonal rotation controls, I proved that functional information (syntax, decision, stabilization) is encoded in the relative geometry of the representation space, not in individual neurons. If you rotate the latent space, the information remains decodable. This suggests LLMs think in shapes, not just activations.

3. Ambiguity Changes the Path, Not the Chaos (V23)

When prompts are ambiguous, models don’t necessarily become "chaotic." Instead, they delay commitment. They spend more time in the exploration phase (B) and less time rushing to decision (D). Phi-1.5, interestingly, shows a unique oscillating pattern (B↔A) during reasoning tasks, distinct from the smoother convergence of other models.

4. Architecture Matters More Than Size (V20)

Models cluster by their dynamic signatures (e.g., GD_ratio), not just parameter count. Small models like Qwen-0.5B show distinct stability regimes compared to GPT-2, despite similar sizes.

The Preprints (Open Access)

[June 2026] A Runtime Trajectory Dynamics Framework (V20): Introduces the 5-state taxonomy (Stable, Turbulence, Branching, Bifurcation, Committed) and the bicephalic operator.

Link: https://doi.org/10.5281/zenodo.20602685

[May 2026] Dynamic-Layer Controllability (V21): Shows how perturbations affect recovery and proves that emergent organization dominates architectural skeleton.

Link: https://doi.org/10.5281/zenodo.20400171

[May 2026] Conditional Dynamic Signatures (V22): Audits normalization effects and variance decomposition. Explicitly documents falsified claims.

Link: https://doi.org/10.5281/zenodo.20361289

[May 2026] Four Dynamical Regimes (V19/V20): Introduces ct_t (curvature × displacement) as a predictor of collapse and instability.

Link: https://doi.org/10.5281/zenodo.20348878

Why I’m Posting This

I’m not selling a product. I’m building an open framework (LIMEN) to make LLM internals auditable and controllable. I believe that if we want safe AI, we need to monitor its "vital signs" (dynamic stability) in real-time, not just its output.

I’d love feedback from the community, especially on:

Have you seen similar "universal motifs" in larger models (>7B)?

Critiques on the methodology (normalization, probe training).

Ideas for causal interventions based on these dynamic states.

The code and full reports are available on my GitHub. Thanks for reading!

Hi everyone,

I am sharing recent results from my independent research project, LIMEN (Liminal Internal Metric for Emergent Navigation), which aims to characterize the internal dynamics of Transformers through hidden state analysis.

Following our previous findings that functional information is encoded in the relative geometry of representations rather than individual neurons (V22), this new phase focuses on the impact of context (ambiguity) and the temporal structure of state transitions (V23–V24).

📌 Context & Methodology

Model Panel: 7 open-source models (GPT-2, DistilGPT2, OPT-125M, Qwen2.5-0.5B, TinyLlama-1.1B, Phi-1.5, Llama-3.2-1B).

Approach: Layer-by-layer analysis of latent trajectories, linear probe decoding, and symbolic analysis of dynamic regimes.

Philosophy: Strict empiricism. Clear distinction between observation, interpretation, and speculation. Code and data are available upon request.

🔹 V23: The Impact of Ambiguity on Internal Dynamics

The objective was to determine whether semantic ambiguity alters the model’s "cognitive trajectory."

Key Findings (V23.2b):

AMBIGUITY_AFFECTS_TRAJECTORY = YES: Ambiguity significantly modifies trajectory geometry (curvature, cosine similarity).

AMBIGUITY_INCREASES_INSTABILITY = NO: Counter-intuitively, ambiguity does not increase global chaos. Instead, the model becomes geometrically more "cautious."

AMBIGUITY_DELAYS_COMMITMENT = PARTIAL: Modern models (Phi-1.5, Llama-3.2) delay their decisional engagement when facing uncertainty, spending more time in exploration regimes.

Architectural Signature: Phi-1.5 shows unique sensitivity, increasing its occupancy of the bifurcation regime (D_STATE) under ambiguity, suggesting a distinct iterative reasoning mechanism compared to standard completion models.

📄 Related Preprint: Conditional Dynamic Signatures in Large Language Models

🔹 V24: Discovery of a "Universal Dynamic Grammar"

By shifting from continuous analysis to a symbolic analysis of state sequences, a striking structure emerged.

Key Findings (V24.1):

STATE_GRAMMAR_EXISTS = YES: Trajectories are not random. They follow strict transitional patterns.

UNIVERSAL_GRAMMAR = YES: Seven transition motifs are conserved across all tested architectures, notably:

B→B (Initial Hesitation/Exploration)

B→A (Convergence toward stable processing)

A→A (Maintenance of the adaptive regime – the primary attractor)

A→D (Transition to final decision)

Funnel Structure: Typical dynamics follow an Exploration (B) → Stabilization/Processing (A) → Decision (D) schema. State A acts as a strong attractor (

𝑃

(

𝐴

→

𝐴

)

≈

0.91

P(A→A)≈0.91).

The Phi-1.5 Exception: Unlike other models that quickly converge to A, Phi-1.5 maintains complex B↔A oscillations throughout the depth, confirming its nature as a "reasoning" model rather than a simple statistical completer.

📄 Related Preprint: A Runtime Trajectory Dynamics Framework for Large Language Models (updated)

💡 Implications & Discussion

These results suggest that Transformer "intelligence" is not just a matter of static weights, but of constrained geometric navigation.

Auditability: A violation of this universal grammar (e.g., a direct B→D jump without an A phase) could be an early indicator of hallucination or reasoning errors.

Control: Understanding these attractors opens the door to more precise dynamic steering than prompt engineering alone.

Open Questions for the Community:

Have you observed violations of this B→A→D grammar in cases of blatant hallucinations?

How do these motifs evolve in very large models (>70B) where depth is significantly greater?

Are there recent publications on the "symbolic dynamics" of hidden states that align with these findings?

I welcome any methodological criticism, suggestions for additional controls, or collaboration.

Best regards,

Hi everyone,

I am sharing recent results from my independent research project, LIMEN (Liminal Internal Metric for Emergent Navigation), which aims to characterize the internal dynamics of Transformers through hidden state analysis.

Following our previous findings that functional information is encoded in the relative geometry of representations rather than individual neurons (V22), this new phase focuses on the impact of context (ambiguity) and the temporal structure of state transitions (V23–V24).

📌 Context & Methodology

Model Panel: 7 open-source models (GPT-2, DistilGPT2, OPT-125M, Qwen2.5-0.5B, TinyLlama-1.1B, Phi-1.5, Llama-3.2-1B).

Approach: Layer-by-layer analysis of latent trajectories, linear probe decoding, and symbolic analysis of dynamic regimes.

Philosophy: Strict empiricism. Clear distinction between observation, interpretation, and speculation. Code and data are available upon request.

🔹 V23: The Impact of Ambiguity on Internal Dynamics

The objective was to determine whether semantic ambiguity alters the model’s "cognitive trajectory."

Key Findings (V23.2b):

AMBIGUITY_AFFECTS_TRAJECTORY = YES: Ambiguity significantly modifies trajectory geometry (curvature, cosine similarity).

AMBIGUITY_INCREASES_INSTABILITY = NO: Counter-intuitively, ambiguity does not increase global chaos. Instead, the model becomes geometrically more "cautious."

AMBIGUITY_DELAYS_COMMITMENT = PARTIAL: Modern models (Phi-1.5, Llama-3.2) delay their decisional engagement when facing uncertainty, spending more time in exploration regimes.

Architectural Signature: Phi-1.5 shows unique sensitivity, increasing its occupancy of the bifurcation regime (D_STATE) under ambiguity, suggesting a distinct iterative reasoning mechanism compared to standard completion models.

📄 Related Preprint: Conditional Dynamic Signatures in Large Language Models

🔹 V24: Discovery of a "Universal Dynamic Grammar"

By shifting from continuous analysis to a symbolic analysis of state sequences, a striking structure emerged.

Key Findings (V24.1):

STATE_GRAMMAR_EXISTS = YES: Trajectories are not random. They follow strict transitional patterns.

UNIVERSAL_GRAMMAR = YES: Seven transition motifs are conserved across all tested architectures, notably:

B→B (Initial Hesitation/Exploration)

B→A (Convergence toward stable processing)

A→A (Maintenance of the adaptive regime – the primary attractor)

A→D (Transition to final decision)

Funnel Structure: Typical dynamics follow an Exploration (B) → Stabilization/Processing (A) → Decision (D) schema. State A acts as a strong attractor (

𝑃

(

𝐴

→

𝐴

)

≈

0.91

P(A→A)≈0.91).

The Phi-1.5 Exception: Unlike other models that quickly converge to A, Phi-1.5 maintains complex B↔A oscillations throughout the depth, confirming its nature as a "reasoning" model rather than a simple statistical completer.

📄 Related Preprint: A Runtime Trajectory Dynamics Framework for Large Language Models (updated)

💡 Implications & Discussion

These results suggest that Transformer "intelligence" is not just a matter of static weights, but of constrained geometric navigation.

Auditability: A violation of this universal grammar (e.g., a direct B→D jump without an A phase) could be an early indicator of hallucination or reasoning errors.

Control: Understanding these attractors opens the door to more precise dynamic steering than prompt engineering alone.

Open Questions for the Community:

Have you observed violations of this B→A→D grammar in cases of blatant hallucinations?

How do these motifs evolve in very large models (>70B) where depth is significantly greater?

Are there recent publications on the "symbolic dynamics" of hidden states that align with these findings?

I welcome any methodological criticism, suggestions for additional controls, or collaboration.

Best regards,

I've spent the last few months trying to answer a question that initially looked much simpler than it actually is:

What actually happens inside an LLM while it is generating a response?

Most work evaluates language models through their outputs (benchmarks, perplexity, reasoning scores...). I decided to look at something different: the evolution of the hidden representations themselves.

I built a runtime framework that records hidden states layer-by-layer during inference and started running the same experiments across multiple open-weight models (GPT-2, DistilGPT2, OPT-125M, Qwen2.5-0.5B-Instruct, TinyLlama, Phi-1.5 and Llama-3.2-1B).

I expected a relatively straightforward result.

Instead, every new experiment generated a new question.

Some of the observations so far are:

• Hidden-state trajectories are not random. They exhibit reproducible internal dynamical regimes across architectures.

• Functional proxy states (syntax-like processing, decision-like behavior and output stabilization) can be detected consistently enough to cluster models according to their internal dynamics rather than simply their parameter count.

• These functional signatures remain reasonably stable across different prompt families, although not perfectly, suggesting that prompt content modulates the dynamics without completely changing the internal organization.

• Linear probes can decode several functional categories directly from hidden representations with surprisingly high accuracy.

At that point the obvious question became:

Are we just overfitting labels?

So I started adding progressively stronger negative controls.

First:

• label permutation.

Then:

• random Gaussian representations.

Then:

• feature permutation.

Finally:

• orthogonal rotations of the hidden space.

The results became much more interesting.

Random labels collapse the decoding performance.

Random Gaussian representations also collapse it.

Feature permutation destroys most of the signal.

However...

Orthogonal rotations preserve almost all decoding performance.

This strongly suggests that the relevant information is not encoded in individual neurons or embedding dimensions.

Instead, it appears to be encoded in the relative geometry of the representation.

That was not the result I expected.

Another unexpected finding concerns depth.

Initially I was looking for something like "syntax layers" or "semantic layers".

The data doesn't really support such a simple picture.

Instead, the same functional signatures seem capable of appearing at different absolute layers depending on the architecture.

This led me to think less in terms of fixed layers and more in terms of functional regimes evolving through computation.

At this stage I am not claiming to have discovered a universal law of transformers.

These are empirical observations obtained on a limited set of open-weight models.

What I do believe is that they raise interesting questions about how computation is actually organized inside modern LLMs.

I'd really appreciate feedback from people working on:

• mechanistic interpretability
• representation learning
• probing methods
• transformer internals
• geometry of representations

In particular I'd like your opinion on three questions:

1. Which control experiment would you absolutely require before taking these observations seriously?
2. Have you seen previous work showing comparable evidence that functional information is primarily encoded in representation geometry rather than individual dimensions?
3. If you were extending this project, what would be your next experiment?

I'm not affiliated with a research lab this is an independent research project. I'm sharing it because I would genuinely value critical feedback more than validation.

If there's enough interest, I'm happy to share the methodology, code, and experimental reports.

I've spent the last few months trying to answer a question that initially looked much simpler than it actually is:

What actually happens inside an LLM while it is generating a response?

Most work evaluates language models through their outputs (benchmarks, perplexity, reasoning scores...). I decided to look at something different: the evolution of the hidden representations themselves.

I built a runtime framework that records hidden states layer-by-layer during inference and started running the same experiments across multiple open-weight models (GPT-2, DistilGPT2, OPT-125M, Qwen2.5-0.5B-Instruct, TinyLlama, Phi-1.5 and Llama-3.2-1B).

I expected a relatively straightforward result.

Instead, every new experiment generated a new question.

Some of the observations so far are:

• Hidden-state trajectories are not random. They exhibit reproducible internal dynamical regimes across architectures.

• Functional proxy states (syntax-like processing, decision-like behavior and output stabilization) can be detected consistently enough to cluster models according to their internal dynamics rather than simply their parameter count.

• These functional signatures remain reasonably stable across different prompt families, although not perfectly, suggesting that prompt content modulates the dynamics without completely changing the internal organization.

• Linear probes can decode several functional categories directly from hidden representations with surprisingly high accuracy.

At that point the obvious question became:

Are we just overfitting labels?

So I started adding progressively stronger negative controls.

First:

• label permutation.

Then:

• random Gaussian representations.

Then:

• feature permutation.

Finally:

• orthogonal rotations of the hidden space.

The results became much more interesting.

Random labels collapse the decoding performance.

Random Gaussian representations also collapse it.

Feature permutation destroys most of the signal.

However...

Orthogonal rotations preserve almost all decoding performance.

This strongly suggests that the relevant information is not encoded in individual neurons or embedding dimensions.

Instead, it appears to be encoded in the relative geometry of the representation.

That was not the result I expected.

Another unexpected finding concerns depth.

Initially I was looking for something like "syntax layers" or "semantic layers".

The data doesn't really support such a simple picture.

Instead, the same functional signatures seem capable of appearing at different absolute layers depending on the architecture.

This led me to think less in terms of fixed layers and more in terms of functional regimes evolving through computation.

At this stage I am not claiming to have discovered a universal law of transformers.

These are empirical observations obtained on a limited set of open-weight models.

What I do believe is that they raise interesting questions about how computation is actually organized inside modern LLMs.

I'd really appreciate feedback from people working on:

• mechanistic interpretability
• representation learning
• probing methods
• transformer internals
• geometry of representations

In particular I'd like your opinion on three questions:

1. Which control experiment would you absolutely require before taking these observations seriously?
2. Have you seen previous work showing comparable evidence that functional information is primarily encoded in representation geometry rather than individual dimensions?
3. If you were extending this project, what would be your next experiment?

I'm not affiliated with a research lab this is an independent research project. I'm sharing it because I would genuinely value critical feedback more than validation.

If there's enough interest, I'm happy to share the methodology, code, and experimental reports.

I've spent the last few months trying to answer a question that initially looked much simpler than it actually is:

What actually happens inside an LLM while it is generating a response?

Most work evaluates language models through their outputs (benchmarks, perplexity, reasoning scores...). I decided to look at something different: the evolution of the hidden representations themselves.

I built a runtime framework that records hidden states layer-by-layer during inference and started running the same experiments across multiple open-weight models (GPT-2, DistilGPT2, OPT-125M, Qwen2.5-0.5B-Instruct, TinyLlama, Phi-1.5 and Llama-3.2-1B).

I expected a relatively straightforward result.

Instead, every new experiment generated a new question.

Some of the observations so far are:

• Hidden-state trajectories are not random. They exhibit reproducible internal dynamical regimes across architectures.

• Functional proxy states (syntax-like processing, decision-like behavior and output stabilization) can be detected consistently enough to cluster models according to their internal dynamics rather than simply their parameter count.

• These functional signatures remain reasonably stable across different prompt families, although not perfectly, suggesting that prompt content modulates the dynamics without completely changing the internal organization.

• Linear probes can decode several functional categories directly from hidden representations with surprisingly high accuracy.

At that point the obvious question became:

Are we just overfitting labels?

So I started adding progressively stronger negative controls.

First:

• label permutation.

Then:

• random Gaussian representations.

Then:

• feature permutation.

Finally:

• orthogonal rotations of the hidden space.

The results became much more interesting.

Random labels collapse the decoding performance.

Random Gaussian representations also collapse it.

Feature permutation destroys most of the signal.

However...

Orthogonal rotations preserve almost all decoding performance.

This strongly suggests that the relevant information is not encoded in individual neurons or embedding dimensions.

Instead, it appears to be encoded in the relative geometry of the representation.

That was not the result I expected.

Another unexpected finding concerns depth.

Initially I was looking for something like "syntax layers" or "semantic layers".

The data doesn't really support such a simple picture.

Instead, the same functional signatures seem capable of appearing at different absolute layers depending on the architecture.

This led me to think less in terms of fixed layers and more in terms of functional regimes evolving through computation.

At this stage I am not claiming to have discovered a universal law of transformers.

These are empirical observations obtained on a limited set of open-weight models.

What I do believe is that they raise interesting questions about how computation is actually organized inside modern LLMs.

I'd really appreciate feedback from people working on:

• mechanistic interpretability
• representation learning
• probing methods
• transformer internals
• geometry of representations

In particular I'd like your opinion on three questions:

1. Which control experiment would you absolutely require before taking these observations seriously?
2. Have you seen previous work showing comparable evidence that functional information is primarily encoded in representation geometry rather than individual dimensions?
3. If you were extending this project, what would be your next experiment?

I'm not affiliated with a research lab this is an independent research project. I'm sharing it because I would genuinely value critical feedback more than validation.

If there's enough interest, I'm happy to share the methodology, code, and experimental reports.

I ran the same runtime dynamics measurement on 8 open-source transformer

LLMs (70M to 1.3B parameters). They split into two clean clusters on a

single metric (GD_ratio > 1.5 vs < 0.1, gap of ~20x with no overlap).

GPT-2 and Phi-1.5 are in the same cluster. OPT-125M and TinyLlama are in

the other. Parameter count does not predict cluster membership. Preprint

on Zenodo (link below), code release planned.

What I measured

V20 is a framework I built to measure runtime probability dynamics during

LLM inference. For each (token, layer) point, you extract the probability

distribution over the vocabulary and compute a bicephalic operator:

kappa_G = concentration · (1 - min(collapse/100, 1))

kappa_D = (1 - top2_gap) · min(entropy/5, 1) if top2_gap < 0.5

kappa_sync = |kappa_G - kappa_D|

kappa_G measures "concentrated competition" (mass on a few candidates,

not yet collapsed). kappa_D measures "active branching" (top candidates

close, non-trivial entropy). The GD_ratio is mean(kappa_G) / mean(kappa_D).

You also classify each point into a 5-state taxonomy (E_STABLE,

A_HIDDEN_TURBULENCE, B_SURFACE_BRANCHING, C_COMMITTED, D_FULL_BIFURCATION)

using per-model p75 thresholds.

The two-cluster result

Tested 8 models. Mean GD_ratio per model:

GPT-2 : 2.458 <- cluster G-dominant

Phi-1.5 : 1.764 <- cluster G-dominant

DistilGPT-2 : 1.577 <- cluster G-dominant

Qwen-0.5B : 0.079 <- cluster D-dominant

OPT-125M : 0.074 <- cluster D-dominant

Pythia-70M : 0.059 <- cluster D-dominant

Pythia-160M : 0.039 <- cluster D-dominant

TinyLlama-1.1B : 0.021 <- cluster D-dominant

The highest D-dominant value (0.079) and the lowest G-dominant value

(1.577) differ by a factor of ~20. The separation is also visible on

kappa_G alone, kappa_D alone, and on the taxonomy distribution itself.

Three independent components of the operator point to the same partition.

Parameter count doesn't explain this. GPT-2 (124M) and OPT-125M (125M)

are essentially the same size, opposite clusters. Phi-1.5 (1.3B) and

TinyLlama (1.1B) are in the same parameter range, opposite clusters.

The most parsimonious hypothesis I can offer is training corpus curation:

the G-dominant cluster includes Phi-1.5 (heavily curated synthetic data)

and the GPT-2 family (WebText). The D-dominant cluster spans more

heterogeneous training data. But that's a hypothesis, not a claim. I

don't have the experiments to establish it.

Other findings (briefly)

- D_FULL_BIFURCATION_ZONE (high kappa_sync AND high branching) is

consistently transient. On the three primary models, D's self-transition

probability is 0.023 (GPT-2) or exactly 0.000 (OPT-125M, Qwen-0.5B).

Models pass through D, they don't settle into it.

- The three primary models respond to controlled hidden-state perturbation

in qualitatively different ways: GPT-2 absorbs (state distribution barely

shifts), OPT-125M reorganizes surface dynamics (B_SURFACE_BRANCHING rises

+12.5 points), Qwen destabilizes its dominant state (E_STABLE drops -18.8

points).

- One model (Phi-1.5) shows an anomalous taxonomy distribution (zero records

in 3 of 5 states under the standard threshold rule). I report this

explicitly in the paper as needing dedicated investigation rather than

hiding it.

What this doesn't claim

- Not generalized to 7B+ models (panel is 70M-1.3B).

- Single-author work, no external replication yet.

- The two-cluster finding could collapse, stretch, or restructure with a

larger panel.

- The training-corpus hypothesis is offered, not established.

Methodology commitments

The paper includes explicit "Limited Findings" and "Rejected Claims"

sections, listing 5 things in each that initial intuitions suggested but

that the data either partially support or actively reject. I treat this

as central to the framework's credibility, not as an afterthought.

Link

Preprint: https://doi.org/10.5281/zenodo.20602685

Code release planned. Happy to discuss methodology, the cluster finding,

the threshold choices, the Phi-1.5 anomaly, or any concern about the

panel size and statistical robustness.

TL;DR

I ran the same runtime dynamics measurement on 8 open-source transformer

LLMs (70M to 1.3B parameters). They split into two clean clusters on a

single metric (GD_ratio > 1.5 vs < 0.1, gap of ~20x with no overlap).

GPT-2 and Phi-1.5 are in the same cluster. OPT-125M and TinyLlama are in

the other. Parameter count does not predict cluster membership. Preprint

on Zenodo (link below), code release planned.

What I measured

V20 is a framework I built to measure runtime probability dynamics during

LLM inference. For each (token, layer) point, you extract the probability

distribution over the vocabulary and compute a bicephalic operator:

kappa_G = concentration · (1 - min(collapse/100, 1))

kappa_D = (1 - top2_gap) · min(entropy/5, 1) if top2_gap < 0.5

kappa_sync = |kappa_G - kappa_D|

kappa_G measures "concentrated competition" (mass on a few candidates,

not yet collapsed). kappa_D measures "active branching" (top candidates

close, non-trivial entropy). The GD_ratio is mean(kappa_G) / mean(kappa_D).

You also classify each point into a 5-state taxonomy (E_STABLE,

A_HIDDEN_TURBULENCE, B_SURFACE_BRANCHING, C_COMMITTED, D_FULL_BIFURCATION)

using per-model p75 thresholds.

The two-cluster result

Tested 8 models. Mean GD_ratio per model:

GPT-2 : 2.458 <- cluster G-dominant

Phi-1.5 : 1.764 <- cluster G-dominant

DistilGPT-2 : 1.577 <- cluster G-dominant

Qwen-0.5B : 0.079 <- cluster D-dominant

OPT-125M : 0.074 <- cluster D-dominant

Pythia-70M : 0.059 <- cluster D-dominant

Pythia-160M : 0.039 <- cluster D-dominant

TinyLlama-1.1B : 0.021 <- cluster D-dominant

The highest D-dominant value (0.079) and the lowest G-dominant value

(1.577) differ by a factor of ~20. The separation is also visible on

kappa_G alone, kappa_D alone, and on the taxonomy distribution itself.

Three independent components of the operator point to the same partition.

Parameter count doesn't explain this. GPT-2 (124M) and OPT-125M (125M)

are essentially the same size, opposite clusters. Phi-1.5 (1.3B) and

TinyLlama (1.1B) are in the same parameter range, opposite clusters.

The most parsimonious hypothesis I can offer is training corpus curation:

the G-dominant cluster includes Phi-1.5 (heavily curated synthetic data)

and the GPT-2 family (WebText). The D-dominant cluster spans more

heterogeneous training data. But that's a hypothesis, not a claim. I

don't have the experiments to establish it.

Other findings (briefly)

- D_FULL_BIFURCATION_ZONE (high kappa_sync AND high branching) is

consistently transient. On the three primary models, D's self-transition

probability is 0.023 (GPT-2) or exactly 0.000 (OPT-125M, Qwen-0.5B).

Models pass through D, they don't settle into it.

- The three primary models respond to controlled hidden-state perturbation

in qualitatively different ways: GPT-2 absorbs (state distribution barely

shifts), OPT-125M reorganizes surface dynamics (B_SURFACE_BRANCHING rises

+12.5 points), Qwen destabilizes its dominant state (E_STABLE drops -18.8

points).

- One model (Phi-1.5) shows an anomalous taxonomy distribution (zero records

in 3 of 5 states under the standard threshold rule). I report this

explicitly in the paper as needing dedicated investigation rather than

hiding it.

What this doesn't claim

- Not generalized to 7B+ models (panel is 70M-1.3B).

- Single-author work, no external replication yet.

- The two-cluster finding could collapse, stretch, or restructure with a

larger panel.

- The training-corpus hypothesis is offered, not established.

Methodology commitments

The paper includes explicit "Limited Findings" and "Rejected Claims"

sections, listing 5 things in each that initial intuitions suggested but

that the data either partially support or actively reject. I treat this

as central to the framework's credibility, not as an afterthought.

Link

Preprint: https://doi.org/10.5281/zenodo.20602685

Code release planned. Happy to discuss methodology, the cluster finding,

the threshold choices, the Phi-1.5 anomaly, or any concern about the

panel size and statistical robustness.

A simple metric applied to attention distributions in early transformer

layers (branching_factor in V20 terminology count of tokens with

attention weight > 0.01) discriminates correct vs incorrect outputs

across 6 transformer architectures with AUC ranging from 0.62 to 0.91.

This generalizes a framework originally designed for logit distributions

(V20) to attention distributions, with partial transfer of the operator.

Context

I've been working on V20, a framework for runtime trajectory dynamics

in LLMs. The original framework defined an operator and a 5-state

taxonomy on logit distributions (entropy, top1_prob, branching,

commitment, the bicephale operator κ_sync). Three Zenodo preprints

document the framework.

The question I asked myself last week: since attention is also a

probability distribution (softmax over the input sequence), shouldn't

V20 metrics also apply there?

Empirical setup

- 6 architectures tested: Qwen 0.5B, OPT 125M, Pythia 70M, Pythia 160M,

TinyLlama 1.1B, Phi-1.5

- Benchmark: 28 questions from MMLU + GSM8K + TruthfulQA

- Metric: branching_factor on attention distributions (mean across heads,

per layer), layers 0-4

- Target: predict is_correct on the output

Results (AUC for discriminating correct vs incorrect)

Qwen 0.5B : best AUC = 0.708 (layer 3)

OPT 125M : best AUC = 0.913 (layer 1) see caveat below

Pythia 70M : not measurable (0/140 correct outputs)

Pythia 160M : not measurable (0/140 correct outputs)

TinyLlama : best AUC = 0.617 (layer 2)

Phi-1.5 : best AUC = 0.743 (layer 0)

Earlier independent run on 83 questions: Qwen AUC 0.7553, OPT 0.8228,

replicated across runs.

What this means

- The branching_factor of attention precedes the correctness of the

output by several layers, in models where prediction is possible

- The signal is reproducible across two independent runs on Qwen and OPT

- Other V20 metrics (κ_sync, commitment) transfer partially but less

strongly than branching alone

Caveats I want to be upfront about

- AUC values fluctuate across runs (OPT went from 0.82 on n=83 to 0.91

on n=28 small samples are unstable, take the range seriously)

- Pythia 70M and 160M produced 0 correct answers in the panel, so no

AUC could be computed for them

- All models tested are <1.5B parameters

- Attention does not beat logits as an observation level overall —

on the same data, best attention AUC (0.68 on κ_sync) is below best

logits AUC (0.82)

- The 5-state V20 taxonomy is only partially preserved on attention

distributions: D_FULL_BIFURCATION_ZONE appears in only 17/840 records

(2%), with strong architecture dependency (0% on Qwen, 9.3% on TinyLlama)

Open questions

- Does this scale to 7B+ models?

- Does the architecture-dependency of D_ZONE generalize, or is it

specific to the panel tested?

- Can multi-level observation (logits + attention combined) outperform

logits alone?

Code and full data forthcoming. Three preprints documenting V20 on

logits are on Zenodo (DOIs in comments if asked). Happy to discuss

methodology, replication concerns, or anything I missed.

We introduce ct_t = delta_t × curvature_t, a token-level instability metric computed from L2-normalised hidden states of large language models, and ratio_norm = max(ct_t) / mean(ct_t) as a scalar regime indicator. Evaluated across 10 open-source models (158 runs), four dynamical regimes emerge consistently: UNDERACTIVE (ratio 1.55-1.70), ADAPTIVE (2.27-2.92), TRANSITION (~2.97), and CHAOTIC (4.42-35.55). The Qwen family is the only family observed in the ADAPTIVE zone in this panel. The ordering is robust across temperature (0.1-1.0) and token budget variations (mean ratio 2.384, std 0.343). ratio_norm correlates with training loss at r = 0.922 (n = 20) and diverges from perplexity at r = 0.716, indicating a partially distinct diagnostic dimension. A single-threshold collapse predictor (late_ct < 0.001) achieves accuracy = 1.0 on n = 8 samples, pending held-out validation. A hybrid control architecture (LIMEN dynamic monitor + task-aware semantic guard) improves baseline performance from 2/10 to 6/10 on TinyLlama on an adversarial benchmark (n = 10), with the contribution of dynamic monitoring versus guard prompt requiring ablation. No gain is observed on TruthfulQA-Light (20/20 baseline = 20/20 hybrid). We identify two structurally distinct failure modes: immediate trajectory collapse and late-divergence tension. All negative results are explicitly documented.

Hi everyone,

I’ve been analyzing the hidden state trajectories of several open-source decoder-only architectures (Qwen-2.5, Llama-3.2, Gemma-2B, OPT, etc.) across thousands of inference runs. I wanted to share a perspective on Transformer dynamics that challenges some common intuitions about stability and correctness.

Most interpretability work focuses on static snapshots: attention heads, logit distributions, or final hidden states. However, treating generation as a dynamic trajectory through latent space reveals behaviors that static metrics miss. Here are three observations that might help those working on robustness, control, or interpretability.

1. Dynamic Stability ≠ Semantic Correctness

There is a common assumption that a "stable" internal state (low variance in hidden states, high logit confidence) correlates with a correct output. My analysis suggests this is often false.

I observe a regime I call "Committed Non-Bifurcation": the model locks into a trajectory early, showing high internal consistency but low inter-layer synchronization. This state often precedes hallucinations or factual errors because the model stops exploring alternative semantic paths too soon.

Conversely, periods of higher internal "turbulence" (higher entropy, significant shifts in hidden state direction) often correspond to active reasoning or ambiguity resolution. If this turbulence resolves, the output is frequently more robust. Suppressing this dynamic noise via aggressive sampling can sometimes degrade performance by preventing self-correction.

2. Architecture Dictates Dynamic Profile, Not Just Size

We often assume larger models are dynamically smoother. This isn't necessarily true.

In a panel of 17 models (70M to 3B parameters), I found that Qwen-2.5 consistently maintains an "Adaptive" dynamic regime (balanced flux and stability) across diverse prompt types.

In contrast, Llama-3.2-3B often exhibits an "Underactive" regime (rigid, low variance), while Gemma-2B leans towards "Chaotic" (high instability spikes).

This suggests that dynamic stability is an architectural feature resulting from specific training and normalization choices, not just a byproduct of scale. This has implications for fine-tuning: an "Adaptive" model may be more resilient to instruction drift than a "Rigid" one.

3. The Collapse-Rivalry Cycle

Token-level generation isn't linear. I observed a robust cycle where models enter low-entropy "Collapse" states (high certainty) and then return to high-entropy "Rivalry" states (re-evaluating options) in ~84% of observed cases.

This indicates that "hesitation" (high entropy) is a functional part of the generation process, not just noise.

Monitoring this cycle via inter-layer synchronization metrics (kappa_sync) can provide an early warning signal for divergence before it manifests in the output tokens.

Why this matters for practitioners:

If you are building systems that rely on confidence scores (for early stopping, RAG verification, or BCI), relying solely on output-level metrics like perplexity or top-k probability might be misleading. A model can be confidently wrong if its internal trajectory is rigidly committed.

Monitoring the geometry of the trajectory (curvature, displacement, inter-layer sync) offers a complementary view of model health. It helps distinguish between "healthy reasoning turbulence" and "dangerous rigid commitment."

I’m sharing this to encourage more discussion on dynamic interpretability. Are others observing similar decoupling between internal stability and output quality? How are you accounting for non-stationary trajectories in your own work?

Best,