u/Desperate-Ad82
Benchmarking Foundation Models (CHGNet, MACE) for Band Gap Prediction — Why they struggle and how 11D spatial message passing fixes it.
reddit.comBenchmarking Foundation Models (CHGNet, MACE) for Band Gap Prediction — Why they struggle and how 11D spatial message passing fixes it.
>
This version invites discussion, shares raw metrics, removes the corporate branding, and asks the community a question at the end. It is much more likely to survive the moderators and spark a great technical debate.Hey everyone, I’ve been doing some deep architectural testing lately on predicting localized electronic properties (specifically band gaps), and I wanted to share some benchmarking results that might be useful if you're working with ML in materials science.
There is a huge push right now to use foundation models like MACE-MP-0 and CHGNet as silver bullets. They are undeniably incredible for total energy and interatomic force predictions. But I wanted to see what happens if we freeze their embeddings to predict band gaps.
The TL;DR: They failed pretty hard in my benchmarks.
I ran a strict, leakage-free empirical benchmark (ensuring formula-disjoint segregation so the test set was completely unseen). I compared frozen embeddings from CHGNet and MACE against a custom-built Graph Neural Network (GNN).
The custom GNN outperformed the foundation models by a massive 78% margin (achieving a Test MAE of 0.3389 eV).
Why the huge difference?
It comes down to how the architectures handle local orbitals. Foundation models pool and compress structural features, which inherently strips away the geometric sensitivity of localized orbitals.
To fix this, I engineered a custom GNN natively with 11-dimensional electronic features. Instead of just looking at coordinates, it explicitly reads electron configurations across $s, p, d,$ and $f$ orbitals and propagates this data via spatial message passing.
The Sanity Check (TiO2):
To prove the model wasn't just guessing based on stoichiometry, I tested it on Titanium Dioxide polymorphs. The model successfully distinguished between Rutile and Anatase phases with a 0.182 eV differential, recognizing the different spatial arrangements despite identical chemical formulas.
I’ve attached a video of the UI I built for the engine processing this in real-time.
Has anyone else run into this specific limitation when trying to repurpose structural foundation models for electronic/quantum properties? Curious to hear how others are handling orbital sensitivity in their pipelines!This version invites discussion, shares raw metrics, removes the corporate branding, and asks the community a question at the end. It is much more likely to survive the moderators and spark a great technical debate.
I built an offline Inverse Design / Generative Algorithm for materials that outputs deterministic synthesis protocols (Graph Neural Networks + Heuristics).
Hey everyone,
A while ago I shared the predictive side of my local offline MoE engine for materials science. The biggest feedback was: "Predicting is cool, but can it generate a novel material based on a target parameter, and actually tell a human how to make it safely?"
So, I built the Inverse Design (Generative Algorithm) pipeline into the Sovereign Materials Engine.
In the attached screenshots, I set a strict target: Find/Evolve a material with a 4.83 eV Band Gap.
The engine generated a scatter plot of candidates hitting the mark. Let's look at the top candidate: $Na_3Ca_3Y(SO_4)_6$.
Here is how the architecture handles the backend and the physical reality:
- The MoE Router & GNN: The smart router dynamically assigned this complex crystal to the GNN Expert (Deep Spatial). I included a SHAP dashboard specifically calculating the feature importance for the GNN's prediction, so it's not a black box.
- The "Redox Crucible Trap" Heuristic: This is my favorite part of the synthesis agent. It detects that the generated structure has both an Alkali metal ($Na$) AND Oxygen. Standard logic says "use Tantalum to avoid alkali attack on Alumina". But at high temps, Tantalum acts as an oxygen getter and will destroy the oxide! So, the engine strictly overrides the protocol to mandate a Silver (Ag), Gold (Au), or Nickel (Ni) crucible.
- Hygroscopic Awareness: It flags the Calcium and Yttrium precursors, enforcing a 900°C pre-drying step to ensure the stoichiometry doesn't get ruined by absorbed $H_2O/CO_2$.
Everything runs completely offline to protect IP.
I’m trying to make the synthesis agent as bulletproof as possible before I put this in the hands of actual lab directors. For the solid-state chemists here: are there any other extreme edge-cases involving rare-earths or complex sulfates I should hard-code into the ruleset?
I built an offline, MoE-based Materials Informatics platform that actually writes deterministic, lab-safe synthesis protocols (No more LLM hallucinations).
Hey everyone,
I’ve been working on a massive bottleneck in computational materials science: predicting a novel material is easy, but generating a physically realistic, safe synthesis protocol for it is incredibly hard.
Most generative AI tools out there act as black boxes and output "hallucinated" chemistry—suggesting things like mixing pure alkali metals with halogens in sealed ampoules, or heating toxic oxides in open air.
So, I built the Sovereign Materials Engine (v8). It runs 100% offline (0 API calls) to protect IP, and it relies heavily on deterministic heuristics rather than just language models.
Here is what the architecture looks like in action (screenshots attached for a stress-test on $NaSbF_6$):
1. The "Chief Scientist" Synthesis Agent (Image 2)
I hard-coded rigorous thermodynamic and safety heuristics into the generation pipeline:
- Crucible Intelligence: For $NaSbF_6$, it immediately recognizes that the Alkali metal ($Na$) will destroy an Alumina crucible and strictly overrides the environment to a Tantalum (Ta) or Niobium (Nb) tube.
- Precursor Overrides: It refuses to use pure $F$ gas. Instead, it dynamically selects $CaF_2$ as a safe counter-ion source, with a strict reactivity warning about mixing alkalis and halogens.
- Vapor Pressure Safety: It flags the internal vapor pressure generated by $Sb$ and enforces a strict ampoule filling fraction (< 20%) and a slow heating ramp to prevent quartz explosion.
2. MoE with Transparent Explainability (Image 1)
We are using a Mixture of Experts (MoE) approach for property prediction (like Band Gap). But because researchers hate black boxes, I integrated a SHAP Waterfall dashboard.
- You can see exactly why the XGBoost High-Gap Expert predicted 4.96 eV (mapping orbital hybridization, electronegativity variance, etc.).
- Note on the UI: I added a specific disclaimer at the bottom to clarify that the SHAP plot explains only the XGBoost V8 expert (4.96 eV), while the final blended MoE prediction shown at the top is 4.83 eV. No hidden math.
I’m currently running more stress tests. If any solid-state chemists or ML engineers here have specific edge-case nightmares (hygroscopic traps, rare toxic volatilizations, crucible incompatibilities) that I should hard-code into the safety protocols, I’d love to hear them!
(Attachments: UI Dashboard showing SHAP Waterfall + Deterministic Synthesis Protocol for NaSbF6).
Next-Gen Sovereign Materials Informatics platform
🚀 AI Is Not Enough: How We Built an Autonomous Engine for 3D Material Design and Lab-Ready Synthesis
In the revolution of advanced materials and semiconductor discovery, many AI researchers fall into the trap of "mathematical perfection." Current generative models can invent crystal structures that achieve the exact target Band Gap with stunning precision. But what if the AI-proposed material designed to solve the clean energy crisis contains radioactive Uranium? Or what if it's mathematically stable, but researchers have absolutely no idea how to synthesize it in the real world?
In real-world laboratories, physical stability is worthless if manufacturing the material threatens lives, drains R&D budgets, or lacks a viable chemical recipe.
Today, we bridge this massive gap. I am thrilled to announce the completion of the Next-Gen Sovereign Materials Informatics platform: a fully offline, deterministic AI engine that leaps from mere theoretical prediction to 3D spatial crystallography and practical, lab-ready synthesis.
⚠️ The Illusion of "Chemical Hallucinations"
During an early stress test targeting a 2.05 eV band gap, our Genetic Algorithm proposed dynamically stable materials like Cs₄U₂V₂O₁₃. Mathematically and physically, it was a groundbreaking invention. But chemically? It contained Uranium. Synthesizing it would require strict radiation protection protocols. The engine was "smart" as a machine, but it lacked the practical "wisdom" of a Lab Director.
🛡️ The 5-Pillar Autonomous Pipeline
To solve this, we re-engineered the infrastructure, passing every novel invention through a strict evaluation pipeline that simulates an entire integrated research team operating in milliseconds:
- 🧠 The Theoretical Physicist (AI MoE Router): Ensures electronic properties perfectly match the target (e.g., targeting exactly 2.16 eV).
- ⚖️ The Thermodynamic Expert (Stability Oracle): Examines the Energy Above Hull. If it exceeds 0.05 eV/atom, an algorithmic "Death Penalty" is applied.
- 🧪 The Lab Director (Sustainability Filter): Instantly kills structures with radioactive elements (U, Th) and heavily penalizes toxic (As, Pb) or exorbitantly expensive elements (Te, Os).
- 🧊 The Crystallographer (Ab-Initio 3D Builder) [NEW]: The AI doesn't just output a formula; it calculates spatial symmetry, packs the atoms into the correct Space Group, and generates an interactive, collision-free 3D unit cell (CIF file) to prove structural viability.
- 👨🔬 The Solid-State Chemist (Synthesis Agent) [NEW]: By analyzing refractory vs. volatile elements, the engine generates a step-by-step lab recipe.
🏆 Results: From Digital Memory to the Physical Lab
The attached screenshots show this end-to-end sovereignty in action. When we targeted a 2.16 eV band gap, the engine bypassed known databases and invented Cs3InS3 (Novel).
Not only did it explain why it chose this material using XAI (SHAP plots), and map its exact atomic coordinates in an interactive 3D rendering, but it also provided the precise solid-state synthesis protocol. Recognizing the volatility of Sulfur and Cesium, the engine proactively instructed the use of an Ar-filled glovebox, an evacuated quartz tube, and specific pre-calcination and sintering temperatures (400°C to 700°C).
🌐 True Tech Sovereignty
This is no longer just a predictive model; it is a Generative Physical Creation engine. Operating entirely offline (0 API calls) with terrifying computational efficiency on consumer GPUs, it protects intellectual property while delivering actionable science.
We are not generating theoretical research papers; we are designing a sovereign engine that invents materials your lab can actually synthesize, commercialize, and use to solve the world's problems... safely and efficiently.
#DeepTech #MaterialsScience #GenerativeAI #MachineLearning #Chemistry #TechSovereignty #Semiconductors #Innovation #Crystallography #ExplainableAI