r/ResearchML

[Research] BCMT: Blockwise Causal Memory Transformer
▲ 9 r/ResearchML+2 crossposts

[Research] BCMT: Blockwise Causal Memory Transformer

Hi everyone,

I'd like to share a research project I've been working on for the past few months.

BCMT (Blockwise Causal Memory Transformer) explores an alternative approach to long-context language modeling.

Instead of propagating long-range information through dense global self-attention, BCMT keeps dense causal self-attention within local blocks and propagates contextual information through a compact exponential causal memory built from adaptive block summaries.

The goal is to investigate whether long-range dependencies can be modeled efficiently while reducing the computational and memory costs associated with global attention.

The paper presents:

  • the complete BCMT architecture,
  • the mathematical formulation,
  • an open-source PyTorch implementation,
  • an initial experimental evaluation on WikiText language modeling.

The initial experiments show validation perplexities close to a dense Transformer baseline while achieving higher training throughput and lower GPU memory usage.

I'd greatly appreciate technical feedback on the architecture, the experimental methodology, or potential limitations. Any technical feedback, suggestions, or criticism would be greatly appreciated.

Code: https://github.com/rachidlabs/BCMT

Paper (DOI): https://doi.org/10.20944/preprints202607.0333.v1

u/Shakythebestlol — 18 hours ago
▲ 12 r/ResearchML+7 crossposts

SALT: Salience-aware lexical trie for long-context compression.

SALT shrinks a long document down to a fixed size before it is sent to a language model, keeping the sentences that carry the most information. It works with any model, produces a shorter plain-text prompt, and cuts the compute, memory, and wait time that long inputs cost. saltChat keeps the theme trie in DRAM across turns, so a document is indexed once and reused for the whole conversation instead of being re-read every message.

github.com
u/No_Sky9786 — 15 hours ago
▲ 11 r/ResearchML+11 crossposts

Cross-Vendor Semantic Void Matrix: Zero-Byte Outputs in GPT/Claude/Gemini/Kimi

A frozen cross-vendor study of 31,430 trials across 11 GPT, Claude, Gemini & Kimi Large Language Models found 11,658 successful executions with exactly zero visible UTF-8 output bytes.

Across 4,290 strict matched semantic pairs, null-condition arms produced 2,505 Voids; matched output-licensed controls produced 0.

These were not refusals, safety blocks, rate limits, or transport failures.

Raw records, event hashes, verification code, and full analysis are public.

doi.org
u/rayanpal_ — 1 day ago
▲ 2 r/ResearchML+1 crossposts

59 public runs on Terminal-Bench 3.0's task, zero passes. Then one passed, using the method from the preprint I posted here.

Ten days ago I posted a theory preprint here and got told, correctly, that it had no evidence behind it. So I built a method out of it and ran it on Terminal-Bench 3.0.

On a binary patching task where the public record shows 59 runs from 11 different model and agent setups and zero passes, one run using the method scored 19 of 19 on the official verifier, inside the original 90 minute limit.

Two ways to poke at this, and I'd genuinely like both.

The easy one: just run that task with whatever setup you already use. It's called ico-path-patch, it's public, 90 minute limit, 19 checks, all or nothing. 59 public runs from 11 different configurations, none passed. If your stack gets through it with none of my stuff involved, that's a much more interesting data point than anything I posted, and it kills my claim. Fine by me.

The harder one: take the method and go after the leaderboard with it. The idea is one line — before solving the task, have the agent build itself a small service for that task, then solve the task through the service. The method is the set of rules for what that service has to pin down. Everything else is your own agent, your own model, your own runs. If it works for you, the score is yours.

My runs took forty to ninety minutes each and cost a few dollars. Nothing in the setup is mine except the method text. Everything I ran is on the repo, including what failed and what I changed in between.

The task: https://hub.harborframework.com/tasks/terminal-bench/ico-path-patch/latest

The 60 trial rows behind that zero-pass baseline, with the query: https://github.com/amingclawdev/charting-loop/blob/main/public/results/ico-path-patch/job-009/PUBLIC-TRIALS.json

How to try the method:
https://github.com/amingclawdev/charting-loop/blob/main/docs/REPLICATION-INVITATION.md

The original preprint post : https://www.reddit.com/r/ResearchML/comments/1vjeznd/the_charting_loop_a_probabilistic_theory_of/

▲ 18 r/ResearchML+1 crossposts

Which CS research areas offer the best combination of career longevity, income, and impact?

I am planning to apply to CS PhD programs and am considering which research areas to pursue. I want to identify areas that:

  • lead to lucrative job opportunities in the short, medium, and long term;
  • offer substantial entrepreneurial opportunities, preferably without requiring large amounts of upfront capital;
  • are likely to remain active research areas for many years; and
  • provide opportunities to make a significant impact.

I am researching publication, hiring, funding, and investment trends, but individual researchers may have insights that are not apparent from publicly available data. Which areas currently offer the strongest combination of income potential, entrepreneurial opportunity, research longevity, and impact?

Conversely, which areas may appear hot but are already producing diminishing marginal returns or becoming crowded? (LLMs?)

I understand that money should not be the sole reason to pursue a PhD or choose a research area. However, financial outcomes are a legitimate consideration. There is no particular virtue in becoming a starving scholar when it may be possible to do meaningful research and also become financially successful.

I also recognize that research direction often develops during the PhD rather than being fixed before admission. Still, I would like to make an informed choice about which areas to explore from the outset.

I am grateful for your feedback.

reddit.com
u/Alone_Reality3726 — 1 day ago

Desk-rejected but received reviews after Reviewer+AC discussion end date

Has anyone recieved reviews on a desk-rejected paper from NeurIPS?

I received desk-reject decision and it was mentioned decision is final and the paper will not be reviewed. But now I have received reviews on the paper.

I am wondering should I reach out to AC members to check whether there is time to address the reviews.

reddit.com
u/jahangir670508 — 1 day ago
▲ 7 r/ResearchML+6 crossposts

Dyslexia/ADHD or just overwhelmed by dense text? We’d love your input

Hey everyone! 👋

We are building a free, AI-powered reading assistant designed to make dense text/complex articles much easier to read and less overwhelming. Whether you experience reading difficulties (like Dyslexia or ADHD) or simply get screen fatigue and overload from heavy reading, we are designing this tool to help you process information effortlessly.

Could you take 3 minutes to fill out our quick survey? Your input will directly shape our design, from fonts to AI features.

(Note: If your specific habits or favorite preferences aren't listed in a question, please use the "Other" option to tell us your unique ideas help us immensely in building this tool!)

🔗 Take the Survey Here: https://forms.gle/8oBydGvCQhnkL9LQA

Thank you so much for your time and support! 🚀

u/Wriosehlia — 1 day ago

Anyone else working on World Models/JEPA in isolation? Looking to connect with peers and chat about latent spaces.

Hi! This is my first post on Reddit and my first post about machine learning in general. I work at a small research institute, mostly staffed by physicists and GIS specialists; we don't have many machine learning engineers.

I recently became interested in world models and tried to understand the topic myself. I initiated a series of experiments: the result was Random-Abstractor Control - a simple and effective test that catches decorative abstractions.

The problem is that I'm completely alone here, and I don't have a large following on Linkedin, so I'd like to find people to discuss the results with.

reddit.com
u/rgnnm — 2 days ago

Need arXiv endorsement

I have a research work on usage of Hybrid -RAG in Data management but need endorsement from someone to upload it in arXiv.
If someone can help me please do let know.

reddit.com
u/HoggRider800 — 1 day ago
▲ 3 r/ResearchML+2 crossposts

Is publishing in AI conferences at 18 worth it, or am I wasting my time?

Hi everyone, I'm an 18-year-old researcher still in high school. I've already published two papers on AI and I'm currently looking into targeting some conferences.

Do you think I'm on the right track with this approach, or am I just wasting my time and energy at this stage? Would love to hear your thoughts!

reddit.com
u/Humble_Can_7351 — 1 day ago

Can I submit to ICLR 2027 and withdraw if my paper survives AAAI Phase 1?

I submitted to AAAI 2027, and Phase 1 rejection hits on Sep 24. ICLR full paper deadline is literally the next day, Sep 25.

Can I submit to ICLR first, and then just withdraw it if I somehow survive AAAI Phase 1?

reddit.com
u/RelationshipIcy7915 — 2 days ago
▲ 8 r/ResearchML+3 crossposts

I built UnFlow: a tool to help researchers with ML experimentation

I've been working on an open-source project called UnFlow:

https://github.com/UnFlow-Labs/mlunflow

The idea is pretty simple:

Most ML experiment tracking looks like a list of independent runs usually stored in a table:

run_001
run_002
run_003
run_004
...

But in practice, experiments are usually related.

You change the learning rate, then the number of epochs, then the model, then some preprocessing code. Eventually you have hundreds of runs, but it's surprisingly difficult to answer:

  • What actually changed between these two experiments?
  • Which experiments are essentially the same computation?
  • Have I already run this experiment before?
  • How did I get from experiment A to experiment B?
  • Can I navigate the history of my experiments rather than just search through runs?

Unflow simply detect code changes in a Python function (limitation that for it is just a single function) and arguments that are passed to this function to build a graph where nodes are "states" and edges are transformations "what has changed", a new state is not added to the graph or executed expect if it has a transformation.

The project is still early, so I'm much more interested in feedback than pretending this is a finished product.

I'm particularly curious about three things:

  1. Does the "experiments as a graph" abstraction make sense to you?
  2. Do you currently run into problems with duplicated/redundant experiments?
  3. If you could see the complete lineage of your ML experiments, what would you want to query or visualize?

Repo: https://github.com/UnFlow-Labs/mlunflow

I'd love to hear how other people currently manage experiment lineage and whether this solves a real problem for you.

u/ha2emnomer — 3 days ago
▲ 13 r/ResearchML+2 crossposts

[Collaboration] Recruiting 4-Person Team (1 CSE, 2 Neuro) for a Machine Learning Neuroscience Project Tracking Prefrontal Executive Shifts

Hey everyone,

I’m launching a collaborative computational neuroscience project and am looking to put together a dedicated 4-person team. The goal is to build and validate a novel machine learning pipeline that identifies exactly which neural oscillations (brain waves) dominate the shifting of executive cycles in the prefrontal cortex.

I currently have the theoretical framework and core pipeline mapped out, but I need to move on to the actual project development and real-world dataset validation and paper submission.

The Target Team:

1 (since I'm one already)Computer Science & Engineering Student: Focus on implementing a unified competitive stacking pipeline, handling robust multi-view feature scaling, and optimizing regularization boundaries without data leakage.

2 Neuroscience Students: Focus on sourcing high-quality public EEG/LFP datasets, validating the biological interpretability of feature selection masks, and aligning model readouts with prefrontal cognitive theories.

This is a serious collaborative project with the ultimate goal of submitting two papers to upcoming tech/neuro conferences. We will be coordinating tasks smoothly and pushing clean code to our repository.

If you are a CSE or Neuro grad looking to apply advanced ML to complex biological signals and want to get a published paper on your resume, drop a comment or send me a DM with your background!

reddit.com
u/Dreamingcleaning — 3 days ago
▲ 4 r/ResearchML+2 crossposts

EarlyStopping in CNN

I am working on a CNN model training, but now I am confused, should I use monitor = "val_loss" or "val_accuracy"?

reddit.com
u/MNT999 — 2 days ago

What are the ML/DL based undergraduate research ideas ?

I’m a SE undergraduate and need to conduct an individual research as a part of my degree. Time period is around 6 months. I’m looking for some research ideas. I would really appreciate your feedbacks.

Thank you

reddit.com
u/Huge_Sheepherder6730 — 3 days ago
▲ 3 r/ResearchML+1 crossposts

NeurIPS rebuttal question: Can I update my linked GitHub repo to address reviewer concerns?

I submitted a NeurIPS paper with an anonymous code repository linked as supplementary material. During the rebuttal period, a reviewer pointed out a discrepancy between the repository and my reported experiments.

I then updated the repository during rebuttal to address this issue. Now I'm worried: could this be considered an impermissible change to the supplementary material? Could a reviewer or the area chair use the fact that I modified the repository after the submission deadline against my paper?

I'm a bit unsure how linked GitHub repositories are treated in this context, since they're technically mutable even after submission. Does only the repository state at the submission deadline count? Or is it acceptable to make corrections that the reviewers themselves identified during the review process?

reddit.com
u/Helpful-Attitude-232 — 3 days ago
▲ 0 r/ResearchML+1 crossposts

Three Months, One Rejection, and a Bigger Question: Where Should Interdisciplinary AI Research Live?

https://preview.redd.it/n3rnhjursujh1.png?width=1536&format=png&auto=webp&s=44ca5a59c6577a89e80bdf8436b8ecf1bc19b4c2

I want to share a research experience—not as a complaint about arXiv, but because I think it raises a broader question about how we publish and discover interdisciplinary AI research.

I recently submitted a research paper on CODA (Constitutional Oversight via Democratic Alignment).

CODA explores a unified approach to AI alignment by connecting three ideas:

  • Democratic Constitution Induction (DCI)
  • Adversarial Debate Supervision (ADS)
  • Recursive Weak-to-Strong Oversight (RWSO)

The basic question behind the framework is:

>

The paper isn't claiming to have solved alignment. The empirical evaluation is deliberately small, and the theoretical components are presented as research hypotheses/conjectures rather than established results.

That's important because I don't want to present a small experiment as a breakthrough.

Then came the publishing experience.

I waited approximately three months for an arXiv decision.

The submission was rejected.

What bothered me wasn't the rejection itself.

Rejection is part of research.

A hypothesis can be wrong.
An experiment can be insufficient.
A methodology can have serious flaws.

Researchers should be challenged.

The difficult part was having limited information about what specifically needed to change in order to make the research more suitable for the platform.

That made me reconsider something.

Where does interdisciplinary AI research belong?

CODA doesn't fit perfectly into one category.

It touches:

AI Alignment → Constitutional AI → Scalable Oversight → AI Safety → Weak-to-Strong Generalization → AI Governance → Philosophy of AI

This creates an interesting problem.

Academic infrastructure is generally organized around disciplines.

But many emerging AI problems are inherently cross-disciplinary.

A question such as:

>

can simultaneously be a machine-learning question, an AI-safety question, a governance question, and a philosophical question.

So after the arXiv experience, I decided to explore another route and make the work more visible within PhilPapers, where its philosophical and foundational dimensions can reach a different research community.

I'm not claiming that PhilPapers is "better than arXiv."

That's not the point.

Different platforms serve different intellectual communities.

The more interesting question is:

>

One thing I learned

Before this experience, my question was:

"How do I get this paper accepted?"

Now it's:

"How do I get this idea in front of people who are capable of breaking it?"

That's a much better research objective.

If CODA is flawed, I want someone to find the flaw.

If the assumptions are wrong, I want them challenged.

If the experimental design is weak, I want someone to improve it.

If the entire architecture is misguided, I'd rather discover that now than after building a much larger system around it.

The paper explicitly acknowledges its limitations: the pilot evaluation is small, uses automated judging, and doesn't establish statistically conclusive superiority. The full DCI and training-time CODA components also remain areas for future work.

So I'm not looking for applause.

I'm looking for adversarial feedback.

The question I'd like to ask Reddit

For researchers working in AI safety, alignment, ML, philosophy of AI, or related fields:

Have you experienced a similar problem with interdisciplinary research?

Have you ever had a paper where the biggest challenge wasn't necessarily the research itself, but finding the right category, venue, or community for it?

And more importantly:

Do you think scientific publishing infrastructure should become more problem-centric rather than discipline-centric?

I'd genuinely appreciate criticism of both the argument and CODA itself.

The goal isn't to prove that one platform is wrong.

The goal is to figure out how potentially useful ideas can reach the people best positioned to test, criticize, reproduce, and improve them.

I'd especially value feedback from people who disagree with me.

That's usually where the interesting research begins.

#AI #AIAlignment #AISafety #ConstitutionalAI #MachineLearning #ScalableOversight #WeakToStrong #AIResearch #PhilosophyOfAI #OpenScience

reddit.com
u/Historical-File-1215 — 3 days ago

How to actually read a technical paper, and how to test what's in it?

I've been trying to become a more rigorous reader of technical/ML papers, and I keep hitting the same wall.

I've tried a few approaches: DFS (jumping into a reference the moment it comes up), BFS (finishing one paper fully before touching the next), and a hybrid. I eventually found a structure I was okay with, but the concepts don't stick unless I implement them, I'll read something 2-3 times, feel like I get it, and it's gone a week later because I never built anything with it.

Two things I'd like input on:

  1. What's your actual reading workflow for a dense paper, order of sections, note-taking, when you chase references vs skip them?
  2. How do you test whether you understood it? I've found implementing it is the only real check, but that's slow and not always feasible. Is there a lighter-weight way you validate your own understanding short of reimplementing everything?

Not looking for one "correct" method, more curious what actually works for people who read a lot of these.

reddit.com
u/Sudden-Theme7554 — 3 days ago