r/allenai

We’re sharing the engineering work behind Ai2’s GPU scheduler—our new approach to allocating compute across research teams. 👇

On our largest H100 cluster, median queue wait fell from 5 minutes to 24 seconds. 

Under our old scheduler, every scheduled workload eventually used HIGH priority. Some researchers kept idle jobs running to reserve GPUs for experiments. The incentives rewarded holding on to capacity even when there were no active jobs.

Managers now assign GPU-time budgets to research programs & projects. Our fair-share scheduler compares recent use with those budgets, moving work from underused allocations ahead in-queue.

Jobs declare the minimum runtime needed to make progress. Budgeted work is protected until then. Afterward, it can keep running or be interrupted; resumable jobs return to the queue.

Over a 30-day test with our new scheduler, teams received 98% of the GPU hours they were owed, accounting for actual demand. Cluster occupancy stayed at 98%, with spare capacity available to interruptible work that didn’t draw down a team’s budget.

📝 Read more about the design decisions behind our scheduler, rollout lessons, & what we’re tackling next: https://allenai.org/blog/impactful-scheduling

We're co-hosting Open Together with Hugging Face!

Join the open AI community at The Midway in San Francisco on Oct 16 to kick off Open Source AI Week. Help us build a future where AI is open, secure, and fully transparent.

The event is free. RSVP: https://luma.com/OpenTogether

Introducing AstaBrief 8B, an open model that turns complex research questions and literature excerpts into cited reports.

Run it locally on your own hardware, with open weights and training data you can inspect & build on. 👇

AstaBrief powers Fast mode in Asta’s Generate a report feature. It writes reports in one pass, skipping intermediate steps that summarize & group the retrieved excerpts so researchers can get a preliminary report sooner.

We built AstaBrief on Qwen3-8B using queries from ScholarQA & our earlier literature-synthesis systems. We removed bot/test traffic & very short prompts and used an LLM to filter non-English or non-scientific requests & prompts with personal info, leaving 90K queries.

From selected queries in the 90K pool, we generated cited reports. Quality filtering left 39.5K examples for supervised fine-tuning (SFT). We used separate queries to build ~6K report pairs where two judge models agreed, then trained with direct preference optimization (DPO).

AstaBrief is built for speed & quality. Across Asta’s full pipeline, Fast mode averages 51.1s per report vs. Claude-powered Thinking mode’s 178.5s—about 3.5× faster. And in our evaluations, AstaBrief is competitive with Thinking mode & DR Tulu on answer + citation measures.

Our example repo provides a starting point for generating reports from your own PDFs locally. For more info, read our blog.

🤗 Download AstaBrief 8B: https://huggingface.co/allenai/AstaBrief_8B

🌐 Try Fast mode: https://asta.allen.ai/

💻 Example repo: https://github.com/allenai/ai2-scholarqa-lib/tree/main/api/scholarqa/lite

📊 Training data: https://huggingface.co/collections/allenai/astabrief
📝 Blog: https://allenai.org/blog/astabrief

We’re releasing Olmo-core 3—open training infrastructure for large mixture-of-experts (MoE) models. 👇

It’s a core system behind the next generation of Olmo, designed to scale into the trillion-parameter range.

MoEs use only some of their specialized components – or experts – for each token. That saves computation, but storing the full model & routing data across GPUs can erode those savings as models grow.

Olmo-core 2 repeatedly gathered model weights for each small batch of training data. Olmo-core 3 keeps experts resident on GPUs & sends data to them, avoiding that repeated weight gathering.

Olmo-core 3 maintains nearly the same training speed across MoEs of increasing total size, w/ roughly the same number of parameters active per token. It delivers about 2.7× the throughput of Olmo-core 2 for the same model on the same number of GPUs.

Explore the concepts behind Olmo-core 3: https://narrative.allen.ai/scaling-up-training

The tech report covers experiments, findings, & failures, including token gerrymandering, where a load-balancing score could improve as workloads became less balanced: https://allenai.org/papers/olmocore3

Train your own MoEs with Olmo-core 3—adapt the framework to your hardware & experiment with distributed training.

💻 GitHub repo: https://github.com/allenai/olmo-core

📝 Learn more in our blog: https://allenai.org/blog/olmocore3