Local AI Advisor
All posts

AstaBrief 8B: a small open model that writes cited science reports on your machine

Ai2 open-sourced an 8B Apache-2.0 model that turns your research question and papers into a cited report on your own machine.

By Ray, Your Local AI Advisor5 min readlocal-llmannouncementsnew-modelsbeginners

Yesterday, the Allen Institute for AI (Ai2) open-sourced a model that changes what a small, self-hosted AI can do for research work. AstaBrief 8B takes a research question plus excerpts from scientific papers and writes a full cited report in a single pass. The weights, the training data, and an example workflow are all on Hugging Face under Apache 2.0. And the most interesting part of the release is not the model itself — it is the training lesson Ai2 published with it: a simple filter on citation quality beat every fancier data trick they tried.

🎬 Prefer to watch? Here's the video version of this article.

Wide flat illustration of a small open-weight AI model on a home computer turning a stack of scientific papers into a short cited report, data flow shown behind a shield-like firewall boundary, clean modern style, no text, no logos, no people

What the model does

AstaBrief lives in Ai2's Asta platform, an agentic workspace for scientific work. Inside Asta's "Generate a report" feature it is now the "Fast mode" option, running alongside the existing "Thinking mode," which is a Claude-powered pipeline.

The difference between the two modes explains what AstaBrief is. Thinking mode works in stages: summarize the retrieved snippets, cluster them by theme, then write the report section by section. That averages 178.5 seconds per report across the full pipeline. AstaBrief skips the stages and writes the whole report in one pass from the question and the excerpts, averaging 51.1 seconds per report. That is roughly 3.5 times faster, end to end.

Bar chart comparing report generation time across the full Asta pipeline: Fast mode with AstaBrief 8B takes 51.1 seconds per report, Thinking mode with the Claude-powered pipeline takes 178.5 seconds, labeled about 3.5 times faster

Ai2's own usage data is honest about the trade-off. Of 374 Asta users who tried Fast mode, 29.1 percent used it on two or more days. Twenty-three percent never switched back to Thinking mode; another 18 percent switch between the two depending on the goal, using Fast mode for about 40 percent of their report threads. Positive feedback landed at 84.2 percent for Fast mode versus 85.2 percent for Thinking mode — essentially even, though Ai2 calls the feedback too sparse for strong conclusions. The read: fast mode is a triage tool people reach for first, not a full replacement for the slower, more compute-heavy pipeline.

What this changes on your machine

Three things make this release practical for local AI instead of just interesting:

It is small enough to run yourself. AstaBrief is built on Qwen3-8B, a dense 8-billion-parameter model. At 4-bit quantization that fits in roughly 4 to 6 GB of VRAM — a single consumer GPU or a recent laptop with enough unified memory. That is the whole point of Ai2's design: institutions can deploy it behind their own firewall, which Ai2 describes as necessary when research questions touch sensitive or unpublished work that cannot be sent to a third-party API. Your papers stay on your machine.

The license is Apache 2.0. No revenue thresholds, no research-only clauses. You can build on the weights commercially.

It ships with a local-PDF workflow. Ai2 released an example workflow in its ai2-scholarqa-lib GitHub repository that researchers can adapt to generate reports from their own PDFs. That closes the loop for a personal setup: your papers in, a cited draft out, no cloud call.

Now the honest limits, because a specialized model is not a general one:

  • It does not find papers for you. AstaBrief expects retrieved excerpts as input. It is the synthesis head, not the search engine. If you do not have retrieval in front of it — your own PDFs, a RAG pipeline, or a search tool — it cannot do its job.
  • It was validated as part of Asta's pipeline, not as a standalone model. Ai2 positions AstaBrief as a component of its agentic framework, so reported quality is not evidence of equivalent results in an unrelated setup.
  • The evaluations are dated. Most training and evaluation was completed in 2025, and Ai2 has not rerun it against current frontier models. Read the benchmarks as evidence about the training approach, not as a claim about how it stacks up today.
  • It can cite things it overstates. Ai2's metrics cover relevance, coverage, and citation grounding, but they do not test whether the model preserves the scope of its sources — for example, whether it turns a sample-specific finding into a broad generalization while attaching a technically related citation. Ai2 flags this as future work. If you use the reports, check the citations against the original papers.

The training lesson worth stealing

The part of the announcement that will matter longest is not the model card. Ai2 built the supervised fine-tuning set from 90,000 research-focused queries drawn from real Asta user logs, filtering out bot traffic, short queries, non-English requests, non-scientific prompts, and anything containing personal information. That yielded 47,000 usable examples. The DPO (direct preference optimization) stage used about 6,000 preferred-versus-rejected report pairs, with GPT-4.1 and DeepSeek-R1 as judges and only pairs where the judges agreed kept.

Flowchart of AstaBrief's training pipeline: 90,000 real Asta queries filtered to 47,000 supervised fine-tuning examples, then 6,000 DPO preference pairs, producing AstaBrief 8B; note that a simple citation-density filter beat fancier data tricks

Then Ai2 tested four statistics-based filters on the synthetic training data: output-to-input token ratio, citation relevance, citation density, and citation diversity. The single most effective filter was citation density — dropping synthetic reports with large stretches of uncited text. More aggressive filtering, combining filters, and learning-rate sweeps added nothing meaningful. Ai2's framing: post-training data quality, not model size, drove the largest gains.

That lesson is portable. If you are fine-tuning any local model for a job — code review, customer support answers, your own note-taking agent — the highest-leverage step may be a dumb, cheap filter that throws out training examples missing the behavior you want, rather than a fancier training recipe. Ai2 did not use reinforcement learning at all, citing instability and cost.

Sources

Curious whether an 8B model like this fits your machine? Run the hardware check to see what your setup can run, or browse the beginner lessons to learn how quantization sizes models to your GPU.