s1: How 1,000 Training Examples Beat OpenAI's o1-preview by 27%

A team of researchers from Stanford, University of Washington, and other institutions asked a simple question: what constitutes the minimum viable approach to test-time scaling?¹ Their answer upends assumptions about the computational requirements for building reasoning models. The s1 model, fine-tuned on just 1,000 carefully selected examples, exceeds OpenAI's o1-preview by up to 27% on competition mathematics benchmarks.²

TL;DR

The s1 paper introduces "budget forcing," a technique that controls how long a model thinks by either terminating reasoning early or appending "Wait" tokens to extend deliberation.³ Researchers curated s1K, a dataset of 1,000 questions selected for difficulty, diversity, and quality from 59,000 candidates.⁴ Fine-tuning Qwen2.5-32B-Instruct on s1K produced a model that scales predictably with inference compute.⁵ On AIME 2024, s1-32B achieves 57% accuracy with extended thinking versus o1-preview's approximate 44%.⁶ The entire approach requires no reinforcement learning, no process reward models, and no specialized infrastructure beyond standard fine-tuning.⁷

The Test-Time Scaling Paradigm

Traditional AI scaling invested compute during training: more parameters, more data, more GPU hours. Test-time scaling inverts the equation by investing compute during inference.⁸ Rather than building larger models, researchers enable smaller models to "think longer" on difficult problems.

OpenAI's o1 family demonstrated this paradigm at scale, generating extended chains of reasoning before producing final answers.⁹ The models achieve state-of-the-art results on mathematical reasoning and coding tasks by spending orders of magnitude more inference compute than standard chat models.¹⁰

The approach raises an obvious question: how much complexity does test-time scaling actually require?

The s1 Approach: Radical Simplicity

The s1 team pursued the simplest possible implementation that still achieves competitive performance.¹¹ Their method involves three components:

1. Dataset Curation (s1K)

Starting from approximately 59,000 questions spanning mathematics, science, and puzzle domains, researchers applied three filtering criteria:¹²

The resulting s1K dataset contains just 1,000 question-answer pairs with detailed reasoning traces.¹⁶ For context, typical instruction-tuning datasets contain tens of thousands to millions of examples.

2. Standard Fine-Tuning

The team fine-tuned Qwen2.5-32B-Instruct using standard supervised learning on s1K.¹⁷ No reinforcement learning from human feedback. No process reward models to score intermediate reasoning steps. No specialized training infrastructure.¹⁸

Training completed in under 26 minutes on 16 H100 GPUs.¹⁹

3. Budget Forcing at Inference

Budget forcing constitutes the paper's core technical contribution. The technique controls inference-time computation through two mechanisms:²⁰

Forced Termination: When the model generates an end-of-thinking token before reaching a target reasoning length, the system removes that token and appends a special "Wait" token instead.²¹ This forces the model to continue deliberating.

Forced Continuation: By repeatedly inserting "Wait" tokens, researchers extend reasoning chains arbitrarily.²² The model interprets these tokens as signals to reconsider its approach, often catching and correcting errors in previous reasoning steps.²³

The name "budget forcing" reflects controlling the compute "budget" spent on each query.

Benchmark Results

s1-32B demonstrates consistent improvements over o1-preview across mathematical reasoning benchmarks:²⁴

The AIME (American Invitational Mathematics Examination) represents a particularly challenging benchmark, featuring competition-level problems that test deep mathematical reasoning.²⁸

Scaling Behavior

Performance improves predictably with inference compute. On AIME 2024, accuracy increased from 50% to 57% when budget forcing extended reasoning chains.²⁹ The model exhibits "sample efficiency" at test time, meaning additional compute produces meaningful gains rather than diminishing returns.³⁰

This scaling property suggests s1's approach captures something fundamental about how extended reasoning improves performance, not merely an artifact of the training data.

Why Budget Forcing Works

The researchers hypothesize that budget forcing succeeds because it triggers specific beneficial behaviors:³¹

Self-Verification: Extended reasoning allows the model to check its own work, identifying logical errors or calculation mistakes before committing to an answer.³²

Alternative Exploration: When forced to continue thinking, models often explore alternative solution paths they would otherwise skip.³³

Error Correction: The "Wait" token appears to function as a metacognitive prompt, signaling the model to pause and reconsider rather than rushing to conclusion.³⁴

Qualitative analysis of reasoning traces shows models frequently change their answers after budget forcing, with corrections more often moving from wrong to right than vice versa.³⁵

Implications for the Field

Democratizing Reasoning Models

The s1 approach suggests reasoning capabilities may prove more accessible than previously assumed. The key requirements:³⁶

• Data: 1,000 high-quality examples (curated from public sources)
• Compute: 26 minutes on 16 H100s for training
• Infrastructure: Standard fine-tuning pipeline
• Expertise: Dataset curation judgment

Organizations without OpenAI's resources can potentially build competitive reasoning models using s1's methodology.

Open Questions

The paper acknowledges limitations and open problems:³⁷

Generalization: Does the approach transfer to non-mathematical domains? Early results on coding and general reasoning appear promising but less dramatic.³⁸

Ceiling Effects: How far can budget forcing scale? Extremely long reasoning chains may eventually produce diminishing or negative returns.³⁹

Training Integration: Would reinforcement learning on top of s1's approach yield further gains? The simplicity advantage might disappear if RL proves necessary for additional capability.⁴⁰

The Research Team

The paper represents a collaboration across multiple institutions:⁴¹

• Stanford University: Niklas Muennighoff, Fei-Fei Li, Percy Liang, Emmanuel Candès, Tatsunori Hashimoto
• University of Washington: Zitong Yang, Weijia Shi, Hannaneh Hajishirzi, Luke Zettlemoyer
• Other Contributors: Xiang Lisa Li

The combination of statistical learning expertise (Candès), NLP research (Hajishirzi, Zettlemoyer), and AI safety/alignment focus (Hashimoto) shaped the paper's emphasis on simple, interpretable methods.

Open Source Release

The researchers released all components publicly:⁴²

• Model Weights: s1-32B available on Hugging Face
• Dataset: s1K published with full reasoning traces
• Code: Training and inference pipelines on GitHub

This openness enables direct replication and extension by the research community.

Key Takeaways

The s1 paper challenges prevailing assumptions about test-time scaling complexity:

1. Quantity vs. Quality: 1,000 excellent examples outperform millions of mediocre ones for reasoning fine-tuning
2. Simplicity Wins: Budget forcing achieves competitive results without RL, reward models, or exotic architectures
3. Scaling Predictability: Performance improves reliably with inference compute, enabling cost-performance tradeoffs
4. Accessibility: The approach requires modest resources compared to training frontier reasoning models from scratch

For organizations evaluating reasoning model strategies, s1 demonstrates that sophisticated capabilities may emerge from surprisingly simple interventions on strong base models.

References