DeepSeek-R1 represents a significant advancement in large language model technology, built upon the DeepSeek-V3 architecture. At its core, it's a 671B parameter model that uses a Mixture-of-Experts (MoE) architecture with 37B activated parameters, allowing for efficient processing while maintaining extensive knowledge capacity.
Architecture and Training
The model was developed through an innovative multi-stage training pipeline that combines reinforcement learning (RL) with supervised fine-tuning (SFT). Unlike its predecessor DeepSeek-R1-Zero, which relied solely on RL training, DeepSeek-R1 incorporates "cold-start" data - consisting of thousands of Chain-of-Thought (CoT) examples - before beginning the RL phase. This approach was implemented to address issues observed in DeepSeek-R1-Zero, such as repetitive outputs and poor readability.
The training process involves multiple stages:
- Initial cold-start data integration
- Two RL stages interspersed with SFT stages
- Focused training on reasoning and non-reasoning data
- Final RL stage optimized for helpfulness and safety
Performance and Capabilities
DeepSeek-R1 demonstrates exceptional performance across various benchmarks, achieving results comparable to OpenAI's o1-1217 model. Notable achievements include:
- AIME 2024: 79.8% Pass@1
- MATH-500: 97.3% Pass@1
- Codeforces: 96.3% percentile
The model particularly excels in reasoning tasks, outperforming its predecessor DeepSeek-V3 across numerous benchmarks including MMLU, MMLU-Pro, and GPQA Diamond. More details about the model's performance can be found in the DeepSeek R1 paper.
Model Family and Distillations
The DeepSeek-R1 family includes several variants:
- DeepSeek-R1-Zero: The original RL-only model
- DeepSeek-R1: The improved version with cold-start data and SFT
- Six distilled variants based on Llama and Qwen architectures:
- DeepSeek-R1-Distill-Qwen-1.5B
- DeepSeek-R1-Distill-Qwen-7B
- DeepSeek-R1-Distill-Llama-8B
- DeepSeek-R1-Distill-Qwen-14B
- DeepSeek-R1-Distill-Qwen-32B
- DeepSeek-R1-Distill-Llama-70B
The distilled models represent a significant achievement in knowledge transfer, with the 32B Qwen-based version notably outperforming OpenAI's o1-mini across various benchmarks. Research has shown that directly distilling from DeepSeek-R1 produces better results than applying RL directly to smaller models.
Technical Specifications and Usage
The model supports a maximum generation length of 32,768 tokens. For optimal performance, recommended parameters include:
- Temperature: 0.5-0.7
- Top-p: 0.95 (for benchmarking)
- Multiple runs should be averaged for accurate benchmarking
- Mathematical answers should include "\boxed" directives
- System prompts should be avoided
The model and its variants are released under the MIT License, permitting commercial use and modifications. However, users should note that the distilled models incorporate code originally licensed under Apache 2.0 (Qwen-2.5) and Llama 3.1/3.3 licenses.
References
