Mistral Small (2409)

Mistral Small v24.09 represents a significant advancement in enterprise-grade language models, featuring 22 billion parameters and delivering improved performance across multiple domains. This instruction-tuned large language model (LLM) sits strategically between Mistral NeMo 12B and Mistral Large 2, offering an optimal balance of capability and efficiency.

Technical Specifications and Architecture

The model boasts a vocabulary size of 32,768 and can process sequences up to 32,000 tokens in length. It supports function calling and is available in the Safetensors format. The architecture requires substantial computational resources - running inference on a single GPU demands at least 44 GB of GPU RAM. For optimal performance, tensor parallelism is recommended to distribute the processing load across multiple devices.

The model demonstrates impressive multilingual capabilities, supporting ten languages: English, French, German, Spanish, Italian, Portuguese, Chinese, Japanese, Russian, and Korean. This makes it particularly valuable for international applications and cross-language tasks.

Performance and Capabilities

Mistral Small v24.09 shows marked improvements over its predecessor (v24.02) in several key areas, including human alignment, reasoning capabilities, and code handling. These enhancements are documented in the release announcement, which includes comprehensive benchmark comparisons.

The model excels in various tasks, including:

• Translation
• Summarization
• Sentiment analysis
• Code generation
• Function calling
• Reasoning tasks

Implementation and Licensing

For implementation, the recommended approach is using the vLLM library (version 0.6.1.post1 or later), which is designed for production-ready inference pipelines. The model can also be deployed using the mistral-inference library or the Hugging Face transformers library, offering flexibility in implementation approaches.

The model is distributed under the Mistral AI Research License (MRL), which restricts usage to non-commercial research purposes. Commercial applications require a separate license directly from Mistral AI. Developers and researchers can view the full license terms for detailed information about usage rights and restrictions.

References

1. Mistral Small v24.09 Model Card - Detailed technical specifications and implementation guidelines
2. September 2024 Release Announcement - Official release information and benchmark comparisons
3. vLLM Repository - Recommended inference library
4. Mistral-Inference Repository - Alternative implementation library
5. MRL License - Full license terms and conditions