The simplest way to self-host Mistral 7B OpenOrca. Launch a dedicated cloud GPU server running Lab Station OS to download and serve the model using any compatible app or framework.
Direct Download
Download model weights for local inference. Must be used with a compatible app, notebook, or codebase. May run slowly, or not work at all, depending on your system resources, particularly GPU(s) and available VRAM.
Open WebUI is an open-source, self-hosted web interface with a polished, ChatGPT-like user experience for interacting with LLMs. Integrates seamlessly with local Ollama installation.
The most full-featured web interface for experimenting with open source Large Language Models. Featuring a wide range of configurable settings, inference engines, and plugins.
Model Report
OpenOrca / Mistral 7B OpenOrca
Mistral-7B-OpenOrca is a fine-tuned version of Mistral-7B using the OpenOrca dataset. It shows notable improvements over the base model, with 129% better AGI Eval scores and 119% better BigBench-Hard performance. Uses ChatML format and is available in multiple quantized versions.
Explore the Future of AI
Your server, your data, under your control
Mistral-7B-OpenOrca represents a significant advancement in open-source language models, built upon the foundation of the Mistral 7B base model and fine-tuned using the OpenOrca dataset. This dataset was created to replicate the methodology described in Microsoft Research's Orca paper, which demonstrated the effectiveness of learning from complex explanation traces and step-by-step thought processes.
Architecture and Training
The model utilizes OpenChat packing and was trained using the Axolotl framework. The training process involved 8x A6000 GPUs over 62 hours, with an approximate commodity cost of $400. The model implements OpenAI's Chat Markup Language (ChatML) format, using <|im_start|> and <|im_end|> tokens, ensuring compatibility with various tools including oobabooga and Hugging Face's Transformers chat template.
Performance and Benchmarks
At the time of its release, Mistral-7B-OpenOrca achieved remarkable performance metrics, leading the Hugging Face Leaderboard among models smaller than 30B parameters. The model demonstrated significant improvements over its base version:
AGI Eval: 129% improvement over the base model, achieving an average score of 0.397
BigBench-Hard: 119% improvement, with an average score of 0.416
Hugging Face Leaderboard: Average score of 65.84, surpassing all other 7B and 13B models
GPT4ALL Leaderboard: Average score of 72.38
MT-Bench: Score of 6.86, comparable to Llama2-70b-chat
Model Variants and Accessibility
The model's efficiency allows it to run on consumer-grade GPUs, making it particularly accessible for researchers and developers. Several quantized versions are available through TheBloke on Hugging Face:
