MythoMax-L2-13b represents an innovative advancement in language model development, built upon the foundation of the Llama architecture. This model emerged as an enhanced iteration of MythoMix, created through a sophisticated merger of two predecessor models: MythoLogic-L2-13b and Huginn-13b. What sets this model apart is its novel tensor type merge technique, which specifically focuses on intermingling tensors at the beginning and end of the models to enhance coherency.
The development process involved a meticulous approach to model merging, with unique ratios and gradients applied to each of its 363 tensors. This careful attention to tensor manipulation can be explored in detail through the BlockMerge_Gradient YAML files, which document the scripts and templates used in the model's creation.
Capabilities and Performance
MythoMax-L2-13b demonstrates particular excellence in two key areas: roleplaying and storywriting. The model leverages the Alpaca formatting structure, which has proven crucial for optimal performance. This formatting approach requires users to include specific components in their prompts: a system prompt or character card, clear instructions, and a designated section for the model's response.
The model is available in multiple quantized versions through TheBloke's Hugging Face repository, offering flexibility in deployment and usage. These versions include:
- GGUF format
- GPTQ format
- AWQ format
Each quantized version maintains the model's core capabilities while optimizing for different deployment scenarios and computational requirements.
Model Family and Ecosystem
MythoMax-L2-13b is part of the broader Mytho Series (Llama 2) Collection, representing an evolution in the series' capabilities. As an improvement over MythoMix, it builds upon the strengths of its predecessors while introducing new architectural innovations through its unique tensor merging approach.
The model operates under an "other" license designation, with the creator promising additional details about the model's architecture and training data in future releases. This transparency approach aligns with the collaborative nature of the AI research community while maintaining appropriate intellectual property protections.
References
- MythoMax-L2-13b Model Card - Primary model repository and documentation
- MythoLogic-L2-13b - One of the foundational models used in creation
- Huginn-13b-FP16 - Second foundational model used in creation
- BlockMerge_Gradient YAML - Technical implementation details and scripts
- Mytho Series Collection - Complete collection of related models
- GGUF Quantized Version - Optimized model variant
- GPTQ Quantized Version - Optimized model variant
- AWQ Quantized Version - Optimized model variant
