Mythalion 13B

Mythalion-13B represents a significant advancement in large language model development, combining the strengths of two powerful base models: Pygmalion-2 13B and MythoMax 13B. Built on the Llama-2 architecture, this merged model demonstrates enhanced capabilities in role-playing and conversational tasks, as detailed in the PygmalionAI blog post.

Technical Architecture and Implementation

The model utilizes the LlamaForCausalLM architecture, implemented through PyTorch and employing Safetensors for model weight storage. With 13 billion parameters, Mythalion-13B is distributed in FP16 format, striking a balance between model capability and computational requirements. The model's development leveraged the Axolotl framework, contributing to its robust implementation.

Training and Capabilities

Mythalion-13B's training incorporated diverse datasets, including:

• PygmalionAI/PIPPA
• Open-Orca/OpenOrca
• Norquinal/claude_multiround_chat_30k
• jondurbin/airoboros-gpt4-1.4.1
• databricks/databricks-dolly-15k

Early testing indicates that Mythalion-13B outperforms its MythoMax 13B predecessor in both role-playing and chat interactions. However, it's important to note that the model wasn't specifically fine-tuned for safety considerations, and users should be aware it may generate offensive or factually inaccurate content. The model's primary application focus is fictional writing.

Usage and Implementation

The model supports two distinct prompting formats:

1. Alpaca Format:

   • Uses "Instruction:" and "Response:" sections
   • Straightforward implementation for basic interactions

2. Pygmalion/Metharme Format:

   • Employs <|system|>, <|user|>, and <|model|> tokens
   • Enables structured conversation history
   • Allows for more complex interactive scenarios

For optimal performance, particularly when using SillyTavern, specific generation settings are recommended, which can be found in the PygmalionAI blog post.

Licensing and Accessibility

Mythalion-13B is released under the Llama-2 license, which permits both commercial and non-commercial use. This licensing structure makes it particularly accessible for a wide range of applications while maintaining appropriate usage guidelines.

Reference Links:

• PygmalionAI/mythalion-13b - Main model repository and documentation
• Pygmalion-2 13B - Base model documentation
• MythoMax L2 13B - Second base model documentation
• PygmalionAI Blog: Mythalion-13B Introduction - Detailed release information and settings
• Pygmalion Prompting Guide - Comprehensive prompting documentation
• Axolotl Framework - Development framework documentation
• Safetensors Documentation - Technical implementation details