Pygmalion 2 13B

Pygmalion-2 13B represents a significant evolution in conversational AI models, built upon Meta AI's Llama-2 13B architecture. This instruction-tuned language model specializes in conversation, role-playing, and creative writing applications, marking a substantial advancement over its predecessors in the Pygmalion family.

Training and Architecture

The model's training methodology combines traditional instruction tuning with specialized data sources, including role-playing content, fictional narratives, and conversational exchanges. A key component of the training data is the PIPPA dataset, alongside carefully curated instruction datasets from role-playing forums. The training process, facilitated by the Axolotl framework, employed synthetic instruction generation to enhance the model's capabilities.

Features and Implementation

A distinctive feature of Pygmalion-2 13B is its sophisticated prompting format, which represents an improvement over the classic Pygmalion model. The system utilizes three specific tokens for different roles in interactions:

• <|system|>: Used for providing background information and context
• <|user|>: Designates user input and queries
• <|model|>: Marks the model's generated responses

This structured approach enables complex interactions and maintains coherent conversation history, making it particularly effective for creative and interactive applications.

Usage Considerations and Licensing

The model operates under the Llama-2 license, which permits both commercial and non-commercial applications. However, potential users should note several important considerations:

• The model has not undergone safety fine-tuning and may generate controversial or inappropriate content
• Its primary intended use is for fictional writing and entertainment purposes
• Users should exercise caution and implement appropriate content filtering when deploying the model

The development of Pygmalion-2 13B was supported by SpicyChat, demonstrating the growing interest in specialized conversational AI models for creative applications.

References

1. Pygmalion-2 13B Model Page - Official model repository and documentation
2. Llama-2 13B Base Model - Original architecture documentation
3. PIPPA Dataset - Training dataset information
4. Axolotl Framework - Training framework documentation
5. SpicyChat - Project sponsor information