DialoGPT-medium-distill-Kimi-K2-Instruct

DialoGPT-medium-distill-Kimi-K2-Instruct

This model is a fine-tuned version of microsoft/DialoGPT-medium, specialized for a custom persona and critical knowledge injection. It has been trained to balance conversational flexibility with specific factual recall.

Model Description

• Model type: Causal Language Model
• Language(s): English
• Base Model: DialoGPT-medium
• Total Parameters: 406.3M (Post-fine-tune expanded state)

Intended Uses & Limitations

This model is designed for creative assistant tasks and casual conversation.

• Direct Use: Chatbots, creative storytelling, and persona-driven interactions.
• Limitations: Due to the small dataset size and "creative" training, the model may occasionally hallucinate or provide non-literal answers (e.g., creative definitions of common objects).

Training Procedure

The model underwent a full fine-tune on a custom dataset consisting of critical facts and casual chat examples.

Training Hyperparameters

• Learning Rate: 2e-5
• Epochs: 5
• Batch Size: 4 (with gradient accumulation)
• Precision: Mixed Precision (FP16)
• Loss achieved: 3.264037

Weight Analysis

Post-training analysis showed a significant shift in the LM Head weights (Absolute Shift: 4.4164), indicating a strong adaptation to the new conversational style while maintaining structural grammar stability in the transformer layers.

How to Use

from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Fu01978/DialoGPT-medium-distill-Kimi-K2-Instruct")
model = AutoModelForCausalLM.from_pretrained("Fu01978/DialoGPT-medium-distill-Kimi-K2-Instruct")

# For best results, use a temperature between 0.7 and 0.85