EVA-D-Qwen2.5-1.5B-v0.0

EVA-D Qwen2.5-1.5B v0.0

<p> An experimental online logit distillation of EVA-Qwen2.5-14B-v0.1 into Qwen2.5-1.5B. Should work as a RP/storywriting specialist, but don't expect superb performance from it, due to it's small size. All in all, it was a fun experiment to do.<br> </p> <p>Note: using quantized KV cache with Qwen2.5 <b>is not recommended</b> and can lead to degraded output quality. On the other hand, Qwen's KV cache is already light enough, so using f16 for it shouldn't be problematic.</p> <p> <p>Prompt format is ChatML.</p><br> <h3>Recommended sampler values:</h3> <ul> <li>Temperature: 1</li> <li>Min-P: 0.02</li> </ul> <h3>Recommended SillyTavern presets (via CalamitousFelicitousness):</h3>

• Context
• Instruct and System Prompt

</p> <p> <br> <h3> Distillation data: </h3> <ul> <li>Arcee.AI's <a href=https://huggingface.co/datasets/arcee-ai/EvolKit-20k>EvolKit-20k</a> dataset, which is specifically made for knowledge distillation purposes.</li> </ul> <h3> Training time and hardware: </h3> <ul><li>1.8 hours on 8xA100 SXM, provided by Garg</li></ul><br> </p> <p>Model was trained by Kearm and Auri.</p> <h4>Special thanks:</h4><ul> <li><b>to Garg for generously providing 8xA100 SXM node for this experiment!</b></li> <li>to Arcee.AI for creating DistillKit and EvolKit-20k dataset, which were used to create this model.</li> <li>and to Allura-org for support and feedback on EVA models.</li></ul>