EVA-Qwen2.5-14B-v0.1-exl2

This repo contains the copy of the original quantized using EXL2. Original: EVA-UNIT-01/EVA-Qwen2.5-14B-v0.1

EVA Qwen2.5 14B 0.1

<p> A RP/storywriting specialist model, full-parameter finetune of Qwen2.5-7B on mixture of synthetic and natural data.<br> It uses Celeste 70B 0.1 data mixture, greatly expanding it to improve versatility, creativity and "flavor" of the resulting model.<br> </p> <p> <b>Version 0.1 notes:</b><br> Dataset was deduped and cleaned from version 0.0, sequence length was also increased. Resulting model seems to be stabler, and 0.0 problems with handling short inputs and min_p sampling seem to be gone.<br> This version seems to be more or less optimal for the current data. It (again) started crashing on each checkpoint after some point, but it was less of a problem this time, as eval/loss already flatlined by that time. This is epoch 2.7 checkpoint. </p> <p>Note: using quantized KV cache with Qwen2.5 <b>is not recommended</b> and can lead to degraded output quality. On the other hand, Qwen's KV cache is already light enough, so using f16 for it shouldn't be problematic.</p> <p> <p>Prompt format is ChatML.</p><br> <h3>Recommended sampler values:</h3> <ul> <li>Temperature: 0.87</li> <li>Top-P: 0.81</li> <li>Min-P: 0.0025</li> <li>Repetition Penalty: 1.03</li> </ul> <p>Temperature lower than 1 is still recommended, but this version seems to perform ok with higher temp and Min-P. Problems with Min-P sampling seem to be finally solved.<p> <h3>Recommended SillyTavern presets (via CalamitousFelicitousness):</h3>

• Context
• Instruct and System Prompt

</p> <p> <br> <h3> Training data: </h3> <ul> <li>Celeste 70B 0.1 data mixture minus Opus Instruct subset. See that model's <a href=https://huggingface.co/nothingiisreal/L3.1-70B-Celeste-V0.1-BF16>card</a> for details.</li> <li>Kalomaze's Opus_Instruct_25k dataset, filtered for refusals.</li> <li>A subset (1k rows) of ChatGPT-4o-WritingPrompts by Gryphe</li> <li>A subset (2k rows) of Sonnet3.5-Charcards-Roleplay by Gryphe</li> <li>A cleaned subset (~3k rows) of shortstories_synthlabels by Auri</li> <li>Synthstruct and SynthRP datasets by Epiculous</li> </ul> <h3> Training time and hardware: </h3> <ul><li>TBA</li></ul><br> </p> Model was trained by Kearm and Auri. <h4>Special thanks:</h4><ul> <li>to Gryphe, Lemmy, Kalomaze, Nopm and Epiculous for the data</li> <li>to Alpindale for helping with FFT config for Qwen2.5</li> <li>to InfermaticAI's community for their continued support for our endeavors</li> <li>and to Allura-org for support and feedback on EVA models.</li></ul>