72B-Qwen2.5-Kunou-v1-FP8-Dynamic

This quant was made for and by Infermatic.ai

Sao10K/72B-Qwen2.5-Kunou-v1

Copy of the original card

72B-Qwen2.5-Kunou-v1

I do not really have anything planned for this model other than it being a generalist, and Roleplay Model? It was just something made and planned in minutes. <br>Same with the 14 and 32B version. <br>Kunou's the name of an OC I worked on for a couple of years, for a... fanfic. mmm...

A kind-of successor to L3-70B-Euryale-v2.2 in all but name? I'm keeping Stheno/Euryale lineage to Llama series for now. <br>I had a version made on top of Nemotron, a supposed Euryale 2.4 but that flopped hard, it was not my cup of tea. <br>This version is basically a better, more cleaned up Dataset used on Euryale and Stheno.

Recommended Model Settings | Look, I just use these, they work fine enough. I don't even know how DRY or other meme samplers work. Your system prompt matters more anyway.

Prompt Format: ChatML
Temperature: 1.1
min_p: 0.1

Future-ish plans: <br>- Complete this model series. <br>- Further refine the Datasets used for quality, more secondary chats, more creative-related domains. (Inspired by Drummer) <br>- Work on my other incomplete projects. About half a dozen on the backburner for a while now.

Special thanks to my wallet for funding this, my juniors who share a single braincell between them, and my current national service. <br>Stay safe. There have been more emergency calls, more incidents this holiday season.

Also sorry for the inactivity. Life was in the way. It still is, just less so, for now. Burnout is a thing, huh?

https://sao10k.carrd.co/ for contact.

---

<img src="https://raw.githubusercontent.com/axolotl-ai-cloud/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>

<details><summary>See axolotl config</summary>

axolotl version: 0.5.2

base_model: Qwen/Qwen2.5-72B-Instruct
model_type: AutoModelForCausalLM
tokenizer_type: AutoTokenizer

load_in_8bit: false
load_in_4bit: false
strict: false
sequence_len: 16384
bf16: auto
fp16:
tf32: false
flash_attention: true

adapter: lora
lora_model_dir:
lora_r: 128
lora_alpha: 16
lora_dropout: 0.1
lora_target_linear: true
lora_fan_in_fan_out:
peft_use_rslora: true

# Data
dataset_prepared_path: last_run_prepared
datasets:
  - path: datasets/amoral-full-sys-prompt.json # Unalignment Data - Cleaned Up from Original, Split to its own file
    type: customchatml
  - path: datasets/mimi-superfix-RP-filtered-fixed.json # RP / Creative-Instruct Data
    type: customchatml
  - path: datasets/hespera-smartshuffle.json # Hesperus-v2-Instruct Data
    type: customchatml
warmup_steps: 15

plugins:
  - axolotl.integrations.liger.LigerPlugin
liger_rope: true
liger_rms_norm: true
liger_layer_norm: true
liger_glu_activation: true
liger_fused_linear_cross_entropy: true

# Iterations
num_epochs: 1

# Batching
gradient_accumulation_steps: 4
micro_batch_size: 1
gradient_checkpointing: "unsloth"

# Optimizer
optimizer: paged_ademamix_8bit
lr_scheduler: cosine
learning_rate: 0.000004
weight_decay: 0.1
max_grad_norm: 25.0

# Iterations
num_epochs: 1

# Misc
deepspeed: ./deepspeed_configs/zero3_bf16.json

</details><br>