Downloads · 30 days
17
9% of all-time downloads
AlekseyCalvin/Lyrical_ru2en_Gemma4_SFT_2658_Opusish
Lyrical_ru2en_Gemma4_SFT_2658_Opusish is a text generation model from AlekseyCalvin. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as gemma.
A reasoning-enhanced fine-tune of google/gemma-4-E4B-it, distilled from Claude Opus 4.6 reasoning traces with supplementary math Chain-of-Thought data.
Downloads · 30 days
17
9% of all-time downloads
All-time downloads
182
Public
Parameters
8B
16 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors16 GB · 100%
From the Hugging Face model README
A reasoning-enhanced fine-tune of google/gemma-4-E4B-it, distilled from Claude Opus 4.6 reasoning traces with supplementary math Chain-of-Thought data.
| Base Model | google/gemma-4-E4B-it (4.5B effective params, 8B with embeddings) |
| Architecture | Dense transformer with Per-Layer Embeddings (PLE), 128K context |
| Fine-tuning Method | LoRA via Unsloth |
| Precision | Merged float16 |
| Training Hardware | NVIDIA A100 80GB (RunPod) |
| Training Framework | Unsloth + HuggingFace TRL (SFTTrainer) |
| Parameter | Value |
|---|---|
| Rank (r) | 16 |
| Alpha | 32 |
| Dropout | 0 |
| Bias | None |
| Target Modules | Attention + MLP (language layers only) |
| Parameter | Value |
|---|---|
| Epochs | 2 |
| Learning Rate | 1e-4 (cosine schedule) |
| Batch Size | 8 (2 per device × 4 gradient accumulation) |
| Warmup Steps | 100 |
| Optimizer | AdamW 8-bit |
| Weight Decay | 0.01 |
| Max Sequence Length | 4096 |
| Response-only Training | Yes (user turns masked) |
| Final Training Loss | ~0.54 |
Around 20,000 samples combining reasoning distillation and math Chain-of-Thought data (~40% math content):
| Dataset | Samples | Purpose |
|---|---|---|
| nohurry/Opus-4.6-Reasoning-3000x-filtered | 2,326 | Claude Opus 4.6 reasoning traces |
| Roman1111111/claude-opus-4.6-10000x | 9,633 | Claude Opus 4.6 extended reasoning |
| AI-MO/NuminaMath-CoT | 4,000 (sampled) | Math Chain-of-Thought solutions |
| TIGER-Lab/MathInstruct | 4,000 (sampled) | Math CoT + Program-of-Thought |
All assistant responses were formatted with <think>...</think> blocks to teach the model structured reasoning before answering.
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained(
"naazimsnh02/gemma-4-e4b-opus-reasoning-v2",
torch_dtype="auto",
device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained("naazimsnh02/gemma-4-e4b-opus-reasoning-v2")
messages = [{"role": "user", "content": "A train travels 60 km/h. How long does it take to cover 255 km?"}]
inputs = tokenizer.apply_chat_template(
messages,
add_generation_prompt=True,
return_tensors="pt",
tokenize=True,
return_dict=True,
).to(model.device)
output = model.generate(**inputs, max_new_tokens=1024, temperature=1.0, top_p=0.95, top_k=64)
print(tokenizer.decode(output[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
<think> blocks. It does not guarantee improved accuracy over the base model on any given task.gemma-4-E4B-it model. Users should apply their own safety measures for production use.This model inherits the Gemma license from the base model. Please review and comply with Google's Gemma Terms of Use.