Eagle-1-Mini

Eagle-1-Mini (falcon-h1-reasoning v1.1.1)

Eagle is a family lightweight code reasoning and generation models built upon the base Falcon-H1-Tiny models. It offers somewhat accurate and quick code snippet generation in a most major programming languages. It's small size (90M parameters) allows it to run comfortably on most laptop/commercial grade GPUs. It also offers rudimentary subject matter expert capabilities on code related subjects.

The Eagle-1 is the pilot model for the Eagle-1 series which incorporates high-end reasoning capabilities into the standard Falcon-H1 architecture.

This 90M variant has been SFT trained on code reasoning traces found here with further RL training carried out via. a custom GRPO algorithm. This endows the model with enhanced reasoning capabilities which allows it to serve higher quality generations.

---

Estimated parameters: 90M

Architecture: Falcon-H1

Intended use: Code snippet generations from natural language, instruction following and advanced reasoning

---

Training data

Phase-1

• Source: deepseek-v4-reasoning-code-2500 dataset (https://huggingface.co/datasets/Banaxi-Tech/Deepseek-V4-Reasoning-Code-2500)
• Rows: ~2,555 rows templated with a custom .jinja chat format
• Training: trained for 2,500 steps on an RTX 3090 (24GB VRAM)

Phase-2

• Source: deepseek-v4-reasoning-code-2500 dataset (https://huggingface.co/datasets/Banaxi-Tech/Deepseek-V4-Reasoning-Code-2500)
• Rows: ~2,555 rows templated with a custom .jinja chat format
• Training: trained via. GRPO for 200 steps on an RTX 3090 (24GB VRAM)

Usage

Install requirements:

pip install -r requirements.txt
pip install transformers datasets accelerate safetensors

Usage (Hugging Face Hub)

You can load it directly from HuggingFace:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, TextStreamer


device = "cuda" if torch.cuda.is_available() else "cpu"
model_id = "DireDreadlord/Eagle-1-Mini"


tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)

model = AutoModelForCausalLM.from_pretrained(model_id, trust_remote_code=True).to(device).eval()

streamer = TextStreamer(tokenizer, skip_prompt=True, skip_special_tokens=False)


prompt = "Give me a python program to implement the fibonacci series."

messages = [
    {
        "role": "user",
        "content": prompt,
    },
]

inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt",
    return_dict=True,
)
inputs = {name: tensor.to(device) for name, tensor in inputs.items()}


with torch.inference_mode():
    output = model.generate(
        **inputs,
        max_new_tokens=1024,
        do_sample=False,
        repetition_penalty=1.1,
        pad_token_id=tokenizer.eos_token_id,
        eos_token_id=tokenizer.eos_token_id,
        use_cache=True,
        streamer=streamer
    )

For optimal long-form generation(with reasoning), set max_new_tokens=1024

Limitations

• Model for experimental use only; users should employ it as such under license.