Athenas-Reasoning-9B

Model Card: Athenas-Reasoning-9B

1. Model Summary

Athenas-Reasoning-9B is an advanced, instruction-tuned Large Language Model (LLM) specifically engineered for the Brazilian Portuguese language. Built upon the robust Qwen 3.5 (9B) architecture, this model has been fine-tuned to excel in complex logical reasoning, academic synthesis, and high-fidelity text generation.

The model was optimized using Low-Rank Adaptation (LoRA) on a heavily filtered, high-quality subset of conversational data, ensuring strict adherence to user instructions and professional output formatting.

Model Identifiers

• Developer: Kodjaoglanian
• Architecture: Qwen 3.5 (9 Billion Parameters)
• Training Stage: Supervised Fine-Tuning (SFT)
• Quantization: Available in standard Safetensors and GGUF (q4_k_m)
• Context Window: 1024 Tokens
• Language: Portuguese (PT-BR)

2. Prompting & Inference

To achieve optimal performance, inferences must strictly follow the ChatML formatting used during training. The model is highly responsive to its designated system prompt.

Recommended System Prompt

"Você é a Athenas, uma assistente virtual brasileira de código aberto, projetada para ser altamente inteligente, versátil e precisa nas suas respostas."

ChatML Template Structure

<|im_start|>system
Você é a Athenas, uma assistente virtual brasileira de código aberto, projetada para ser altamente inteligente, versátil e precisa nas suas respostas.<|im_end|>
<|im_start|>user
[Your instruction here]<|im_end|>
<|im_start|>assistant

3. Training Architecture & Methodology

The model underwent Supervised Fine-Tuning utilizing the Unsloth library to ensure memory-efficient gradient checkpointing and optimized CUDA kernel execution.

Data Curation

• Dataset: Polygl0t/gigaverbo-v2-sft (Native OpenAI messages format).
• Heuristic Filtering: The training manifold was strictly limited to interactions possessing an instruct_score > 0.8. This exclusion of sub-optimal data prevented model degradation and mode collapse, ensuring a dense, high-quality learning signal.

Hyperparameters & LoRA Configuration

• Optimizer: AdamW (8-bit)
• Learning Rate: 1e-4 (Cosine Scheduler)
• Warmup Steps: 100
• Effective Batch Size: 8
• Weight Decay: 0.01
• LoRA Rank (r): 32
• LoRA Alpha: 16
• Target Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• Rank-Stabilized LoRA (rsLoRA): Enabled
• Convergence: Training was halted at Step 900, achieving a highly stable evaluation loss of ~1.19, indicating optimal structural retention without overfitting.

4. Intended Use & Limitations

Out-of-Scope Applications

Athenas-Reasoning-9B is an autoregressive language model and operates probabilistically. It is strictly prohibited for use in:

• Automated critical decision-making systems (e.g., medical diagnostics, autonomous judicial sentencing, financial trading algorithms) without human-in-the-loop verification.
• Generation of deterministic factual datasets without external Retrieval-Augmented Generation (RAG) pipelines.

Known Limitations

• Contextual Truncation: Inputs exceeding the 1024-token context window will result in degraded attention mapping.
• Hallucination Variance: While highly aligned, the model may occasionally produce plausible but factually incorrect assertions (hallucinations), particularly in highly niche or mathematically deterministic domains.

5. Benchmarks & Evaluation

(Pending - Evaluation metrics against standard Brazilian datasets such as OAB Exams, ENEM Challenge, BLUEX, and ASSIN2 are currently being processed using the lm-evaluation-harness-pt framework and will be updated in future revisions).

6. Deployment Environment

For edge deployment and local execution, it is highly recommended to utilize the GGUF quantized variant (Athenas-Reasoning-9B-GGUF) using inference engines such as llama.cpp, Ollama, or LM Studio. The q4_k_m quantization provides near-fp16 fidelity while operating within the VRAM constraints of consumer-grade hardware (approx. 5.5 GB VRAM required).

7. Citation

@misc{kodjaoglanian2026athenas,
  author = {Kodjaoglanian},
  title = {Athenas-Reasoning-9B: Advanced Instruction-Tuned LLM for Brazilian Portuguese},
  year = {2026},
  publisher = {Hugging Face},
  url = {https://huggingface.co/Kodjaoglanian/Athenas-Reasoning-9B}
}