Llama-3-8B-CESAR-PT-BR

Model Card for Llama-3-8B-CESAR-PT-BR

This is the first model fine-tuned within CESAR. It is specifically designed to be a Small Language Model (SLM) specialist in the Portuguese (Brasil) language.

Model Details

Model Description

This model is an instruction-tuned version of unsloth/llama-3-8b-Instruct-bnb-4bit. It has been fine-tuned using a handcrafted database of Brazilian Portuguese instructions to improve naturalness, cultural context, and grammar.

• Developed by: CESAR
• Model type: LoRA Adapter (PEFT)
• Language(s) (NLP): Portuguese (Brasil)
• License: Meta Llama 3 Community License
• Finetuned from model: unsloth/llama-3-8b-Instruct-bnb-4bit

Current Status: Work in Progress (WIP)

[!IMPORTANT] This model is currently a Work in Progress. The team is actively working to minimize hallucinations and improve factual accuracy in complex scenarios.

Usage

To use this model with 2x faster inference and lower memory usage, we recommend using the unsloth library.

1. Installation

pip install "unsloth[colab-new] @ git+[https://github.com/unslothai/unsloth.git](https://github.com/unslothai/unsloth.git)"
pip install --no-deps "xformers<0.0.27" "trl<0.9.0" peft accelerate bitsandbytes

Loading the Model

Unsloth's FastLanguageModel automatically handles loading the base model and attaching your LoRA adapter in one step.

from unsloth import FastLanguageModel
import torch

model_id = "cesarbr/Llama-3-8B-CESAR-PTBR"

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = model_id,
    max_seq_length = 2048,
    dtype = None,           # None for auto-detection
    load_in_4bit = True,    # Use 4-bit quantization
)

# Enable 2x faster inference
FastLanguageModel.for_inference(model) 

# Example Inference
inputs = tokenizer(
    ["Qual é o principal objetivo da Inovação?"],
    return_tensors = "pt"
).to("cuda")

outputs = model.generate(**inputs, max_new_tokens = 128)
print(tokenizer.batch_decode(outputs))

Training Details

Training Technique: SFT (Supervised Fine-Tuning)

Optimization: LoRA (via Unsloth)

Dataset: Handcrafted instruction dataset focused on Brazilian Portuguese.

Limitations and Biases

Hallucinations: As a WIP, factual consistency is still being optimized. Dialect: Optimized for PT-BR. Performance in PT-PT or other variants may vary.

Ethical Considerations

This model is currently under active development and evaluation.

Dataset and Question Analysis: The instruction dataset and evaluation questions used for fine-tuning are still being analyzed. As such, there may be undetected biases, incomplete coverage, or unintended patterns present in the data.

Bias and Fairness: While care was taken to craft culturally appropriate Brazilian Portuguese instructions, the model may still reflect linguistic, cultural, or societal biases present in the source data or inherited from the base model. Outputs should not be considered neutral or authoritative in sensitive contexts.

Hallucinations and Factual Accuracy: This model may produce hallucinations or inaccurate statements, especially when responding to complex, ambiguous, or fact-sensitive queries. Users are encouraged to verify critical information independently.

Intended Use: The model is intended for research, experimentation, and general-purpose text generation in PT-BR. It is not recommended for high-stakes applications, including but not limited to legal, medical, financial, or safety-critical decision-making.

Ongoing Mitigation Efforts: The CESAR team is actively working to evaluate the dataset, reduce hallucinations, and improve robustness and ethical alignment in future versions.

License Notice

This model is a LoRA adapter fine-tuned from Meta Llama 3. Use of this model is subject to the Meta Llama 3

This model is provided “as is”, without warranties of any kind, express or implied. The authors and contributors are not liable for any damages or misuse arising from the use of this model.