odia-mistral-7b-daft

Odia Mistral 7B DAFT Model

This model is a fine-tuned version of Mistral-7B for Odia (ଓଡ଼ିଆ) text generation using DAFT (Discriminative Alignment-Free Training).

Model Details

• Base Model: unsloth/mistral-7b-bnb-4bit
• Training Method: DAFT (Discriminative Alignment-Free Training)
• Language: Odia (ଓଡ଼ିଆ)
• Training Data: 25,000 Odia text examples
• Training Steps: 3,125
• Framework: Unsloth (2x faster training)

Training Details

• Method: LoRA fine-tuning with r=16
• Batch Size: 2 (effective: 8 with gradient accumulation)
• Learning Rate: 1e-4
• Optimizer: AdamW 8-bit
• Precision: FP16

Usage

from unsloth import FastLanguageModel

# Load model
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="abhilash88/odia-mistral-7b-daft",
    max_seq_length=400,
    dtype=torch.float16,
    load_in_4bit=True,
)

# Enable inference mode
FastLanguageModel.for_inference(model)

# Generate Odia text
prompt = "ଓଡ଼ିଆରେ ଲେଖ:"
formatted_prompt = f"<|user|>\n{prompt}\n<|assistant|>\n"

inputs = tokenizer(formatted_prompt, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=50, temperature=0.7)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)

print(response.split("<|assistant|>")[-1].strip())

Sample Outputs

Prompt: ଲେଖ: (Write) Output: [Your model's generated Odia text]

Performance

• Successfully trained on 25,000 Odia examples
• Optimized for Odia text generation
• Supports various prompting styles in Odia

Training Infrastructure

• GPU: Optimized for 16GB+ VRAM
• Framework: Unsloth + Transformers
• Time: ~2-4 hours training time

Limitations

• Trained on limited Odia dataset
• May require additional fine-tuning for specific tasks
• Best performance with Odia prompts

Citation

If you use this model, please cite:

@misc{odia-mistral-daft,
  title={Odia Mistral 7B DAFT Model},
  author={Your Name},
  year={2025},
  publisher={Hugging Face},
  url={https://huggingface.co/abhilash88/odia-mistral-7b-daft}
}

License

Apache 2.0 License - see LICENSE file for details.