Fanar-1-9B-Instruct-PalmX-Culture

Fanar-1-9B-Instruct LoRA Merged Model

This model is a merge of the base model QCRI/Fanar-1-9B-Instruct with a LoRA adapter trained on PalmX, Palm, and NativQA datasets.

Base Model

• Base Model: QCRI/Fanar-1-9B-Instruct
• LoRA Training: 3 epochs, 0.0002 learning rate, 0.1 dropout
• Training Data: PalmX (train+dev), Palm, NativQA

Training Details

• Epochs: 3
• Learning Rate: 0.0002
• Dropout: 0.1
• Datasets: PalmX (train_dev) + Palm + NativQA

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "hunzed/Fanar-1-9B-Instruct-PalmX-Culture",
    torch_dtype=torch.float16,
    device_map="auto",
    trust_remote_code=True
)

tokenizer = AutoTokenizer.from_pretrained(
    "hunzed/Fanar-1-9B-Instruct-PalmX-Culture",
    trust_remote_code=True
)

# Generate text
inputs = tokenizer("Your prompt here", return_tensors="pt")
outputs = model.generate(**inputs, max_length=100, do_sample=True, temperature=0.7)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Model Details

• Language: Arabic (primarily)
• Type: Causal Language Model
• Architecture: Based on Fanar-1-9B-Instruct
• Training: LoRA fine-tuning merged into base weights