Alpaca-Llama-3-8B-Arabic-v1

This model is a fine-tuned version of Llama-3-8B, specifically optimized to enhance its capabilities in understanding and generating Modern Standard Arabic (MSA), with a focus on high-quality instruction following.

1. Model DetailsDeveloped by: [shamil88]Base Model:
2. unsloth/llama-3-8b-bnb-4bit
3. Language: Arabic
4. (Primary)License: Apache-2.0F
5. inetuning Method: LoRA (Low-Rank Adaptation) via Unsloth library.
6. 1. Dataset InformationThe model was trained on a high-quality,
   2. curated dataset:
   3. Name: FreedomIntelligence/alpaca-gpt4-arabic
   4. Size: 49,969 samples
   5. (Conversation Pairs).Format: The data was converted from ShareGPT (conversations) format into a structured Alpaca Prompt format to ensure consistent instruction following.
   6. 1. Training Infrastructure & ResourcesThe training was conducted on a Google Compute Engine backend (GPU) within Google Colab.
      2. ResourceUtilizationTotal CapacityGPUNVIDIA Tesla T41 GPUGPU RAM7.
      3. 5 GB15.0 GBSystem RAM6.3 GB12.7 GBDisk Space52.9 GB112.6 GB4. Training HyperparametersTraining Duration: ~17 minutes (for 150 steps).
      4. Max Sequence Length: 2048 tokens.Batch Size per Device: 2Gradient Accumulation Steps: 4 (Effective Batch Size = 8)
      5. Learning Rate: 2e-4Optimizer:
      6. AdamW 8-bitPrecision:
      7. Mixed Precision (FP16)5.
      8. Training ProgressThe training showed steady convergence:
      9. Initial Loss (Step 1): 1.9637Final Loss (Step 150): 1.8752Lowest Observed Loss: ~1.16 during later steps.