phi3-mini-ds-ai-finetuned

Model Card: phi3-mini-ds-ai-finetuned

Model Description

• Base model: microsoft/Phi-3-mini-4k-instruct (3.8B parameters)
• Fine-tuning method: QLoRA (4-bit quantization) + LoRA adapters via Unsloth
• Developed by: Vijendra Pokharkar (DS-AI-75D roadmap, Day 61)
• Model type: Causal Language Model (instruction-following)
• Language: English
• HF Hub: https://huggingface.co/VijendraHuggingface/phi3-mini-ds-ai-finetuned

Intended Use

• Primary use: Answering Data Science and AI concept questions
• Intended users: Students and practitioners learning ML/AI fundamentals
• Out-of-scope: Medical advice, legal decisions, financial recommendations, production customer-facing applications without further evaluation

Training Data

• 10 hand-crafted instruction-following examples covering: overfitting, precision/recall, transformers, RAG, gradient descent, dropout, vector databases, LoRA, attention mechanism, supervised vs unsupervised learning
• Format: Phi-3 instruction template (<|user|>/<|end|>/<|assistant|>)
• No personally identifiable information in training data

Training Details

• Platform: Google Colab (Tesla T4 GPU, 15GB VRAM)
• LoRA rank (r): 16 | lora_alpha: 16
• Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• Trainable parameters: 29,884,416 / 3,850,963,968 (0.78%)
• Epochs: 3 | Effective batch size: 8 | Learning rate: 2e-4
• Optimizer: AdamW 8-bit

Evaluation

• Manual testing on held-out DS/AI questions
• Model correctly explained RAG, transformer architecture after fine-tuning
• No formal benchmark evaluation conducted (dataset too small for reliable metrics)

Limitations

• Trained on only 10 examples — generalisation is very limited
• May hallucinate on topics not covered in training data
• Not evaluated on diverse demographic groups or languages
• Should not be used for high-stakes decisions without thorough evaluation

Biases and Risks

• Training data was hand-crafted by a single author — may reflect individual perspective biases in explanations
• Base model (Phi-3-mini) may carry biases from its pretraining data
• Small fine-tuning dataset limits ability to detect or correct base model biases

Responsible AI Considerations

• EU AI Act risk tier: Minimal Risk (educational/learning tool)
• No automated decision-making for consequential decisions
• Recommended: human review of any output used in educational materials

How to Use

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained("microsoft/Phi-3-mini-4k-instruct")
model = PeftModel.from_pretrained(base,
            "VijendraHuggingface/phi3-mini-ds-ai-finetuned")
tokenizer = AutoTokenizer.from_pretrained(
            "VijendraHuggingface/phi3-mini-ds-ai-finetuned")