tinyllama-ai-ml-tutor-qlora

Model Card for tinyllama-ai-ml-tutor-qlora

A QLoRA fine-tuned adapter for TinyLlama-1.1B-Chat-v1.0 that specializes it into an AI/ML tutor — it answers conceptual, "why", "how", comparison, code, debugging, math, interview, and real-world questions about machine learning, PyTorch, and Transformers, in a consistent, structured teaching format (Definition → Why It Matters → Intuition → Example → Key Points → Summary).

Model Details

Model Description

This is a LoRA adapter (not a full model) trained on top of TinyLlama/TinyLlama-1.1B-Chat-v1.0 using QLoRA (4-bit NF4 quantized base + LoRA adapters) and Hugging Face's SFTTrainer (TRL). It was trained on 2,000 synthetically generated AI/ML tutor Q&A examples covering topics like Self-Attention, Gradient Descent, PyTorch Autograd, LoRA, and QLoRA itself. The goal was not to expand the model's general knowledge, but to make its explanations of AI/ML topics consistently structured and teaching-oriented, compared to the base model's more generic, sometimes factually inconsistent answers.

• Developed by: Ashish Kumar Jha (@Ash8389 / @ash270)
• Funded by: Self-funded / personal project
• Shared by: Ashish Kumar Jha
• Model type: Causal decoder-only language model (LoRA adapter for a 1.1B-parameter Llama-architecture chat model)
• Language(s) (NLP): English
• License: Apache 2.0 (inherited from the base TinyLlama model; confirm this matches your intended license for the adapter itself)
• Finetuned from model: TinyLlama/TinyLlama-1.1B-Chat-v1.0

Model Sources

• Repository: Ash8389/finetune_tinyllama_ai_ml_tutor
• Paper: N/A — uses the QLoRA method from Dettmers et al., 2023
• Demo: Not currently hosted publicly; the repo includes a local FastAPI comparison service (src/api/) that serves base vs. fine-tuned answers side by side (uvicorn src.api.main:app --reload, then GET /chat?q=...)

Uses

Direct Use

Ask the model AI/ML conceptual questions (e.g. "What is self-attention?", "Why does QLoRA use 4-bit quantization?", "Debug this PyTorch autograd error") and receive a structured, tutor-style explanation. Well suited for learners who want consistent, scaffolded explanations rather than open-ended chat.

Downstream Use

Could be embedded as the explanation engine behind an educational chatbot, a study-assistant app, or a lightweight local tutoring tool where compute is constrained (the 1.1B base + 4-bit quantization make it runnable on modest/free-tier GPUs).

Out-of-Scope Use

• Not intended as a general-purpose chatbot or assistant — it is specialized for a narrow band of AI/ML topics (see Training Data).
• Not intended for high-stakes technical decisions (production ML system design, security-critical code, etc.) without independent verification.
• Not evaluated for languages other than English, safety-critical deployment, or any domain outside AI/ML education.

Bias, Risks, and Limitations

• Narrow topic coverage: trained on ~2,000 examples covering a limited set of topics (Self-Attention, Multi-Head Attention, Gradient Descent, PyTorch Autograd, LoRA, QLoRA). Performance outside these topics is not guaranteed to have improved over the base model.
• Synthetic training data: the training set was itself LLM-generated (via Groq's llama-3.1-8b-instant), so the model can inherit that generator's inaccuracies, oversimplifications, or repetitive phrasing.
• Small base model: as a 1.1B-parameter model, it is more prone to factual errors than larger models — e.g. in held-out evaluation, the base model confidently mis-expanded "QLoRA" as "Quantized Logistic Regression" (incorrect; it stands for Quantized Low-Rank Adaptation). While fine-tuning measurably improved structure and topical grounding, independent verification of technical claims is still recommended.
• No formal safety/bias evaluation has been performed on this adapter.

Recommendations

Users (both direct and downstream) should treat this as an educational aid for a narrow set of AI/ML topics, not an authoritative source — verify important technical claims against primary references (papers, official docs). Be aware the model may confidently state incorrect information, especially outside its trained topic set.

How to Get Started with the Model

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel
import torch

base_model_id = "TinyLlama/TinyLlama-1.1B-Chat-v1.0"
adapter_id = "ash270/tinyllama-ai-ml-tutor-qlora"

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.float16,
    bnb_4bit_use_double_quant=True,
)

tokenizer = AutoTokenizer.from_pretrained(adapter_id)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id, quantization_config=bnb_config, device_map="auto"
)
model = PeftModel.from_pretrained(base_model, adapter_id)
model.eval()

messages = [{"role": "user", "content": "Explain the bias-variance tradeoff simply."}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)

with torch.no_grad():
    output = model.generate(**inputs, max_new_tokens=300, temperature=0.7, do_sample=True)

input_len = inputs["input_ids"].shape[1]
print(tokenizer.decode(output[0][input_len:], skip_special_tokens=True))

Training Details

Training Data

2,000 synthetically generated AI/ML tutor Q&A examples, created by prompting Groq's llama-3.1-8b-instant (OpenAI-compatible API) to produce one example at a time, cycling through:

• Topics: Self Attention, Multi-Head Attention, Gradient Descent, PyTorch Autograd, LoRA, QLoRA
• Question types (9): concept, why, how, comparison, code, debugging, mathematics, interview, real_world

Each generated example was required to follow the target teaching structure (Definition → Why It Matters → Intuition → Example → Key Points → Summary) and was validated (single user/assistant turn, ≥100-character answer) before being kept. Data was split 80/10/10 (seed=40): 1,600 train / 200 validation / 200 test.

Training Procedure

Preprocessing

Metadata columns (id, topic, domain, category, difficulty) were dropped, keeping only the messages field, which was formatted using the tokenizer's chat template (apply_chat_template).

Training Hyperparameters

• Training regime: fp16 mixed precision, with the base model loaded in 4-bit NF4 (QLoRA)
• Quantization: load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.float16, bnb_4bit_use_double_quant=True
• LoRA config: r=8, lora_alpha=16, target_modules=["q_proj","k_proj","v_proj","o_proj"], lora_dropout=0.05, bias="none", task_type="CAUSAL_LM"
• Epochs: 3
• Per-device train batch size: 1
• Gradient accumulation steps: 4 (effective batch size 4)
• Learning rate: 2e-4
• LR scheduler: cosine
• Warmup ratio: 0.03
• Optimizer: paged_adamw_8bit
• Seed: 42
• Framework: trl.SFTTrainer, evaluated/saved every epoch with load_best_model_at_end=True (metric: eval_loss)

Speeds, Sizes, Times

Training ran for the full 3 epochs (1,365 steps total); the best checkpoint by eval_loss was the final one (checkpoint-1365). Exact wall-clock time and hardware were not logged in the repository — recommend adding these if you have them (e.g. from your Colab session).

Evaluation

Testing Data, Factors & Metrics

Testing Data

200 held-out test examples (data/raw/test.jsonl), drawn from the same synthetic generation pipeline and topic/question-type distribution as the training data, not used during training.

Factors

Evaluation was disaggregated informally by topic (Self-Attention, Gradient Descent, Autograd, LoRA, QLoRA) and question type (concept, why, how, comparison, code, debugging, mathematics, interview, real_world) via qualitative inspection of evaluation/result.jsonl, rather than a formal per-factor breakdown.

Metrics

• Quantitative: evaluation loss and mean token accuracy, computed automatically during training via SFTTrainer.
• Qualitative: side-by-side comparison of base vs. fine-tuned model responses on all 200 test questions (evaluation/result.jsonl), generated with max_new_tokens=300, temperature=0.7, sampling enabled — inspected for response structure, topical grounding, and adherence to the target teaching format.

Results

Qualitatively, the fine-tuned model reliably follows the Definition → Why It Matters → Intuition → Example → Key Points → Summary structure on held-out questions, while the base model produces more generic, less consistently structured, and occasionally factually incorrect answers (e.g. the base model, unprompted, misdefined "QLoRA" as "Quantized Logistic Regression").

Summary

Fine-tuning measurably improved token-level prediction accuracy on held-out AI/ML tutoring data (eval loss 0.625, eval token accuracy ~82%) and, qualitatively, made responses far more structured and topically grounded than the base model. No formal automatic quality benchmark (e.g. ROUGE, human preference scoring) has been run yet — this is planned future work.

Model Examination

Not performed. No interpretability analysis (attention visualization, probing, etc.) has been done on this adapter.

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

• Hardware Type: Not logged (trained via a Colab notebook — likely a single T4/A100-class GPU; confirm and fill in)
• Hours used: Not logged
• Cloud Provider: Google Colab
• Compute Region: Not logged
• Carbon Emitted: Not estimated — given the small scale (1.1B model, 4-bit quantized, 1,365 training steps on a single GPU), emissions are expected to be modest, but no formal estimate has been calculated.

Technical Specifications

Model Architecture and Objective

LoRA adapter (rank 8, alpha 16, targeting q_proj/k_proj/v_proj/o_proj) applied to TinyLlama/TinyLlama-1.1B-Chat-v1.0, a Llama-architecture decoder-only transformer, trained with a standard causal language modeling (next-token prediction) objective via supervised fine-tuning (SFT) on chat-formatted instruction/response pairs.

Compute Infrastructure

Hardware

Not precisely logged; training was run via a Colab notebook (notebooks/run_pipeline.ipynb). Fill in the specific GPU (e.g. T4, A100) if known.

Software

• Python
• transformers==4.51
• peft==0.16.0
• trl==0.19.1
• accelerate==1.8.1
• bitsandbytes==0.46.1
• datasets==3.6.0
• sentencepiece
• openai (used for the Groq-based synthetic data generation step)

Citation

BibTeX:

@misc{jha2025tinyllamaaimltutor,
  author       = {Ashish Kumar Jha},
  title        = {TinyLlama AI/ML Tutor: A QLoRA Fine-Tuned Adapter for AI/ML Education},
  year         = {2025},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/ash270/tinyllama-ai-ml-tutor-qlora}}
}

APA:

Jha, A. K. (2025). TinyLlama AI/ML Tutor: A QLoRA fine-tuned adapter for AI/ML education [LoRA adapter]. Hugging Face. https://huggingface.co/ash270/tinyllama-ai-ml-tutor-qlora

Glossary

• QLoRA: Quantized Low-Rank Adaptation — a fine-tuning method that quantizes the base model's weights (here, to 4-bit NF4) and trains small low-rank adapter matrices (LoRA) on top, drastically reducing memory needed for fine-tuning.
• LoRA: Low-Rank Adaptation — injects small trainable low-rank matrices into specific model layers (here, the attention projection layers) instead of updating all model weights.
• SFT (Supervised Fine-Tuning): Training a pretrained model further on labeled input/output pairs (here, tutor-style Q&A) using a standard next-token prediction loss.
• NF4: "NormalFloat4," a 4-bit data type used in QLoRA's quantization scheme, optimized for normally distributed weights.

More Information

See the full training pipeline, dataset generation code, evaluation results, and a FastAPI comparison demo in the GitHub repository.

Model Card Authors

Ashish Kumar Jha (@Ash8389 / @ash270)

Model Card Contact

Open an issue on the GitHub repository or reach out via Hugging Face: @ash270

Framework versions

• PEFT 0.16.0