llm-course-hw3-tinyllama-qlora-structured

Model Card: QLoRA Tuned TinyLlama 1.1B Chat v1.0

Model Overview

• Base Model: TinyLlama/TinyLlama-1.1B-Chat-v1.0

• Fine Tuning Strategy: QLoRA (Efficient Fine Tuning of Quantized LLMs)

• Architecture & Quantization Details:

o The model is loaded with a 4 bit quantization configuration (using BitsAndBytes with nf4 quantization type and bfloat16 compute data type), enabling significant memory savings.

o PEFT is applied through a LoraConfig with carefully chosen hyperparameters such as lora_alpha, r, and lora_dropout to balance robustness and training speed.

o Target modules are selected from the projection layers (e.g., q_proj, k_proj, v_proj, and o_proj) where LoRA–style adaptation is applied.

Intended Use

• Primary Application: Low resource sentiment classification on tweets, leveraging large model capacity with reduced memory footprint.

• Task Suitability: Ideal for environments with constrained computational resources while still benefiting from a 1.1B parameter model’s performance.

• Deployment: Suitable for both research and production settings where the combination of quantization and parameter efficient fine tuning offers an optimal trade off between latency and accuracy.

Training Data & Process

• Dataset: cardiffnlp/tweet_eval (sentiment subset)

• Task: Classify tweets into one of three sentiment categories: negative, neutral, or positive.

• Processing Pipeline:

o A conversational prompt structure is used with a system prompt instructing for a single-word sentiment output.

o Tokenization and prompt formatting are carefully managed to ensure compatibility with SFTTrainer.

o Advanced training settings include gradient accumulation, cosine learning rate scheduling, and optimized batch sizes to accommodate the quantized model.

• Quantization Configuration:

o The BitsAndBytes configuration employs 4 bit quantization with nf4 and double quantization enabled, ensuring an efficient memory usage profile during training and inference.

Evaluation & Performance

• Metrics: Evaluated on Macro F1 score using the standard test split for sentiment classification.

• Observations: The QLoRA tuned model shows competitive performance despite its reduced memory footprint, successfully balancing speed and accuracy in sentiment prediction.

• Usage Recommendation: This model card is ideal for users seeking to deploy a larger conversational model in memory constrained environments without significantly compromising performance.

Results： tried my best to get as much F1 as I could , but the best was 0.21, was sure it could be better , but somehow I cant re-realize that result, either it's because of the input or the hyper params. Not sure so far.

Limitations & Caveats

• Quantization may introduce variability in output quality; therefore, benchmarking across different hardware is recommended.

• The extensive prompt‐processing pipeline requires careful tuning; minor changes in prompt template may affect performance.

• Domain adaptation is critical: while optimized for tweet sentiment, further tuning might be necessary for other text genres.