lora-slm-adapter

Qwen2.5-0.5B-Instruct LoRA Adapter

This repository contains a Low-Rank Adaptation (LoRA) adapter trained on top of the Qwen/Qwen2.5-0.5B-Instruct base Small Language Model (SLM). The model was trained as part of an SLM research project.

Model Details

Model Description

This model is a parameter-efficient adapter module designed to modify the text-generation behaviour of Qwen's 0.5B parameter instruction-tuned model. By targeting internal attention blocks, the adapter allows for specialized downstream capability while preserving the vast linguistic footprint of the base model.

• Developed by: rounak26 (Internship Research Project, IIIT Agartala)
• Model type: PeftModel (LoRA Adapter)
• Language(s) (NLP): English / Multilingual (Inherited from Qwen2.5)
• License: Apache 2.0 (Inherited from Qwen2.5 base model)
• Finetuned from model: Qwen/Qwen2.5-0.5B-Instruct

Model Sources

• Repository: rounak26/lora-slm-adapter
• Paper: Quantifying the Carbon Emissions of Machine Learning (Referenced for reporting frameworks)

Uses

Direct Use

This adapter cannot be initialized standalone. It must be explicitly injected into the base Qwen/Qwen2.5-0.5B-Instruct model weights at runtime using the peft library.

Downstream Use

It is tailored for resource-constrained edge deployments, CPU inference configurations, or low-latency conversational agent environments leveraging Small Language Models (SLMs).

Out-of-Scope Use

This model should not be deployed in high-risk autonomous operational pipelines or medical triage frameworks without rigorous safety wrappers. It is explicitly not designed to operate outside the bounded parameters of its targeted text-generation tasks.

Bias, Risks, and Limitations

Given its small size (0.5 Billion parameters), this architecture is highly prone to structural hallucinations when prompted with complex logical constraints or out-of-distribution math. It also inherits any downstream data biases present in the core Qwen2.5 pretraining mix.

Recommendations

Users are strongly encouraged to employ strict output verification pipelines, log alignment checking, and temperature scaling down to low variations (e.g., 0.2 or 0.1) during generations to maintain contextual consistency.

How to Get Started with the Model

Use the code snippet below to initialize the base architecture and merge your custom LoRA weights:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model_id = "Qwen/Qwen2.5-0.5B-Instruct"
adapter_id = "rounak26/lora-slm-adapter"

# Load base model tokenizer and model weights
tokenizer = AutoTokenizer.from_pretrained(base_model_id)
model = AutoModelForCausalLM.from_pretrained(
    base_model_id, 
    torch_dtype=torch.float16, 
    device_map="auto"
)

# Inject the LoRA adapter layers
model = PeftModel.from_pretrained(model, adapter_id)

Training Details

Training Data

The model was optimized using specific contextual fine-tuning datasets formatted into Qwen’s native ChatML structure to ensure stable dialogue tracking.

Training Procedure

Training Hyperparameters

• Training regime: fp16 non-mixed precision / Auto LFS optimization
• LoRA Rank (r): 8
• LoRA Alpha ($\alpha$): 16
• Target Modules: q_proj, v_proj, k_proj, o_proj (Attention layers)
• Bias Configuration: none

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

• Hardware Type: User Local GPU (e.g., NVIDIA RTX series)
• Hours used: ~1-2 hours of local compute time per run
• Cloud Provider: Local Hardware (Private Infrastructure)
• Compute Region: India
• Carbon Emitted: Minimal local baseline footprints (< 0.5 kg CO2eq)

Technical Specifications

Model Architecture and Objective

Causal Language Modeling (CAUSAL_LM) objective utilizing low-rank weight updates directly inserted adjacent to the Query, Key, Value, and Output projection matrices of the base Transformer block.

Citation

BibTeX:

@article{lacoste2019quantifying,
  title={Quantifying the Carbon Emissions of Machine Learning},
  author={Lacoste, Alexandre and Luccioni, Alexandra and Schmidt, Victor and Dandres, Thomas},
  journal={arXiv preprint arXiv:1910.09700},
  year={2019}
}

Model Card Contact

For questions regarding this research run, reach out via the Hugging Face Community hub at rounak26.

Framework versions

• PEFT 0.20.0
• Transformers (Qwen2ForCausalLM architecture)