llama_3_2_1B_dsp-llm-bilingual

LLaMA 3.2–1B DSP (English–Albanian)

A lightweight LLaMA 3.2–1B model fine-tuned to act as a bilingual (English–Albanian)
Digital Signal Processing (DSP) teaching assistant.

This model is part of a university project exploring whether small modern LLMs can handle
technical DSP reasoning when trained on a verified synthetic dataset using QLoRA.

---

Intended Use

• Coursework-level DSP questions (FFT, aliasing, sampling)
• Short conceptual explanations in English or Albanian
• Mixed-language prompts (EN ↔ SQ)
• As a compact teaching assistant for students

Not intended for high-stakes or professional engineering decisions.

---

Training Overview

The model was fine-tuned using:

• QLoRA (4-bit quantisation + LoRA adapters)
• A bilingual DSP dataset (EN/SQ)
• Programmatically generated and numerically verified questions
• Teacher–student refinement using larger LLaMA and Qwen models

Training ran within a ~5–6GB VRAM budget on consumer hardware.

---

📂 Dataset

The training data includes:

• Manually written English and Albanian DSP questions
• A small English–Albanian DSP glossary
• Synthetic tasks for:
  • FFT bin indices
  • aliasing frequencies
  • basic sampling relations
• Verification scripts that recompute all numeric answers

Only samples that passed numeric checks and basic quality filters were included.

---

Evaluation (Summary)

The model was evaluated on three pipelines:

The fine-tuned model clearly outperforms the base LLaMA 3.2–1B on DSP tasks and comes close to mid-sized models in the 3B–7B range.

---

How to Use

from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "Irfanuruchi/llama_3_2_1B_dsp-llm-bilingual"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto"
)

prompt = "A signal is sampled at 8000 Hz. What FFT bin corresponds to 1000 Hz for N=256?"

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=128)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

---

License

These fine-tuned weights are based on meta-llama/Llama-3.2-1B and are therefore governed by the LLaMA 3.2 license.

The training code and surrounding project are released separately under the MIT license on GitHub:

• https://github.com/IrfanUruchi/dsp-llm-bilingual-finetuning

Please make sure you comply with both:

The LLaMA 3.2 license for the model weights, and the MIT license for the code.

---

Acknowledgements

This model was developed as part of the Introduction to Data Science course at South East European University.

Thanks to Professor Nuhi Besimi for his guidance and feedback throughout the project.