Downloads · 30 days
43
16% of all-time downloads
lamp-qwen-training
MrMoeeee/lamp-qwen-training
lamp-qwen-training is a text generation model from MrMoeeee. Use it when you need the model to write or continue text. The card lists the license as apache-2.0.
Fine-tuned variants of Qwen 2.5-1.5B-Instruct for the lampAI project, which controls a 172-LED lamp via natural language. The model generates JSON light programs from plain English descriptions.
All-time downloads
272
Public
Repo size
2 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf2 GB · 99%
At a glance
- Task
- Text Generation
- License
- apache-2.0
- Access
- Public
- Created
- Feb 20, 2026
- Updated
- Feb 20, 2026
- SHA
- 74c6ca0c
Try a prompt
Base models
Datasets
Model card
From the Hugging Face model README
LAMP Qwen 2.5-1.5B — LoRA vs Full Fine-Tune
Fine-tuned variants of Qwen 2.5-1.5B-Instruct for the lampAI project, which controls a 172-LED lamp via natural language. The model generates JSON light programs from plain English descriptions.
Models
| Variant | File | Size | Method | Eval Loss |
|---|---|---|---|---|
| LoRA | lamp-qwen-1.5b-lora-unsloth.Q4_K_M.gguf | 941 MB | QLoRA (rank=32, alpha=64) | 0.0263 |
| Full | lamp-qwen-1.5b-full-unsloth.Q4_K_M.gguf | 941 MB | Full fine-tune (all 1.5B params) | 0.0278 |
Both models are quantized to Q4_K_M for efficient inference on Raspberry Pi 5 (16 GB RAM) via Ollama.
Training Details
Dataset
- 2,268 training / 253 validation examples
- Each example: natural language request -> JSON light program for 172 LEDs
- System prompt instructs the model to output valid JSON with LED color/animation data
Hyperparameters
| Setting | LoRA | Full Fine-Tune |
|---|---|---|
| Base model | unsloth/Qwen2.5-1.5B-Instruct | unsloth/Qwen2.5-1.5B-Instruct |
| Trainable params | 36.9M (3.5%) | 1.54B (100%) |
| Learning rate | 2e-4 | 2e-4 |
| Batch size | 4 x 4 grad accum = 16 effective | 4 x 4 grad accum = 16 effective |
| Max epochs | 20 | 20 |
| Early stopping patience | 3 evals | 5 evals |
| Eval frequency | Every 50 steps | Every 50 steps |
| Optimizer | AdamW 8-bit | AdamW 8-bit |
| LR scheduler | Cosine | Cosine |
| Precision | bf16 | bf16 |
| Warmup | 5% | 5% |
Results
| Metric | LoRA | Full Fine-Tune |
|---|---|---|
| Final eval loss | 0.0263 | 0.0278 |
| Final train loss | 0.0686 | 0.0510 |
| Early stop epoch | 6.0 (step 850) | 5.6 (step 800) |
| Training time | 23.1 min | 20.6 min |
Key finding: LoRA slightly outperformed full fine-tune on eval loss (0.0263 vs 0.0278) while training only 3.5% of parameters. Both converged to similar quality in similar time on an NVIDIA H200.
Eval Loss Curves
LoRA (best: 0.0263 at epoch ~5.3):
| Epoch | 0.4 | 1.1 | 1.4 | 1.8 | 2.1 | 2.5 | 2.8 | 3.2 | 3.5 | 3.9 | 4.2 | 4.6 | 4.9 | 5.3 | 5.6 | 6.0 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Loss | .059 | .043 | .037 | .033 | .033 | .030 | .029 | .028 | .028 | .028 | .027 | .027 | .026 | .027 | .027 | .027 -> stop |
Full Fine-Tune (best: 0.0278 at epoch ~2.8):
| Epoch | 0.4 | 0.7 | 1.1 | 1.4 | 1.8 | 2.1 | 2.5 | 2.8 | 3.2 | 3.5 | 3.9 | 4.2 | 4.6 | 4.9 | 5.3 | 5.6 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Loss | .040 | .056 | .104 | .035 | .032 | .030 | .029 | .028 | .029 | .029 | .028 | .029 | .029 | .029 | .031 | .030 -> stop |
Hardware
- Training: NVIDIA H200 (140 GB VRAM), RunPod
- Inference target: Raspberry Pi 5 (16 GB RAM), Ollama
Usage
Deploy on Raspberry Pi 5 with Ollama
Download the GGUF and Modelfile, then:
# LoRA variant
ollama create lamp-qwen-1.5b-lora -f Modelfile.lamp-qwen-1.5b-lora
ollama run lamp-qwen-1.5b-lora "warm and cozy"
# Full fine-tune variant
ollama create lamp-qwen-1.5b-full -f Modelfile.lamp-qwen-1.5b-full
ollama run lamp-qwen-1.5b-full "warm and cozy"
Example
Input: "warm and cozy"
Output: A JSON program with LED colors, animations, and timing for a 172-LED lamp.
Files
exports/
lamp-qwen-1.5b-lora-unsloth.Q4_K_M.gguf # LoRA model (941 MB)
lamp-qwen-1.5b-full-unsloth.Q4_K_M.gguf # Full fine-tune model (941 MB)
Modelfile.lamp-qwen-1.5b-lora # Ollama config for LoRA
Modelfile.lamp-qwen-1.5b-full # Ollama config for full
logs/
lamp-qwen-1.5b-lora/training_summary.json # LoRA training metrics
lamp-qwen-1.5b-full/training_summary.json # Full FT training metrics
checkpoint-*/trainer_state.json # Checkpoint states
Project
Part of the lampAI project — fine-tuning small LLMs to control a 172-LED lamp via natural language on a Raspberry Pi 5.
License
Apache 2.0 (same as base Qwen 2.5 model)
- Task
- Text Generation
- License
- apache-2.0
- Languages
- en
- Created
- Feb 20, 2026
- Updated
- Feb 20, 2026