katyusha-3.5-2B

🩺 Radiology Assistant (Qwen3.5-2B Fine-Tuned with QLoRA)

📌 Model Overview

This model is a fine-tuned version of Qwen3.5-2B using QLoRA (4-bit) for efficient training. It is specialized for radiology-related tasks, trained on the unsloth/Radiology_mini dataset.

The goal of this model is to assist with:

• Radiology report understanding
• Medical question answering (radiology domain)
• Clinical text interpretation

---

⚙️ Model Details

• Base Model: unsloth/Qwen3.5-2B
• Fine-Tuning Method: QLoRA (4-bit)
• Framework: Likely Unsloth + Hugging Face Transformers
• Task Type: Instruction Fine-Tuning / Text Generation

---

📊 Training Configuration

Dataset

• Name: unsloth/Radiology_mini
• Split Used: train
• Subset: default
• Evaluation Split: None

Hyperparameters

• Epochs: 30
• Max Sequence Length: 2048
• Learning Rate: 2e-4
• Training Method: Parameter-Efficient Fine-Tuning (LoRA)

---

🚀 Intended Use

✅ Suitable for:

• Radiology Q&A systems
• Medical NLP research (radiology domain)
• Clinical report summarization
• Educational tools for radiology

❌ Not suitable for:

• Real-world medical diagnosis
• Clinical decision-making
• Use without human medical supervision

---

⚠️ Limitations

• Trained on a small dataset → may lack generalization
• May produce hallucinated medical facts
• Not validated for clinical accuracy
• Bias may exist depending on dataset content

---

🧪 Example Usage

from transformers import AutoTokenizer, AutoModelForCausalLM

model_name = "gawadx1/katyusha-3.5-2B"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

prompt = "Explain what a chest X-ray showing opacity might indicate."

inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=150)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

---

🏗️ Training Procedure

This model was trained using:

• QLoRA (4-bit quantization) for memory efficiency
• Fine-tuning only LoRA adapters instead of full model weights
• Optimized for low-resource hardware

---

📈 Evaluation

⚠️ No formal evaluation metrics were used.

Suggested future evaluation:

• BLEU / ROUGE for report generation
• Medical QA benchmarks
• Human expert evaluation

---

🤝 Acknowledgments

• Dataset: unsloth/Radiology_mini
• Base Model: Qwen team
• Training Framework: Unsloth

---

📜 License

Check the license of:

• Base model (Qwen3.5-2B)
• Dataset (Radiology_mini)

---

👨‍💻 Author

• Gawadx1

---

🔥 Future Improvements

• Add evaluation dataset
• Train on larger medical datasets
• Add safety alignment (RLHF / DPO)
• Improve hallucination control

---