qwen3-flask-full

🔧 Qwen3-Flask Full Fine-Tuned Model (Merged LoRA Adapter)

This is a fully merged fine-tuned model based on Qwen/Qwen3-0.6B-Base. It was trained on a rich developer-focused Q&A dataset covering Flask internals. Fine-tuning was done using LoRA (Low-Rank Adaptation) and later merged into the base model for ease of deployment.

---

🧠 Project Objective

Flask’s documentation, while comprehensive, often lacks developer-centric summaries or Q&A-style explanations. This project bridges that gap by:

• Turning raw Flask source code and docstrings into instructional Q&A data
• Fine-tuning a strong open LLM (Qwen) to produce developer-style responses
• Providing both LoRA and full-weight versions for flexible deployment

---

📚 Use Cases

• 🔍 Explaining internal APIs and decorators (before_request, url_defaults, etc.)
• 🔄 Clarifying Flask upgrade/migration behaviors
• 📘 Summarizing docstring-heavy logic in natural Q&A form
• ⚙️ Assisting junior devs learning Flask internals

---

🧪 Dataset Creation

A custom script extracted:

• All functions, classes, methods, and docstrings from the Flask codebase (.py files)
• Filtered to 345 valid logic-rich chunks out of 804 total
• Each chunk was passed to Gemini using a Q&A generation prompt

Total Q&A pairs generated: 1425

Example:

{
  "instruction": "What does `before_request` do in Flask?",
  "input": "This function runs before each request, useful for checking login sessions, etc.",
  "output": "`before_request` is a Flask decorator used to register a function that runs before each request. It is commonly used to implement access control logic or session checks."
}

## 🧪 Fine-Tuning Details

- **Model**: Qwen/Qwen3-0.6B-Base
- **PEFT Type**: LoRA (r=8, alpha=16)
- **Quantization**: 4-bit NF4 using `bitsandbytes`
- **Training Library**: `transformers`, `peft`, `datasets`
- **Device**: Single NVIDIA RTX 3060 6GB VRAM (consumer laptop)
- **Dataset**: 1000+ cleaned Q&A pairs from Flask official documentation

---

## 🧠 Prompt Format

The model was fine-tuned on Alpaca-style prompts:

```text
### Instruction:
<What do you want to know?>

### Input:
<Any supporting context>

### Response:
<Model-generated answer>

🏗️ Training

Used PEFT + LoRA with:

• Rank: 16
• Alpha: 32
• Target modules: query_key_value
• Epochs: 3
• Dataset: 1425 Q&A JSONL entries

📊 Evaluation

• Evaluated using BLEU, ROUGE, and manual inspection.
• Improved consistency and structured response formatting for framework-specific queries.

🔍 Inference

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("devanshdhir/qwen3-flask-full", trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained("devanshdhir/qwen3-flask-full", trust_remote_code=True)

prompt = """### Instruction:
What is the purpose of `url_defaults` in Flask?
### Input:
Related excerpt from docs...
### Response:"""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=300)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))