Downloads · 30 days
11
14% of all-time downloads
JoaoZaokk/Qwen3-4B-Thinking-2507-Heretic-CodeFeedback-Agentic-Tessa-1K-LoRA
Qwen3-4B-Thinking-2507-Heretic-CodeFeedback-Agentic-Tessa-1K-LoRA is a text generation model from JoaoZaokk. Use it when you need the model to write or continue text. It is set up for peft. The card lists the license as other.
This repository contains an experimental LoRA adapter trained on top of:
Downloads · 30 days
11
14% of all-time downloads
All-time downloads
81
Public
Repo size
132 MB
Likes
1
Public
Click a slice to open those files.
.safetensors132 MB · 100%
From the Hugging Face model README
This repository contains an experimental LoRA adapter trained on top of:
JoaoZaokk/Qwen3-4B-Thinking-2507-Heretic-CodeFeedback
This adapter is a small continuation experiment using:
The goal was to test whether a small amount of agentic coding data could improve or preserve coding behavior without degrading strict code-output performance.
This is a candidate / experimental adapter, not a claimed major improvement.
I'll be testing some datasets to make the model better for coding, it a tiny improvement, not a game changer, but compared to the previous one this model didn't get worse.
In a small local Python coding benchmark, this adapter preserved the previous score:
| Model | Adapter | Passed | Pass rate | Avg tokens/s |
|---|---|---|---|---|
| Before | heretic_F_lora_python5000_codefeedback5000 | 9/10 | 90.00% | 7.80 |
| After | heretic_F_lora_tessa_agentic_1000_test | 9/10 | 90.00% | 7.86 |
Delta:
| Metric | Value |
|---|---|
| Passes | 0 |
| Pass rate | 0.00% |
| Avg tokens/s | +0.05 |
Unlike the OpenCodeInstruct continuation experiment, this Tessa-based adapter did not regress on the small strict-code benchmark.
| Item | Value |
|---|---|
| Base model | JoaoZaokk/Qwen3-4B-Thinking-2507-Heretic-CodeFeedback |
| Input adapter | heretic_F_lora_python5000_codefeedback5000 |
| Dataset | smirki/Agentic-Coding-Tessa |
| Samples used | 1,000 |
| Sequence length | 1024 |
| Epochs | 1 |
| Learning rate | 1e-6 |
| Training method | QLoRA / LoRA |
| Quantized loading during training | 4-bit NF4 |
Benchmark artifacts are included under:
benchmark/
Files:
benchmark/before_summary.md
benchmark/after_summary.md
benchmark/COMPARISON.md
benchmark/before_results.jsonl
benchmark/after_results.jsonl
This adapter is intended for testing:
It should be compared against the main CodeFeedback model before use in any serious coding workflow.
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel
import torch
base_model = "JoaoZaokk/Qwen3-4B-Thinking-2507-Heretic-CodeFeedback"
adapter = "JoaoZaokk/Qwen3-4B-Thinking-2507-Heretic-CodeFeedback-Agentic-Tessa-1K-LoRA"
tokenizer = AutoTokenizer.from_pretrained(base_model, trust_remote_code=True)
bnb_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_quant_type="nf4",
bnb_4bit_compute_dtype=torch.float16,
bnb_4bit_use_double_quant=True,
)
model = AutoModelForCausalLM.from_pretrained(
base_model,
quantization_config=bnb_config,
device_map="auto",
trust_remote_code=True,
)
model = PeftModel.from_pretrained(model, adapter)
model.eval()
This is an experimental LoRA adapter.
The benchmark used here is small and should not be treated as a formal coding leaderboard. It is mainly useful for local before/after regression testing.
This adapter preserved the current local benchmark score, but further testing is needed before treating it as a better general-purpose coding model.