Downloads · 30 days
6
11% of all-time downloads
KocKaankk/asure-12b-tr-attr-grpo-v0.7-8field
asure-12b-tr-attr-grpo-v0.7-8field is a text generation model from KocKaankk. Use it when you need the model to write or continue text. It is set up for peft. The card lists the license as cc-by-nc-4.0.
LoRA adapter for Trendyol/Trendyol-LLM-Asure-12B, trained with GRPO (RLVR) on a custom Turkish e-commerce attribute-extraction reward function across 8 fields (vs 4 fields in the v0.6 sibling adapter).
Downloads · 30 days
6
11% of all-time downloads
All-time downloads
56
Public
Repo size
182 MB
Likes
0
Public
Click a slice to open those files.
.safetensors148 MB · 81%
From the Hugging Face model README
LoRA adapter for Trendyol/Trendyol-LLM-Asure-12B, trained with GRPO (RLVR) on a custom Turkish e-commerce attribute-extraction reward function across 8 fields (vs 4 fields in the v0.6 sibling adapter).
If you only need the 4 most common fields, the v0.6 adapter (KocKaankk/asure-12b-tr-attr-grpo) is the better choice — it beats Claude Opus 4.7 by +0.026 macro F1 on a 578-row out-of-distribution test (P<0.001).
This v0.7 adapter trades broader field coverage (8 fields instead of 4) for a slight in-distribution loss to Claude (-0.026 macro F1, not statistically significant). On OOD, our 8-field adapter still beats the base by +0.049 (P<0.001), but Claude's 8-field OOD score is currently unmeasured (API credits ran out partway during eval).
| model | macro F1 | 95% CI | Renk | Materyal | Kalıp | Desen | Kumaş Tipi | Yaka Tipi | Kol Tipi | Bel |
|---|---|---|---|---|---|---|---|---|---|---|
| Asure-12B base + v4 prompt | 0.382 | [0.342, 0.421] | 0.460 | 0.190 | 0.278 | 0.475 | 0.361 | 0.628 | 0.417 | 0.264 |
| Claude Opus 4.7 v4 (frontier reference) | 0.474 | [0.438, 0.508] | 0.444 | 0.158 | 0.258 | 0.574 | 0.429 | 0.721 | 0.593 | 0.621 |
| Asure-12B + this adapter (step 250) | 0.444 | — | 0.449 | 0.189 | 0.276 | 0.571 | 0.427 | 0.676 | 0.390 | 0.575 |
| Asure-12B + this adapter, step 150 | 0.448 | [0.410, 0.484] | 0.450 | 0.196 | 0.286 | 0.571 | 0.434 | 0.676 | 0.390 | 0.583 |
Δ vs Claude on in-dist (step 150): -0.026, paired-bootstrap CI [-0.054, +0.001], P=0.97 — Claude wins by a hair, CI just barely includes zero.
The entire gap is on Kol Tipi (sleeve type), where Claude is +0.203 ahead. On all other 7 fields combined, the trained adapter is roughly at parity with Claude.
| model | macro F1 OOD | 95% CI | notes |
|---|---|---|---|
| Asure-12B base + v4 prompt | 0.323 | [0.277, 0.369] | |
| Asure-12B + this adapter (step 250) | 0.372 | [0.325, 0.418] | +0.049 over base, P<0.001 |
| Claude Opus 4.7 v4 | not measured | — | Anthropic credits ran out during eval |
| field | base v4 | Claude v4 | this adapter | Δ vs base | Δ vs Claude |
|---|---|---|---|---|---|
| Renk | 0.460 | 0.444 | 0.450 | -0.010 | +0.006 |
| Materyal | 0.190 | 0.158 | 0.196 | +0.006 | +0.038 |
| Kalıp | 0.278 | 0.258 | 0.286 | +0.008 | +0.028 |
| Desen | 0.475 | 0.574 | 0.571 | +0.096 | -0.003 |
| Kumaş Tipi | 0.361 | 0.429 | 0.434 | +0.073 | +0.005 |
| Yaka Tipi | 0.628 | 0.721 | 0.676 | +0.048 | -0.045 |
| Kol Tipi | 0.417 | 0.593 | 0.390 | -0.027 | -0.203 ← see caveats |
| Bel | 0.264 | 0.621 | 0.583 | +0.319 | -0.038 |
Where this adapter wins or matches: Materyal, Kalıp, Renk, Kumaş Tipi, Desen.
Where this adapter loses: Yaka Tipi, Kol Tipi (much), Bel (slightly).
The Bel win is the headline: GRPO learned the catalog default Normal Bel (modal value at 69%) and lifted F1 from 0.264 → 0.583 — same dynamic as v0.6's Desen → Düz story but ~3x bigger.
Given a Turkish product title (and optionally a brand), produce a strict JSON object with up to eight catalog attributes:
Renk (color), Materyal (material), Kalıp (fit), Desen (pattern) — same as v0.6.Kumaş Tipi (fabric type) — "Dokuma" (woven) is the catalog default for most clothing.Yaka Tipi (neckline) — TOPS ONLY; emit null for everything else.Kol Tipi (sleeve type) — TOPS ONLY; emit null for sleeveless or non-tops.Bel (waist style) — PANTS/SKIRTS ONLY; emit null elsewhere; default "Normal Bel".Use null (literal JSON null, not the string "null") when an attribute can't be inferred or doesn't apply to the product type.
Input: Marka: Mavi\nÜrün adı: Erkek Slim Fit %100 Pamuk Basic Tişört Beyaz
Output:
{"Renk": "Beyaz", "Materyal": "Pamuklu", "Kalıp": "Slim", "Desen": "Düz",
"Kumaş Tipi": "Örme", "Yaka Tipi": "Bisiklet Yaka", "Kol Tipi": "Kısa Kol", "Bel": null}
This is a PEFT/LoRA adapter — load it on top of the base model.
vllm serve Trendyol/Trendyol-LLM-Asure-12B \
--enable-lora \
--lora-modules grpo-v0.7=KocKaankk/asure-12b-tr-attr-grpo-v0.7-8field \
--max-lora-rank 16 \
--served-model-name asure-12b-grpo-v0.7 \
--dtype bfloat16 --max-model-len 4096
Then call the OpenAI-compatible endpoint with model="asure-12b-grpo-v0.7". Use the v4 prompt — there is no other supported prompt for this adapter; see eval/prompts.py (V4 constant).
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
base = AutoModelForCausalLM.from_pretrained(
"Trendyol/Trendyol-LLM-Asure-12B",
torch_dtype=torch.bfloat16,
device_map="auto",
)
model = PeftModel.from_pretrained(base, "KocKaankk/asure-12b-tr-attr-grpo-v0.7-8field")
tok = AutoTokenizer.from_pretrained("KocKaankk/asure-12b-tr-attr-grpo-v0.7-8field")
| value | |
|---|---|
| algorithm | GRPO via TRL |
| base model | Trendyol/Trendyol-LLM-Asure-12B (12B params, Turkish base) |
| adapter | LoRA, rank=16, alpha=32, dropout=0.05, target_modules="all-linear" |
| quantization (training) | 4-bit NF4 (QLoRA, double-quant, bf16 compute) |
| reward function | per-row macro F1 across 8 fields, strict normalized exact-match |
| reward parser | strict JSON; markdown fences stripped; case-canonicalized field names |
| rollouts | vLLM colocated, G=8 generations per prompt, T=0.8 |
| optimizer | AdamW 8-bit |
| learning rate | 1.5e-5, cosine schedule, 10% warmup |
| KL coefficient (beta) | 0.02 |
| effective batch | 4 prompts × 8 generations × 2 grad-accum = 64 rollouts per gradient update |
| training steps | 300 (saved every 50, this is checkpoint-250 — peak by held-out OOD F1) |
| training data | 2,009 Turkish Trendyol product titles + Trendyol-provided gold attribute labels |
| held-out test | 200 prompts (deterministic SHA1 split, frozen across v0.6 and v0.7 — same SHA256) |
| compute | 1× NVIDIA L40S (48 GB), AWS g6e.4xlarge, ~3:00 wallclock |
| training cost | ~$7 in on-demand GPU |
| step | macro F1 in-dist | Δ vs base |
|---|---|---|
| base | 0.382 | ref |
| 50 | 0.410 | +0.026 |
| 100 | 0.418 | +0.034 |
| 150 | 0.448 | +0.064 (peak — tied with step 300) |
| 200 | 0.441 | +0.057 |
| 250 | 0.444 | +0.060 (peak by OOD F1 — this published checkpoint) |
| 300 | 0.448 | +0.064 (slight overfit to in-dist) |
We publish step 250 (peak OOD) rather than step 150 (peak in-dist) because OOD generalization is the better signal for production use. The two are within 0.004 in-dist anyway.
null when uncertain rather than guessing the modal value. Recall is 0.283 vs Claude's 0.660; precision (when it does emit) is actually higher than Claude (0.625 vs 0.538). This is the entire reason v0.7 doesn't beat Claude in-dist. A targeted v0.8 prompt fix is planned.Yaka/Kol Tipi apply only to tops and Bel only to pants/skirts. It does this correctly most of the time, but when in doubt about Kol Tipi specifically, it defaults to null too aggressively (see above).Full code, dataset scrapers, training script, eval pipeline, Terraform infra: github.com/KocKaankk/tr-rl (repo will be live once published).
The training run that produced this adapter is documented end-to-end in notes/v0.7-8field.md of that repo.
If you use this adapter in research:
@misc{karakas2026asuregrpov07,
author = {Karakaş, Kaan},
title = {Asure-12B + GRPO v0.7: Turkish e-commerce 8-field attribute extraction},
year = {2026},
publisher = {Hugging Face},
howpublished = {\url{https://huggingface.co/KocKaankk/asure-12b-tr-attr-grpo-v0.7-8field}},
}
CC BY-NC 4.0 — inherited from the base model (Trendyol/Trendyol-LLM-Asure-12B).