Downloads · 30 days
364
69% of all-time downloads
slvrfivo/qwen3-4b-writing-eval-v1-merged
qwen3-4b-writing-eval-v1-merged is a text generation model from slvrfivo. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
A Qwen3-4B model adapted for the 2026 National Institute of Korean Language (NIKL) AI말평 Korean writing-scoring evaluation.
Downloads · 30 days
364
69% of all-time downloads
All-time downloads
528
Public
Parameters
4B
8.1 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors8 GB · 100%
From the Hugging Face model README
A Qwen3-4B model adapted for the 2026 National Institute of Korean Language (NIKL) AI말평 Korean writing-scoring evaluation.
Given a writing prompt and a completed Korean argumentative essay, the model generates a strict JSON object containing 1–5 scores and Korean rationales for three dimensions:
Qwen/Qwen3-4B-Instruct-2507The source repository, training configuration, evaluator, experiment log, and export pipeline are available at:
https://github.com/slvrfivo/ai-writing-eval
| Model | Mean RMSE ↓ | Mean Spearman ↑ |
|---|---|---|
| Statistical mean baseline | 0.7870 | undefined* |
| Qwen3-4B zero-shot | 1.1612 | 0.2753 |
| Selected weighted QLoRA v1 | 0.6384 | 0.5237 |
| Class-balanced QLoRA v1b | 0.6452 | 0.5189 |
*The rounded statistical baseline is constant, so Spearman correlation is undefined.
After the preliminary round, the model also took part in the AI말평 Arena expert human-evaluation stage.
The project used a score-focused weighted causal language-model objective. Supervised target tokens were assigned role-specific weights:
| Token role | Weight |
|---|---|
| Prompt | 0.00 |
| JSON structure | 0.25 |
| Score | 10.00 |
| Rationale | 0.05 |
These weights were chosen heuristically for the competition setup. A uniform-loss QLoRA ablation was not run, so the improvement over zero-shot and statistical baselines should be attributed to the overall QLoRA fine-tuning setup rather than specifically to the weighted objective.
Training used one essay per micro-batch with 8 gradient-accumulation steps, and the weighted loss was normalized by each essay's own weight sum. Weights that differ between tokens of one essay act as listed above; weights that scale a whole essay, such as the class weights in the v1b ablation, are largely divided back out.
The final competition artifact used:
{
"repetition_penalty": 1.05
}
This setting was added after repetitive degeneration was observed on a small subset of merged-model generations. It is recorded in the source repository at configs/submission_generation.json and is applied by the BF16 export pipeline.
If a serving system overrides the model's generation configuration, preserve repetition_penalty=1.05 to reproduce the final competition serving setup.
A minimal serving command is:
vllm serve slvrfivo/qwen3-4b-writing-eval-v1-merged \
--dtype bfloat16
For competition-equivalent scoring, use the versioned prompt in the source repository:
prompts/official/writing_scoring_2026-07-20/
The repository's prompt builder and inference pipeline show the exact message construction. If the serving layer does not honor the model's saved generation_config.json, pass repetition_penalty=1.05 explicitly in the request/server generation settings.
The expected output is one JSON object:
{
"content": {
"score": 4,
"rationale": "..."
},
"organization": {
"score": 3,
"rationale": "..."
},
"expression": {
"score": 4,
"rationale": "..."
}
}
Each score must be an integer from 1 to 5.
Qwen/Qwen3-4B-Instruct-2507 (Apache-2.0)This model artifact is provided as a competition/research artifact, not as a validated high-stakes scoring system. The model card does not grant additional rights over competition data or third-party materials.