talvion-support-sft-kb-v2-lora

Talvion Support SFT KB v2 LoRA

LoRA adapter for Qwen/Qwen2.5-3B-Instruct, trained for structured analysis and draft responses to Russian B2B SaaS support tickets.

The training set contains 1,741 synthetic domain examples and 1,741 knowledge-base-grounded safe variants. The frozen test set of 171 tickets was not used for training or preference-pair selection.

Evaluation

Deterministic generation on the frozen 171-ticket test set:

These metrics measure structure and classification quality. They do not prove that every generated factual statement is grounded. The production pipeline therefore validates drafts against the local knowledge base and escalates unsupported answers to an operator.

Training

• Method: QLoRA SFT, 3 epochs
• LoRA: rank 16, alpha 32, dropout 0.05
• Target modules: attention and MLP projection layers
• Context length: 1,536 tokens
• Quantization: NF4 with double quantization and bfloat16 compute
• Optimizer: paged AdamW 8-bit
• Learning rate: 2e-4 with cosine schedule

Apply this adapter to Qwen/Qwen2.5-3B-Instruct with PEFT. The runtime system prompt and response schema from the project are required for comparable output.

Limitations

The corpus is synthetic and specific to the fictional Talvion product domain. The model can still produce unsupported statements when used without retrieval and output validation. It should not be used as an unrestricted factual source.