oss-ai-1_5b-gguf

OSS AI event extractor (1.5B)

Fine-tune of Qwen/Qwen2.5-1.5B-Instruct, quantised to Q4_K_M, that pulls structured event data out of free-form Discord messages for the Vaktovian Systems Department.

What it does

Given a message like:

SURGE @ VAK Eular Station LOSS

it returns:

{"CLAN": "The Imperial Insurgence", "BASE": "Eular Station", "TYPE": "Defense", "RESULT": "Loss"}

Only those four fields. The date, the author and the screenshot come off the Discord message object instead, because those were never a language problem and asking a model for them just adds somewhere for it to be wrong.

Results

Measured on 892 held-out messages written the way members actually post, scored as exact match across all four fields.

The middle row matters. Most of the work is done by a lookup table and a few deterministic rules, and the model is worth roughly 7 points on top of that. Worth having, but not the whole story. In production the bot also overrules the model with the posted abbreviation when the database recognises it, which recovers most of the remaining misses.

Training

• 8,020 examples, 2 epochs, LoRA r=16 alpha=32 on all attention and MLP projections
• Loss masked to the answer only, so it is not trained on the prompt or on padding
• Dynamic padding rather than a fixed length, the median example is 209 tokens

Usage

from llama_cpp import Llama

llm = Llama(model_path="oss_ai_1_5b_q4.gguf", n_ctx=512)
out = llm.create_chat_completion(
    messages=[
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": "SURGE @ VAK Eular Station LOSS"},
    ],
    temperature=0,
)

The system prompt has to match the one it was trained on, which is in utils/constants.py in the bot repo. A different prompt will make it noticeably worse.