tiny-log-parser

tiny-log-parser

LoRA adapter for Qwen3-4B that normalizes log lines from six wire formats into a canonical 7-field JSON record. Paired with a deterministic epoch pre-pass it reaches 100% exact match vs 83.5% for gemini-3.1-pro-preview on a 200-example held-out test set.

• Code, eval harness, writeup: https://github.com/arshirazi97/tiny-log-parser
• Runnable demo:
• Base model: unsloth/qwen3-4b-unsloth-bnb-4bit

What it does

Takes a log line in syslog RFC3164, nginx combined, logfmt, Java/log4j, container JSON, or a bracketed application format, and emits:

timestamp (ISO8601 UTC, second precision) · level (one of CRITICAL, ERROR, WARNING, INFO, DEBUG) · service · trace_id · status_code · latency_ms (integer) · message

Results

200-example held-out test set, same spec given to both systems, same exact-match verifier, all seven fields must match.

The adapter alone loses. Every one of its 54 misses is a bare-epoch timestamp — integer division into calendar arithmetic the model cannot do reliably. Scaling training data 5k → 20k moved that 0.5 points, so the conversion is routed to datetime.fromtimestamp() instead of learned. It fires on 41 of 200 inputs.

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

BASE = "unsloth/qwen3-4b-unsloth-bnb-4bit"
tok = AutoTokenizer.from_pretrained(BASE)
model = PeftModel.from_pretrained(
    AutoModelForCausalLM.from_pretrained(BASE, device_map="auto"),
    "arshirazi/tiny-log-parser").eval()

Requires a CUDA GPU — the base is 4-bit bitsandbytes, which does not run on Apple Silicon or CPU. The adapter expects the exact prompt spec in eval.py (build_prompt(line, [])) zero-shot; a different prompt format degrades output. The epoch pre-pass lives in score_hybrid.py.

Training

4-bit QLoRA, r=16, 2 epochs, response-masked so loss lands on the JSON only. 20,000 synthetic examples generated canonical-record-first — the label exists before the input, so every example is correct by construction. Train and test draw from disjoint time windows (Jan–May vs Jun–Jul). Single RTX 2000 Ada (16 GB), ~2.5 hours.

Limitations

The test set is synthetic, drawn from the same six renderers as training. Disjoint time windows prevent timestamp memorization but not format memorization. Read the 100% as "solved within its stated distribution," not as a claim about production logs.

Real logs are harder: multiline stack traces, truncated lines, vendor quirks, and formats outside these six are absent. Hand-written lines outside the generator's parameter range surfaced two gaps the test set did not catch — syslog severity 5/6 mapping, and a placeholder service name invented on a truncated line.

Compared against one baseline, scored once, at temperature 0.

Framework versions

• PEFT 0.20.0