scene-seg-llama-3-2-3b-instruct-fold_A

scene-seg-llama-3-2-3b-instruct-fold_A

Status: E0 debug / pilot — metrics are not reportable.

LoRA adapter from the Pilot 0 smoke run (E0_smoke). Trained on fold_A (debug split: Excel + Aus guter Familie → eval Effi Briest). This artifact validates the HF Jobs finetuning pipeline only; it is not a paper-comparable baseline.

Task

German literary scene-border detection: given a target sentence and local context, predict whether the sentence starts a new scene (BORDER) or continues the current scene (NOBORDER).

Training summary

Training data and config: RuthonField/scene-seg-sft (fold_A).

Metrics (not for reporting)

Uploaded metrics file: metrics_llama-3-2-3b-instruct_fold_A.json.

Headline metric: scenarios.none.tol_3.macro_f1 (tolerant F1 at ±3 sentences, no post-processing).

All scenarios scored 0.0 (model predicted no borders on eval). Expected for a 684-example, 1-epoch pilot — do not cite these numbers in papers or comparisons.

For paper-comparable results, use the E1 anchor run (train_full → stss_test_2) instead.

Load and inference

Requires the 4-bit base model plus this LoRA adapter.

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="RuthonField/scene-seg-llama-3-2-3b-instruct-fold_A",
    max_seq_length=1280,
    load_in_4bit=True,
)
FastLanguageModel.for_inference(model)

messages = [{"role": "user", "content": "<your prompt>"}]
prompt = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=96, do_sample=False)
print(tokenizer.decode(output[0], skip_special_tokens=True))

Re-evaluate locally (project repo):

python src/finetune/eval_finetuned.py \
  --adapter RuthonField/scene-seg-llama-3-2-3b-instruct-fold_A \
  --eval data/processed/finetune/fold_A/eval.jsonl \
  --family L --use_unsloth --max_seq_len 1280

Links

• Project plan: FINETUNING_EXPERIMENTS_PLAN
• Next baseline: E1 anchor (train_full__to__stss_test_2)