Llama-3.2-3B-TechWriter-Instruct

Llama-3.2-3B TechWriter (merged)

Built with Llama

Merged QLoRA fine-tune of meta-llama/Llama-3.2-3B-Instruct for semiconductor and data-center interconnect technical-marketing and documentation style: product briefs, datasheets, application notes, and user-guide CLI sections.

This is the inference repo. Load it with AutoModelForCausalLM or pipeline("text-generation"). You do not need gated access to the Meta base once these merged weights are on the Hub — you still must follow the Llama 3.2 Community License and Acceptable Use Policy.

This is an unofficial specialist checkpoint trained on a private mix of extracted vendor PDFs plus cleaned synthetic docs. It will still invent SKUs if you ask it to write a brief for a product that was never in the gold data.

Adapter-only (smaller download, resume / compose):

Shankarblr/Llama-3.2-3B-TechWriter-LoRA

What it is for

• Product briefs, datasheet feature lists, application notes
• Host-adapter / switch CLI user-guide sections
• Spec extraction and short grounded QA over a pasted excerpt
• Internal draft generation that should stay on-genre and internally consistent (one process node, one primary throughput, one form factor)

Not for:

• Authoritative datasheet numbers without a human / source check
• Legal, safety, or customer-facing specs shipped as-is
• Languages other than English
• General chat outside semiconductor interconnect / storage / DPU content

Load and run

Use the Llama 3.2 Instruct chat template. Do not hand-roll Qwen ChatML (<|im_start|>). The SFT data is ChatML-shaped JSONL (messages[{role, content}]); training ran it through tokenizer.apply_chat_template, so inference must do the same.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

REPO = "Shankarblr/Llama-3.2-3B-TechWriter-Instruct"

tokenizer = AutoTokenizer.from_pretrained(REPO)
if tokenizer.pad_token is None:
    tokenizer.pad_token = tokenizer.eos_token

model = AutoModelForCausalLM.from_pretrained(
    REPO,
    torch_dtype=torch.float16,
    device_map="auto",
)

messages = [
    {
        "role": "system",
        "content": (
            "You are a technical marketing and documentation writer for semiconductor "
            "and data-center interconnect products. Write clear, structured content. "
            "Match the requested document type. Keep specifications internally consistent: "
            "one process node, one primary throughput, and one form factor unless the "
            "source explicitly lists options. Do not invent conflicting SKUs or CLI syntax."
        ),
    },
    {
        "role": "user",
        "content": "Draft a product brief covering a 40/50/100GbE converged network adapter series.",
    },
]

prompt = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
out = model.generate(
    **inputs,
    max_new_tokens=1024,
    do_sample=False,
    pad_token_id=tokenizer.eos_token_id,
)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Pipeline

from transformers import pipeline, AutoTokenizer

REPO = "Shankarblr/Llama-3.2-3B-TechWriter-Instruct"
tok = AutoTokenizer.from_pretrained(REPO)
if tok.pad_token is None:
    tok.pad_token = tok.eos_token

pipe = pipeline(
    "text-generation",
    model=REPO,
    tokenizer=tok,
    max_new_tokens=1024,
    do_sample=False,
)

prompt = tok.apply_chat_template(
    [
        {"role": "system", "content": "You are a technical marketing and documentation writer for semiconductor and data-center interconnect products."},
        {"role": "user", "content": "For an enterprise 1/10/25GbE Ethernet switch series, author the Key Features portion of the product brief."},
    ],
    tokenize=False,
    add_generation_prompt=True,
)
print(pipe(prompt, return_full_text=False)[0]["generated_text"])

If you see Both max_new_tokens and max_length(=20), the shipped generation_config.json still has a leftover max_length. Prefer passing only max_new_tokens at generate time, or edit that file before upload. Greedy decode (do_sample=False) will also ignore leftover temperature / top_p flags — that warning is expected.

Training

Same recipe as the Qwen2.5-1.5B TechWriter run; only the base, wall time, and eval numbers changed.

PEFT printed a 401 on meta-llama/Llama-3.2-3B-Instruct/config.json at save time and assumed the vocabulary was not modified. That assumption is correct — this run did not add tokens.

Eval (teacher-forced next token, not open generation)

Eval runtime 92.6 s · 7.308 samples/s · 170 steps · 8.021e+06 eval tokens.

Mean train loss over the full run: 0.3507 (early epochs are higher; late-epoch train batches sit around 0.11–0.13 with mean token accuracy ~0.95–0.958). Grad norms stayed small (~0.05–0.09). Cosine LR decayed from ~1.2e-5 at epoch 2.55 to 6.43e-9 at the last step.

Read token accuracy correctly. 95.1% is “the model assigned the gold next token” on the held-out ChatML strings. It is not factual accuracy on silicon SKUs, and it is not a win-rate against a human editor.

Same recipe, Qwen 1.5B vs this 3B

Slightly tighter teacher-forced numbers, as expected from the larger base. Open-generation quality still needs a human pass.

Dataset mix

6,765 rows after dropping dirty synthetic gold (multi-node, multi-throughput, grammar doubles, stubs).

Doc types: user_guide 2,772 · product_brief 2,528 · datasheet 613 · application_note 305 · technology_brief 229 · white_paper 222 · competitive_report 96.

Extracted PDFs cover converged network adapters, Ethernet switch silicon, host-adapter CLI, storage / fabric features, and related interconnect material. A large share of generate gold is still synthetic house-style prose, so the model can emit invented series names if the prompt does.

Intended prompt style

The train-time system prompts were:

1. Writer — semiconductor / interconnect house style; one process node / throughput / form factor.
2. Editor — revise copy; do not add contradictory specs.
3. Product specialist — answer only from the pasted excerpt; otherwise say you cannot determine it.

Match those at inference. Grounded QA quality collapses if you drop the specialist system prompt.

Limitations

• 3.21B parameters. Long, consistent datasheets still drift, less than the 1.5B Qwen run but not gone.
• Synthetic SKUs in the pre-clean pool taught a habit of inventing product families. The recommended set drops the worst contradictions; it does not delete every invented family.
• Extracted CLI gold is the reliable part. Treat generated flags and identifiers as drafts.
• No DPO / GRPO on this published checkpoint (those datasets exist separately).
• Eval is teacher-forced loss/accuracy only.
• Base is gated. Merged inference does not call Meta’s repo; adapter reload does.

Files to upload (merged repo)

Do not upload checkpoint-*, optimizer.pt, rng_state.pth, or the 4-bit train-time weights.

Related

• Base: meta-llama/Llama-3.2-3B-Instruct
• Method: QLoRA (NF4) + PEFT LoRA, TRL SFT
• Adapter: Shankarblr/Llama-3.2-3B-TechWriter-LoRA
• Sister run (same data/recipe): Shankarblr/Qwen2.5-1.5B-TechWriter-Instruct

License

Built with Llama

Llama 3.2 is licensed under the Llama 3.2 Community License, Copyright © Meta Platforms, Inc. All Rights Reserved.

Use of this model is also subject to the Llama 3.2 Acceptable Use Policy.

This checkpoint is an unofficial style model and is not affiliated with any semiconductor vendor.