gemma-3-4b-bitext-support

gemma-3-4b-bitext-support

A LoRA fine-tune of mlx-community/gemma-3-4b-it-qat-4bit (Gemma 3 4B, quantization-aware-trained to 4-bit) for customer-support-style conversations — order status, cancellations, refunds, shipping-address changes, account management, invoices, and payment methods. Trained with mlx-lm on Apple Silicon (an M5 MacBook, 16GB unified memory).

Training code and the full step-by-step tutorial this was built from: github.com/Asharib706/Finetuning

Model details

• Base model: mlx-community/gemma-3-4b-it-qat-4bit (already 4-bit quantized; this fine-tune keeps it quantized rather than dequantizing)
• Method: LoRA (via mlx_lm.lora / mlx_lm.fuse), not a full fine-tune
• LoRA target modules: self_attn.{q,k,v,o}_proj on the model's top 16 transformer layers (of 34 total)
• LoRA rank: 16, scale: 40.0 (mlx-lm expresses LoRA strength as a single scale value; the equivalent alpha in frameworks that separate the two is scale × rank = 640)
• Dropout: 0.05 on the LoRA branch
• Trainable parameters: 4.194M / 4551.516M base params (0.092%)
• Precision: trained and fused entirely in the base model's native 4-bit quantization — no dequantization step

Training data

bitext/Bitext-customer-support-llm-chatbot-training-dataset (26,872 rows, license: CDLA-Sharing-1.0). Preprocessing:

• The dataset's {{Placeholder}} tags (order numbers, support email/phone, account names, etc. — 30 distinct tags) were substituted with a fixed set of concrete, fictional values so the model learns to produce realistic, fully-formed answers instead of literal {{Order Number}} tokens. Rows with any tag outside that fixed set were dropped.
• Deduplicated on (instruction, response) pairs.
• Wrapped into chat-format JSONL with a generic support-agent system prompt (see Usage below).
• Seeded shuffle (random.seed(42)), split 96/2/2 into train/valid/test (~25.8K / 537 / 537 examples).

Coverage caveat (read this before assuming full-dataset coverage): training ran for 2000 iterations at batch size 4 (~8,000 examples, ~31% of one epoch), then was resumed for another 2000 iterations. Because mlx-lm reseeds its data shuffle fresh on every process launch, and this project's config pins a fixed seed: 0, the resumed run reproduced the identical batch order as the first run rather than advancing to unseen examples — so in total this model saw roughly the same ~31%-of-epoch slice of the training set twice, not ~62% of the full dataset once. It has not been trained on the remaining ~69% of the training set at all. Categories/intents underrepresented in that first slice may be weaker than ones that were well represented in it.

Training procedure

Two training runs on the same adapter, back to back:

Run 1 — fresh start, 2000 iterations:

Test loss 0.658, test perplexity 1.930.

Run 2 — resumed from Run 1's adapter (--resume-adapter-file, optimizer momentum reset), 2000 more iterations:

Test loss 0.627, test perplexity 1.871.

Train and validation loss stayed close throughout both runs (no train/val divergence), so there's no sign of overfitting despite the repeated-data caveat above — though that's also partly because of it (more passes over less-diverse data rather than one pass over more-diverse data).

Example outputs

Prompts run through both the base model and this fine-tune, using the system prompt below.

Q: I need to cancel my order ORD-48213, how do I do that?

Base model: generic step-by-step with a placeholder website link ([Insert Website Link Here — *Please replace this with your actual website link*]) and a vague mention of a possible processing fee.

Fine-tuned: "I've decoded that you would like to cancel your order with the number ORD-48213. I'm here to assist you with that. To cancel your order, please follow these steps: 1. Log in to your account on our website. 2. Navigate to the 'Order History' section. 3. Locate the specific order with the number ORD-48213. 4. Click on the 'Cancel Order' button next to the order details. 5. Follow any additional prompts or instructions provided to confirm the cancellation..."

Q: What's your policy on refunds for a damaged item?

Base model: generic apology plus a placeholder email/phone ([email protected], 1-800-555-1212) invented on the spot.

Fine-tuned: "I'm on the same wavelength, your concern about a damaged item. Our company takes full responsibility for ensuring that your purchase arrives in perfect condition... To initiate a refund for a damaged item, please follow these steps: 1. Contact our customer support team as soon as possible... 2. Provide them with the necessary details... 3. Our team will review your case and guide you through the refund process..."

Across all six test categories tried (cancellation, refunds, shipping address changes, account deletion, invoice status, payment methods), the fine-tune consistently produces shorter, more confident, procedural answers in a single consistent voice, versus the base model's longer, hedged answers full of bracketed placeholders for a human to fill in later.

Usage

from mlx_lm import load, generate

model, tokenizer = load("asharib12/gemma-3-4b-bitext-support")

SYSTEM_PROMPT = (
    "You are a helpful, professional customer support assistant. Answer "
    "customer questions about orders, payments, refunds, shipping, and "
    "accounts clearly, courteously, and in line with standard policy."
)

messages = [
    {"role": "system", "content": SYSTEM_PROMPT},
    {"role": "user", "content": "How do I check the status of my invoice online?"},
]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True)
print(generate(model, tokenizer, prompt=prompt, verbose=True))

Using the system prompt above matters — without it the model falls back to a generic assistant persona rather than the trained support-agent voice.

Limitations and biases

• Only ~31% of the training dataset's unique examples were ever seen (see Coverage caveat above) — categories/intents outside that slice may perform noticeably worse than the examples shown here.
• Training data is synthetic and templated (Bitext's dataset is machine-generated), so the model occasionally produces slightly stiff or formulaic phrasing ("I'll make it happen!") inherited from that data.
• Order numbers, phone numbers, emails, and other specifics the model produces come from a fixed set of fictional placeholder values used during training substitution — it has no knowledge of any real company's actual policies, contact details, or live order data. Do not use its output as a source of real policy information.
• Inherits any limitations, biases, and safety considerations of the base Gemma 3 model.
• English only; not evaluated on other languages.

License

The base model is subject to Google's Gemma Terms of Use (license tag: gemma) — review those terms before using or redistributing this fine-tune. The training data is licensed under CDLA-Sharing-1.0; that license's share-alike terms concern redistributing the dataset, and their applicability to model weights trained on it is not entirely clear-cut — this is not legal advice, review both licenses yourself if you plan on wide redistribution.

Links

• Training code / tutorial: github.com/Asharib706/Finetuning
• Base model: mlx-community/gemma-3-4b-it-qat-4bit
• Training dataset: bitext/Bitext-customer-support-llm-chatbot-training-dataset