qwen3-1.7b-banking77-intent

Qwen3-1.7B — Banking Intent Classification (LoRA fine-tune)

A LoRA fine-tune of unsloth/qwen3-1.7b-unsloth-bnb-4bit for 77-class intent classification on customer banking queries, trained on the banking77 dataset.

Model description

This model takes a free-text customer banking query and classifies it into one of 77 fine-grained intents (e.g. card_arrival, transfer_not_received_by_recipient, declined_transfer). It was fine-tuned using QLoRA (4-bit base + LoRA adapter) with Unsloth, keeping the entire training run within a free Google Colab T4 GPU.

• Developed by: nikhilwalia23
• Base model: unsloth/qwen3-1.7b-unsloth-bnb-4bit
• Language: English
• License: apache-2.0
• Fine-tuning method: LoRA (r=16, alpha=16) via Unsloth + TRL's SFTTrainer
• Task: Text classification (intent detection), formatted as instruction-following generation

Intended use

Classifying customer support queries in a banking/fintech context into a fixed set of 77 intent categories — e.g. for automated ticket routing or intent-based FAQ retrieval. This is a portfolio/learning project demonstrating an end-to-end fine-tuning pipeline; it has not been validated for production use.

Out of scope: general-purpose chat, domains outside retail banking support queries, languages other than English.

Training data

banking77 — 13,083 real customer service queries from an online banking platform, labeled across 77 fine-grained intents. Trained on a 4,000-example subset of the 9,993-row train split; evaluated on a 300-example sample from the 3,076-row test split.

Training procedure

• Method: QLoRA — base model loaded in 4-bit (bitsandbytes), LoRA adapters trained on top
• LoRA config: r=16, alpha=16, dropout=0, target modules: q/k/v/o_proj, gate/up/down_proj
• Framework: Unsloth + Hugging Face TRL (SFTTrainer)
• Epochs: 1
• Batch size: 8 (effective 16 with gradient accumulation ×2)
• Learning rate: 2e-4, linear schedule
• Optimizer: adamw_8bit
• Hardware: 1× NVIDIA T4 (free Google Colab)
• Sequence length: 1024

Each training example was formatted as a chat turn via tokenizer.apply_chat_template, with the query as the user message and the ground-truth intent string as the assistant response. The model was not shown the list of 77 valid intents in the prompt — it learned the label space directly from training examples.

Evaluation

Accuracy measured on a held-out 300-example sample from the test split, exact-match against the gold intent string:

Note on the comparison: part of the gap reflects the base model not knowing the exact intent-string format to output, in addition to task accuracy — fine-tuning here teaches both the classification task and the expected output format. banking77 is also noted to have some near-duplicate intents (e.g. multiple card-related categories), which caps achievable accuracy for any model at this scale.

How to use

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = "nikhilwalia23/qwen3-1.7b-banking77-intent",
    max_seq_length = 1024,
    load_in_4bit = True,
)
FastLanguageModel.for_inference(model)

messages = [{"role": "user", "content": "My card hasn't arrived yet, it's been two weeks"}]
inputs = tokenizer.apply_chat_template(
    messages, tokenize=True, add_generation_prompt=True, return_tensors="pt"
).to("cuda")
outputs = model.generate(input_ids=inputs, max_new_tokens=15)
print(tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True))

Limitations

• Trained on a 4,000-example subset (not the full 9,993 rows) to keep training fast on free hardware — accuracy would likely improve with the full training set and more epochs.
• English-only, retail-banking domain only.
• As a LoRA adapter, requires the base model (unsloth/qwen3-1.7b-unsloth-bnb-4bit) to be loaded alongside it.
• Not evaluated for robustness to adversarial or out-of-distribution queries.

Training framework

This qwen3 model was trained 2x faster with Unsloth and Hugging Face's TRL library.

<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>