Downloads · 30 days
20
27% of all-time downloads
WaseemGh98/waseem-bankassist-qwen3-lora
waseem-bankassist-qwen3-lora is a text generation model from WaseemGh98. Use it when you need the model to write or continue text. It is set up for peft. The card lists the license as apache-2.0.
This is a LoRA adapter fine-tuned from Qwen/Qwen3-0.6B for banking intent classification using the Banking77 dataset.
Downloads · 30 days
20
27% of all-time downloads
All-time downloads
74
Public
Repo size
20.6 MB
Likes
0
Public
Click a slice to open those files.
.json11.4 MB · 55%
From the Hugging Face model README
This is a LoRA adapter fine-tuned from Qwen/Qwen3-0.6B for banking intent classification using the Banking77 dataset.
The model takes a customer banking message and predicts one of 77 banking intent labels.
Customer:
"My card still hasn't arrived."
Prediction:
card_arrival
Qwen/Qwen3-0.6BThis adapter can be loaded on top of Qwen/Qwen3-0.6B to classify customer banking messages into one of the 77 intent categories defined by Banking77.
Example intents include:
card_arrivalcard_not_workingpending_transfercash_withdrawalgetting_virtual_cardThe adapter can be used for experimentation with banking customer-support intent classification, model evaluation, LoRA fine-tuning workflows, or as a starting point for further research and training.
This model is an educational fine-tuning project and is not intended to be used as a production banking system.
It should not be used for:
The model was trained on the Banking77 dataset and is therefore limited to the language patterns and 77 intent categories represented in that dataset.
The fine-tuned model achieved 47.79% exact-match accuracy on the full Banking77 test split, meaning it still makes a significant number of incorrect classifications.
The model may also occasionally generate a label that is not one of the valid Banking77 intents.
Because the model generates intent names using a causal language model rather than a fixed classification head, its output is not strictly constrained to the 77 available labels.
This model should be treated as an experimental fine-tuned model rather than a production-ready banking classifier.
Predictions should be validated before being used in any downstream system.
Install the required libraries:
pip install torch transformers peft
Then load the Qwen base model and the BankAssist LoRA adapter:
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
BASE_MODEL = "Qwen/Qwen3-0.6B"
ADAPTER = "WaseemGh98/waseem-bankassist-qwen3-lora"
tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL)
base_model = AutoModelForCausalLM.from_pretrained(
BASE_MODEL,
torch_dtype="auto",
device_map="auto"
)
model = PeftModel.from_pretrained(
base_model,
ADAPTER
)
model.eval()
The resulting model is:
Qwen3-0.6B
+
BankAssist LoRA adapter
↓
Fine-tuned BankAssist model
The adapter was fine-tuned on the Banking77 dataset available through Hugging Face Datasets.
Dataset:
mteb/banking77
The dataset contains:
Each training example contains a customer banking message and its corresponding intent label.
Example:
Customer message:
"I am still waiting for my card."
Intent:
card_arrival
Each Banking77 example was converted into Qwen's conversational chat format.
For example:
System:
Classify the customer's banking message into the correct banking intent.
Return only the intent name.
User:
I am still waiting for my card.
Assistant:
card_arrival
The conversation was formatted using Qwen's chat template and tokenized using the tokenizer provided with Qwen/Qwen3-0.6B.
Training used supervised fine-tuning with LoRA.
Instead of updating all parameters in Qwen3-0.6B, the original model weights remained frozen while small LoRA adapter matrices were trained.
LoRA was applied to the attention projection layers:
q_projk_projv_projo_projThe training loop was handled by TRL's SFTTrainer, including:
Forward pass
↓
Loss calculation
↓
Backpropagation
↓
Gradient calculation
↓
Optimizer update
↓
LoRA parameters updated
Training was completed locally on a laptop GPU.
Techniques such as LoRA, FP16, gradient checkpointing, batch size 1, and gradient accumulation were used to make fine-tuning possible within the available GPU memory.
The model was evaluated on the complete Banking77 test split:
The original Qwen/Qwen3-0.6B model and the fine-tuned Qwen + BankAssist LoRA model were evaluated on the exact same test examples using the same prompt and generation settings.
The evaluation measured:
The primary metric was exact-match classification accuracy.
The evaluation also counted invalid predictions where the model generated an intent name that was not part of Banking77's valid label set.
| Model | Accuracy | Invalid Predictions |
|---|---|---|
| Qwen3-0.6B | 22.07% | 335 |
| Qwen3-0.6B + BankAssist LoRA | 47.79% | 160 |
Fine-tuning increased classification accuracy from:
22.07% → 47.79%
This represents an improvement of:
+25.72 percentage points
Invalid predictions also decreased:
335 → 160
The experiment shows that LoRA fine-tuning substantially improved Qwen3-0.6B's performance on Banking77 while training only a small adapter instead of the entire model.
The resulting 47.79% accuracy still leaves significant room for improvement and should not be interpreted as production-level performance.
The model was examined by comparing the original Qwen3-0.6B model with the LoRA fine-tuned version on the complete Banking77 test set.
An initial manual evaluation was also performed on individual customer messages to verify that the adapter was loading correctly and producing Banking77-style intent labels.
Training was performed locally rather than through a cloud GPU provider.
No estimate of carbon emissions is provided because power-consumption data sufficient for an accurate calculation was not collected.
The base model is Qwen/Qwen3-0.6B, a causal language model.
Instead of adding a traditional classification head, this project treats intent classification as a text-generation task.
The model receives:
Customer banking message
and generates:
intent_name
LoRA provides a parameter-efficient way to adapt the pretrained Qwen model to this task without updating all of its original parameters.
Training and evaluation were performed locally.
VRAM usage was reduced using:
No standalone research paper is associated with this adapter.
When using this model, users should reference the original Qwen3 model and Banking77 dataset where appropriate.
This adapter was created as part of the BankAssist project to explore the complete LLM fine-tuning workflow:
Hugging Face dataset
↓
Data preparation
↓
Qwen chat formatting
↓
Tokenization
↓
LoRA fine-tuning
↓
Local GPU training
↓
Evaluation against base model
↓
Publishing adapter to Hugging Face
Waseem Ghanem
Hugging Face: WaseemGh98