Downloads · 30 days
10
16% of all-time downloads
himu1780/ainbondhu-legal-lora
ainbondhu-legal-lora is a machine learning model from himu1780. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft.
AinBondhu is a fine-tuned LoRA adapter based on Qwen/Qwen2.5-7B-Instruct, specifically trained to act as a legal assistant for Bangladeshi law. It understands both Bengali and English, providing accurate guidance on v…
Downloads · 30 days
10
16% of all-time downloads
All-time downloads
62
Public
Repo size
657 MB
Likes
0
Public
Click a slice to open those files.
.safetensors646 MB · 98%
From the Hugging Face model README
AinBondhu is a fine-tuned LoRA adapter based on Qwen/Qwen2.5-7B-Instruct, specifically trained to act as a legal assistant for Bangladeshi law. It understands both Bengali and English, providing accurate guidance on various legal scenarios, laws, and constitutional matters in Bangladesh.
To seamlessly use this model, load the base model (Qwen/Qwen2.5-7B-Instruct) and then apply these LoRA adapters using peft and transformers.
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
from peft import PeftModel
# Define models
LORA_MODEL = "himu1780/ainbondhu-legal-lora"
BASE_MODEL = "Qwen/Qwen2.5-7B-Instruct"
# 4-bit Quantization Config (Great for free Colab T4 GPUs)
bnb_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_quant_type="nf4",
bnb_4bit_use_double_quant=True,
bnb_4bit_compute_dtype=torch.float16,
)
# Load Tokenizer & Base Model
tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL)
base_model = AutoModelForCausalLM.from_pretrained(
BASE_MODEL,
quantization_config=bnb_config,
device_map="auto",
)
# Apply AinBondhu LoRA Adapters
model = PeftModel.from_pretrained(base_model, LORA_MODEL)
model.eval()
# Chat with AinBondhu
prompt = "বাংলাদেশে সাইবার বুলিং এর শাস্তি কি? বিস্তারিত বল।"
messages = [
{"role": "system", "content": "You are AinBondhu, a helpful legal AI assistant for Bangladesh."},
{"role": "user", "content": prompt},
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
with torch.no_grad():
out = model.generate(
**inputs,
max_new_tokens=512,
temperature=0.2, # Keep low for factual accuracy
top_p=0.9,
pad_token_id=tokenizer.eos_token_id,
)
print("AinBondhu says:
" + tokenizer.decode(out[0][inputs['input_ids'].shape[1]:], skip_special_tokens=True))
unsloth/qwen2.5-7b-instruct-bnb-4bit (Qwen 2.5 7B Instruct)unsloth and trl.This AI model is designed for educational and informational purposes only. It does not provide professional legal advice. Always consult a qualified lawyer or legal professional in Bangladesh for actual legal proceedings or advice.