Downloads · 30 days
10
17% of all-time downloads
advaitshewale/onyx-legal-70b-adapter
onyx-legal-70b-adapter is a text generation model from advaitshewale. Use it when you need the model to write or continue text. It is set up for peft. The card lists the license as llama3.3.
Onyx is a fine-tuned LoRA adapter for Llama 3.3 70B Instruct, trained to analyze legal documents, court records, medical bills, insurance documents, and contracts with precision and clarity.
Downloads · 30 days
10
17% of all-time downloads
All-time downloads
60
Public
Repo size
5.5 GB
Likes
0
Public
Click a slice to open those files.
.safetensors5.4 GB · 100%
From the Hugging Face model README
Onyx is a fine-tuned LoRA adapter for Llama 3.3 70B Instruct, trained to analyze legal documents, court records, medical bills, insurance documents, and contracts with precision and clarity.
Built by Advait Shewale @Advait's Tech Hub.
Paste in any legal document and Onyx will:
| Category | Examples |
|---|---|
| 🏛️ Court Records | SCOTUS opinions, federal court decisions, case holdings |
| 📄 Contracts | Employment, NDA, license, merger, supply, indemnification agreements |
| 🏥 Medical Bills | CPT/ICD-10 coded bills, EOBs, dispute letters |
| 🛡️ Insurance | Policy analysis, denial letters, coverage disputes, appeals |
| 🌍 Human Rights | ECtHR cases, ECHR Article analysis |
| 🇪🇺 EU Regulations | EurLex directives, regulatory compliance |
| 🏛️ Legislation | US Congressional bills, federal regulations |
| 🔒 Privacy / ToS | GDPR/CCPA compliance, terms of service red flags |
| Property | Value |
|---|---|
| Base Model | meta-llama/Llama-3.3-70B-Instruct |
| Adapter Type | LoRA (PEFT) |
| LoRA Rank | r=64, alpha=128 |
| Target Modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Quantization (Training) | 4-bit NF4 (bitsandbytes) |
| Training Hardware | NVIDIA GH200 96GB HBM3e |
| Max Sequence Length | 4,096 tokens |
| Eval Loss | 0.7236 |
| Accuracy | 83.2% |
| Training Epochs | 2 (Phase 2) |
| Training Steps | 260 |
| Optimizer | paged_adamw_8bit |
Onyx was trained in two phases:
| Source | Type | Count |
|---|---|---|
| LexGLUE SCOTUS | U.S. Supreme Court opinions | 1,800 |
| LexGLUE LEDGAR | Real SEC-filed contract clauses | 1,500 |
| LexGLUE ECtHR | European Court of Human Rights cases | 500 |
| LexGLUE EurLex | EU regulatory documents | 300 |
| InsuranceQA | Insurance questions & answers | 300 |
| CMS Medical Billing | Real CPT/ICD-10 dispute scenarios | 200 |
pip install transformers peft bitsandbytes accelerate torch
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from transformers import BitsAndBytesConfig
from peft import PeftModel
BASE_MODEL = "meta-llama/Llama-3.3-70B-Instruct"
ADAPTER_PATH = "advaitshewale/onyx-legal-70b-adapter"
SYSTEM = (
"You are Onyx, an expert AI legal analyst. You analyze legal documents, "
"court records, medical bills, insurance documents, and contracts with "
"precision and clarity. Identify key clauses, obligations, risks, "
"relevant statutes, and provide actionable insights. If you do not recognize"
"something in the analyzed text, simply reply I do not know. Do not make things up"
)
# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL)
tokenizer.pad_token = tokenizer.eos_token
# Load base model in 4-bit
bnb_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_quant_type="nf4",
bnb_4bit_compute_dtype=torch.bfloat16,
bnb_4bit_use_double_quant=True,
)
model = AutoModelForCausalLM.from_pretrained(
BASE_MODEL,
quantization_config=bnb_config,
device_map="auto",
torch_dtype=torch.bfloat16,
)
# Load Onyx adapter
model = PeftModel.from_pretrained(model, ADAPTER_PATH)
model.eval()
# Run inference
messages = [
{"role": "system", "content": SYSTEM},
{"role": "user", "content": "Analyze this contract clause:\n\nThe party hereby waives all rights to appeal under Section 14(b), provided that the indemnification clause in Exhibit A remains in full force."}
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
with torch.no_grad():
output = model.generate(
**inputs,
max_new_tokens=1024,
temperature=0.2,
top_p=0.9,
do_sample=True,
repetition_penalty=1.1,
)
response = tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
print(response)
| Setup | VRAM | Notes |
|---|---|---|
| 4-bit NF4 (recommended) | 40GB+ | A100 40GB, 2x A40, GH200 |
| 4-bit NF4 + CPU offload | 24GB+ GPU | Slower, uses system RAM |
| AWQ base model | 40GB+ | Use hugging-quants/Meta-Llama-3.3-70B-Instruct-AWQ-INT4 |
model:
name: meta-llama/Llama-3.3-70B-Instruct
max_seq_length: 4096
quantization:
load_in_4bit: true
bnb_4bit_quant_type: nf4
bnb_4bit_compute_dtype: bfloat16
bnb_4bit_use_double_quant: true
lora:
r: 64
lora_alpha: 128
target_modules: [q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj]
training:
num_train_epochs: 2
per_device_train_batch_size: 2
gradient_accumulation_steps: 16
learning_rate: 2.0e-5
gradient_checkpointing: true
optim: paged_adamw_8bit
This adapter inherits the Llama 3.3 Community License. For commercial use, review Meta's license terms.
Built with ❤️ by Advait Shewale -- Advait'sTechHub