TinyLlama-ToS-Finetuned

TinyLlama-ToS-Finetuned

A LoRA-finetuned version of TinyLlama-1.1B-Chat-v1.0 for detecting unfair / anomalous Terms of Service clauses. The model classifies clauses as Fair or Unfair based on anomalous patterns in legal text.

---

Model Details

Model Description

• Developed by: Noshitha Padma Pratyusha Juttu (UMass Amherst, MS CS 2024–25)
• Model type: Causal LM + LoRA adapters for classification
• Base model: TinyLlama-1.1B-Chat v1.0
• Total parameters (base + LoRA): ~1.101B
• LoRA trainable parameters: ~1.13M (≈0.1% of base model)
• Language(s): English
• License: Apache-2.0 (same as base model)

This model was finetuned with LoRA adapters. During training, only ~1.13M parameters were updated, while the 1.1B base model parameters remained frozen. The final uploaded model contains both the base weights and the adapter weights.

📚 Citation

If you use this model in your research or work, please cite the following paper:

Juttu, Noshitha Padma Pratyusha. Text to Trust: Evaluating Fine-Tuning and LoRA Trade-Offs in Language Models for Unfair Terms of Service Detection. arXiv preprint arXiv:2510.22531, 2025.
https://arxiv.org/abs/2510.22531

Model Sources

• Repository: GitHub – UnfairTOSAgreementsDetection

---

Uses

Direct Use

• Clause-level classification of Terms of Service agreements.
• Detects if a clause is likely “Unfair” or “Fair”.

Downstream Use

• Legal NLP research and experiments.
• Integrating into compliance assistants for contract review.

Out-of-Scope Use

• Not a substitute for professional legal advice.
• Not guaranteed to generalize beyond English contracts.

---

Bias, Risks, and Limitations

• Limited to Claudette ToS dataset → may not represent all legal documents.
• May produce false positives/negatives, especially on borderline clauses.
• Outputs can be sensitive to prompt phrasing.

Recommendations

Use this model as assistive tool, not for automated legal decision-making.

---

How to Get Started with the Model

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

base = "TinyLlama/TinyLlama-1.1B-Chat-v1.0"
adapter = "Noshitha98/TinyLlama-ToS-Finetuned"

tokenizer = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base)
model = PeftModel.from_pretrained(model, adapter)

prompt = "<s>[CLAUSE]: You agree that we may suspend your account at any time. \n[Is this anomalous?]:"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=5)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))