Downloads · 30 days
18
4% of all-time downloads
davidquicast/gemma-2b-it-peru-legal-es-V2
gemma-2b-it-peru-legal-es-V2 is a text generation model from davidquicast. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
<p align="center" <img src="https://cdn-uploads.huggingface.co/production/uploads/64461026e1fd8d65b27e6187/bUbvGpmsKC7YsJs4aBt7T.jpeg" alt="Model Illustration" width="300" </p
Downloads · 30 days
18
4% of all-time downloads
All-time downloads
453
Public
Parameters
2.5B
10 GB on disk
Likes
1
Public
Click a slice to open those files.
.safetensors5 GB · 100%
From the Hugging Face model README
The Gemma-2B-IT-Peru-Legal-ES model is a state-of-the-art language model fine-tuned specifically for legal text comprehension and generation tasks in Spanish, focusing on the legal context of the Peruvian Constitution. Leveraging advanced techniques such as Low-Rank Adaptation (LoRA) and Bits and Bytes Quantization (BNB), this model provides accurate and contextually relevant responses to legal queries, making it a valuable tool for legal professionals, researchers, and AI enthusiasts interested in the legal domain.
Gemma-2b-it-peru-legal-es-V2 is a contextual legal language model designed to provide personalized legal advice in Spanish based on Peruvian legal texts. Leveraging advanced techniques like LoRA, the model offers accurate and contextually relevant responses to legal queries, covering various aspects of Peruvian law and regulation. Whether it's understanding rights, navigating legal procedures, or interpreting statutes, gemma-2b-it-peru-legal-es-V2 empowers users with comprehensive and reliable legal guidance tailored to the Peruvian legal landscape.
Users should verify model outputs against current regulations and consult with professionals for critical applications. Awareness of the model's scope and limitations is crucial for effective use.
from transformers import AutoModelForCausalLM, AutoTokenizer
# Cargar el modelo y el tokenizador
model_name = "somosnlp/gemma-2b-it-peru-legal-es-V2"
model = AutoModelForCausalLM.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)
# Ejemplo de uso
input_text = "Una persona ha sido despedida de su trabajo injustamente y necesita entender cuáles son sus derechos laborales según la Constitución Política del Perú."
input_ids = tokenizer.encode(input_text, return_tensors="pt")
output = model.generate(input_ids, max_length=100, num_return_sequences=1, temperature=0.7)
print(tokenizer.decode(output[0], skip_special_tokens=True))
The Gemma-2B-IT-Peru-Legal-ES-V2 model was fine-tuned exclusively on the "Constitución Política del Perú dataset available through Hugging Face Datasets. This dataset serves as a rich source of questions and answers pertaining to the legal framework outlined in the Peruvian Constitution. The dataset encompasses a wide range of topics and provisions within the Peruvian Constitution, providing comprehensive coverage of constitutional principles, rights, and legal interpretations.
The training process was executed within a Python environment, utilizing essential libraries to facilitate various tasks:
QLoRA (Quantization LoRA) was employed to optimize the model's computational efficiency and memory footprint while preserving its accuracy. Two configurations were utilized:
BitsAndBytesConfig: This configuration enabled the model to load in 4-bit quantization, leveraging the nf4 quantization type with a torch.bfloat16 compute data type for enhanced efficiency during inference.
bnb_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_quant_type="nf4",
bnb_4bit_compute_dtype=torch.bfloat16,
bnb_4bit_use_double_quant=False,
)
LoRAConfig: This configuration applied Low-Rank Adaptation (LoRA) to the model, optimizing its parameters for the specific task of language modeling in the legal domain. Key parameters include:
config = LoraConfig(
r=8,
lora_alpha=16,
target_modules=['q_proj', 'k_proj', 'v_proj', 'o_proj'],
bias="none",
lora_dropout=0.025,
task_type="CAUSAL_LM",
)
These configurations were crucial for optimizing the model's performance and resource utilization during training and inference, ensuring efficient deployment.
After fine-tuning, the LoRA-adjusted weights were merged back with the base Gemma model to create the final gemma-2b-it-peru-legal-es-V2. The model was then saved and made available through Hugging Face for easy access and further development.
During the training process, Wandb (Weights & Biases) was used for comprehensive logging and visualization of key metrics. Wandb's powerful tracking capabilities enabled real-time monitoring of training progress, evaluation metrics, and model performance. Through interactive dashboards and visualizations, Wandb facilitated deep insights into the training dynamics, allowing for efficient model optimization and debugging. This logging integration with Wandb enhances transparency, reproducibility, and collaboration among researchers and practitioners.
The training of gemma-2b-it-peru-legal-es-V2 was conducted optimizing the computational expenditure required.
This project is distributed under the Apache 2.0 license.