Downloads · 30 days
0
absa_10_domains_large_lora
SilvioLima/absa_10_domains_large_lora
absa_10_domains_large_lora is a machine learning model from SilvioLima. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Access
Public
Updated Jun 26, 2024
Repo size
734 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors151 MB · 98%
At a glance
- License
- apache-2.0
- Access
- Public
- Created
- Jun 24, 2024
- Updated
- Jun 26, 2024
- SHA
- ee3c7d89
Datasets
Model card
From the Hugging Face model README
Load the tokenizer, model, and data collator
MODEL_NAME = "google/flan-t5-large"
model = AutoModelForSeq2SeqLM.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)
data_collator = DataCollatorWithPadding(tokenizer=tokenizer) # do padding automatic according size of input_str
Configuração do LoRA
lora_config = LoraConfig(
task_type=TaskType.SEQ_2_SEQ_LM,
r=16, # Exemplo de quantização para 8 bits
lora_alpha=32,
bias='none',
target_modules=['q', 'v'], # Exemplo de módulos alvo para quantização
lora_dropout=0.01,
modules_to_save=['lm_head'],
)
Aplicar LoRA ao modelo base
model = get_peft_model(model, lora_config)
Configuração inicial do treinamento
num_train = 10500
num_valid = 1300
num_epochs = 10
max_length = 256
batch_size = 2
- License
- apache-2.0
- Languages
- en
- Created
- Jun 24, 2024
- Updated
- Jun 26, 2024