Downloads · 30 days
13
4% of all-time downloads
Mistral-7B-v0.1-hitl
KasparZ/Mistral-7B-v0.1-hitl
Mistral-7B-v0.1-hitl is a text generation model from KasparZ. Use it when you need the model to write or continue text. It is set up for transformers.
basemodel = "mistralai/Mistral-7B-v0.1" datasetname = "KasparZ/HITL-2"
All-time downloads
323
Public
Parameters
7.2B
22.5 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors14.9 GB · 66%
At a glance
- Task
- Text Generation
- Library
- transformers
- Model type
- mistral
- Access
- Public
- Created
- Mar 26, 2024
- Updated
- Mar 30, 2024
- SHA
- 1d6464a7
Try a prompt
Model card
From the Hugging Face model README
base_model = "mistralai/Mistral-7B-v0.1" dataset_name = "KasparZ/HITL-2"
load_in_4bit= True, bnb_4bit_quant_type= "nf4", bnb_4bit_compute_dtype= torch.bfloat16, bnb_4bit_use_double_quant= False,
device_map="auto", trust_remote_code=True,
lora_alpha=16, lora_dropout=0.1, r=64, bias="none", task_type="CAUSAL_LM", target_modules=["q_proj", "k_proj", "v_proj", "o_proj","gate_proj"]
model.gradient_checkpointing_enable()
tokenizer.padding_side = 'right' tokenizer.add_eos_token = True
num_train_epochs=4,
per_device_train_batch_size=4,
gradient_accumulation_steps=1,
optim="paged_adamw_32bit",
save_steps=50,
logging_steps=1,
learning_rate=2e-4,
weight_decay=0.001,
fp16=False,
bf16=False,
max_grad_norm=0.3,
max_steps=-1,
warmup_ratio=0.03,
group_by_length=True,
lr_scheduler_type="constant",
- Task
- Text Generation
- Library
- transformers
- Type
- mistral
- Created
- Mar 26, 2024
- Updated
- Mar 30, 2024