Downloads · 30 days
11
17% of all-time downloads
LLama-2-70b-hf-Orca100k
iamplus/LLama-2-70b-hf-Orca100k
LLama-2-70b-hf-Orca100k is a text generation model from iamplus. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
All-time downloads
66
Public
Repo size
138 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin138 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- mit
- Model type
- llama
- Access
- Public
- Created
- Aug 14, 2023
- Updated
- Aug 23, 2023
- SHA
- 5b8b608a
Try a prompt
Datasets
Model card
From the Hugging Face model README
Base model : meta-llama/Llama-2-70b-hf
Data : 100k from 1M Gpt-4 Orca data (Dolphin)
Training Params :
batch_size_training: '16'
checkpoint_type: StateDictType.FULL_STATE_DICT
dataset: orca_dolphin_100k_gpt4
dist_checkpoint_folder: fine-tuned
dist_checkpoint_root_folder: model_checkpoints
enable_fsdp: 'True'
freeze_layers: 'False'
fsdp_activation_checkpointing: 'True'
gamma: '0.85'
low_cpu_fsdp: 'True'
lr: 1e-05
micro_batch_size: '16'
mixed_precision: 'True'
model_name: meta-llama/Llama-2-70b-hf
num_epochs: '1'
num_freeze_layers: '1'
num_workers_dataloader: '1'
one_gpu: 'False'
optimizer: anyprecision
output_dir: ~/llama-recipes-70b/output
peft_method: lora
pure_bf16: 'True'
quantization: 'False'
run_validation: 'True'
save_model: 'True'
save_optimizer: 'True'
seed: '42'
sharding_strategy: ShardingStrategy.FULL_SHARD
use_fast_kernels: (False,)
use_fp16: 'False'
use_peft: 'False'
val_batch_size: '16'
weight_decay: '0.0'
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- mit
- Created
- Aug 14, 2023
- Updated
- Aug 23, 2023