Downloads · 30 days
15
5% of all-time downloads
LLama-2-70b-chat-hf-Orca100k
iamplus/LLama-2-70b-chat-hf-Orca100k
LLama-2-70b-chat-hf-Orca100k is a text generation model from iamplus. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
Base model : meta-llama/Llama-2-70b-chat-hf
All-time downloads
274
Public
Parameters
69B
138 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors138 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- mit
- Model type
- llama
- Access
- Public
- Created
- Aug 17, 2023
- Updated
- Aug 23, 2023
- SHA
- d118b42a
Try a prompt
Datasets
Model card
From the Hugging Face model README
Base model : meta-llama/Llama-2-70b-chat-hf
Data : 100k from 1M Gpt-4 Orca data (Dolphin)
Training Params :
batch_size_training: '16'
checkpoint_type: StateDictType.FULL_STATE_DICT
dataset: orca_dolphin_100k_gpt4
dist_checkpoint_folder: fine-tuned
dist_checkpoint_root_folder: model_checkpoints
enable_fsdp: 'True'
freeze_layers: 'False'
fsdp_activation_checkpointing: 'True'
gamma: '0.85'
low_cpu_fsdp: 'True'
lr: 1e-05
micro_batch_size: '16'
mixed_precision: 'True'
model_name: meta-llama/Llama-2-70b-chat-hf
num_epochs: '1'
num_freeze_layers: '1'
num_workers_dataloader: '1'
one_gpu: 'False'
optimizer: anyprecision
output_dir: ~/llama-recipes-70b/output
peft_method: lora
pure_bf16: 'True'
quantization: 'False'
run_validation: 'True'
save_model: 'True'
save_optimizer: 'True'
seed: '42'
sharding_strategy: ShardingStrategy.FULL_SHARD
use_fast_kernels: (False,)
use_fp16: 'False'
use_peft: 'False'
val_batch_size: '16'
weight_decay: '0.0'
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- mit
- Created
- Aug 17, 2023
- Updated
- Aug 23, 2023