Downloads · 30 days
12
20% of all-time downloads
LLama-2-70b-chat-hf-Orca100k
manojpreveen/LLama-2-70b-chat-hf-Orca100k
LLama-2-70b-chat-hf-Orca100k is a text generation model from manojpreveen. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
Base model : meta-llama/Llama-2-70b-chat-hf
All-time downloads
61
Public
Parameters
69B
138 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors138 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- mit
- Model type
- llama
- Access
- Public
- Created
- Feb 29, 2024
- Updated
- Feb 29, 2024
- SHA
- baa0415f
Try a prompt
Datasets
Model card
From the Hugging Face model README
Base model : meta-llama/Llama-2-70b-chat-hf
Data : 100k from 1M Gpt-4 Orca data (Dolphin)
Training Params :
batch_size_training: '16'
checkpoint_type: StateDictType.FULL_STATE_DICT
dataset: orca_dolphin_100k_gpt4
dist_checkpoint_folder: fine-tuned
dist_checkpoint_root_folder: model_checkpoints
enable_fsdp: 'True'
freeze_layers: 'False'
fsdp_activation_checkpointing: 'True'
gamma: '0.85'
low_cpu_fsdp: 'True'
lr: 1e-05
micro_batch_size: '16'
mixed_precision: 'True'
model_name: meta-llama/Llama-2-70b-chat-hf
num_epochs: '1'
num_freeze_layers: '1'
num_workers_dataloader: '1'
one_gpu: 'False'
optimizer: anyprecision
output_dir: ~/llama-recipes-70b/output
peft_method: lora
pure_bf16: 'True'
quantization: 'False'
run_validation: 'True'
save_model: 'True'
save_optimizer: 'True'
seed: '42'
sharding_strategy: ShardingStrategy.FULL_SHARD
use_fast_kernels: (False,)
use_fp16: 'False'
use_peft: 'False'
val_batch_size: '16'
weight_decay: '0.0'
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- mit
- Created
- Feb 29, 2024
- Updated
- Feb 29, 2024