Downloads · 30 days
8
3% of all-time downloads
llama-2-7b-sampling-watermark-distill-aar-k2
cygu/llama-2-7b-sampling-watermark-distill-aar-k2
llama-2-7b-sampling-watermark-distill-aar-k2 is a text generation model from cygu. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as llama2.
Sampling-based watermark distilled Llama 2 7B using the Aar \\(k=2\\) watermarking strategy in the paper On the Learnability of Watermarks for Language Models.
All-time downloads
237
Public
Repo size
27 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin27 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- llama2
- Model type
- llama
- Access
- Public
- Created
- Dec 31, 2023
- Updated
- May 1, 2024
- SHA
- cb4c5575
Try a prompt
Model card
From the Hugging Face model README
Model description
Sampling-based watermark distilled Llama 2 7B using the Aar \(k=2\) watermarking strategy in the paper On the Learnability of Watermarks for Language Models.
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- distributed_type: multi-GPU
- num_devices: 4
- gradient_accumulation_steps: 4
- total_train_batch_size: 128
- total_eval_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 500
- num_epochs: 1.0
Framework versions
- Transformers 4.29.2
- Pytorch 2.0.1+cu117
- Datasets 2.13.1
- Tokenizers 0.13.3
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- llama2
- Created
- Dec 31, 2023
- Updated
- May 1, 2024