Downloads · 30 days
7
3% of all-time downloads
llama-2-7b-sampling-watermark-distill-kth-shift2
cygu/llama-2-7b-sampling-watermark-distill-kth-shift2
llama-2-7b-sampling-watermark-distill-kth-shift2 is a text generation model from cygu. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as llama2.
Sampling-based watermark distilled Llama 2 7B using the KTH \\(s=2\\) watermarking strategy in the paper On the Learnability of Watermarks for Language Models.
All-time downloads
236
Public
Repo size
27 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin27 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- llama2
- Model type
- llama
- Access
- Public
- Created
- Dec 31, 2023
- Updated
- May 1, 2024
- SHA
- 82aea0a4
Try a prompt
Model card
From the Hugging Face model README
Model description
Sampling-based watermark distilled Llama 2 7B using the KTH \(s=2\) watermarking strategy in the paper On the Learnability of Watermarks for Language Models.
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- distributed_type: multi-GPU
- num_devices: 4
- gradient_accumulation_steps: 4
- total_train_batch_size: 128
- total_eval_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 500
- num_epochs: 1.0
Framework versions
- Transformers 4.29.2
- Pytorch 2.0.1+cu117
- Datasets 2.13.1
- Tokenizers 0.13.3
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- llama2
- Created
- Dec 31, 2023
- Updated
- May 1, 2024