Downloads · 30 days
10
5% of all-time downloads
llama-2-7b-sampling-watermark-distill-kth-shift1
cygu/llama-2-7b-sampling-watermark-distill-kth-shift1
llama-2-7b-sampling-watermark-distill-kth-shift1 is a text generation model from cygu. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as llama2.
Sampling-based watermark distilled Llama 2 7B using the KTH \\(s=1\\) watermarking strategy in the paper On the Learnability of Watermarks for Language Models.
All-time downloads
219
Public
Repo size
27 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin27 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- llama2
- Model type
- llama
- Access
- Public
- Created
- Dec 31, 2023
- Updated
- May 1, 2024
- SHA
- 4c70c711
Try a prompt
Model card
From the Hugging Face model README
Model description
Sampling-based watermark distilled Llama 2 7B using the KTH \(s=1\) watermarking strategy in the paper On the Learnability of Watermarks for Language Models.
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- distributed_type: multi-GPU
- num_devices: 4
- gradient_accumulation_steps: 4
- total_train_batch_size: 128
- total_eval_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 500
- num_epochs: 1.0
Framework versions
- Transformers 4.29.2
- Pytorch 2.0.1+cu117
- Datasets 2.13.1
- Tokenizers 0.13.3
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- llama2
- Created
- Dec 31, 2023
- Updated
- May 1, 2024