Downloads · 30 days
15
5% of all-time downloads
opt-30b-deepspeed-inference-fp16-shard-4
lucadiliello/opt-30b-deepspeed-inference-fp16-shard-4
opt-30b-deepspeed-inference-fp16-shard-4 is a text generation model from lucadiliello. Use it when you need the model to write or continue text. It is set up for transformers.
This is a copy of the original OPT weights that is more efficient to use with the DeepSpeed-MII and DeepSpeed-Inference. In this repo the original tensors are split into 4 shards to target 4 GPUs, this allows the user…
All-time downloads
301
Public
Repo size
65.1 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pt65.1 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- Model type
- opt
- Access
- Public
- Created
- Mar 21, 2023
- Updated
- Mar 22, 2023
- SHA
- 3349362a
Try a prompt
Model card
From the Hugging Face model README
This is a copy of the original OPT weights that is more efficient to use with the DeepSpeed-MII and DeepSpeed-Inference. In this repo the original tensors are split into 4 shards to target 4 GPUs, this allows the user to run the model with DeepSpeed-inference Tensor Parallelism.
For specific details about the OPT model itself, please see the original OPT model card.
For examples on using this repo please see the following:
- Task
- Text Generation
- Library
- transformers
- Type
- opt
- Created
- Mar 21, 2023
- Updated
- Mar 22, 2023