lucadiliello

opt-30b-deepspeed-inference-fp16-shard-8

lucadiliello/opt-30b-deepspeed-inference-fp16-shard-8

opt-30b-deepspeed-inference-fp16-shard-8 is a text generation model from lucadiliello. Use it when you need the model to write or continue text. It is set up for transformers.

This is a copy of the original OPT weights that is more efficient to use with the DeepSpeed-MII and DeepSpeed-Inference. In this repo the original tensors are split into 8 shards to target 8 GPUs, this allows the user…