microsoft

bloom-deepspeed-inference-fp16

microsoft/bloom-deepspeed-inference-fp16

bloom-deepspeed-inference-fp16 is a feature extraction model from microsoft. Use it when you need embeddings to search or compare text. It is set up for transformers. The card lists the license as bigscience-bloom-rail-1.0.

This is a copy of the original BLOOM weights that is more efficient to use with the DeepSpeed-MII and DeepSpeed-Inference. In this repo the original tensors are split into 8 shards to target 8 GPUs, this allows the us…