Downloads · 30 days
20
2% of all-time downloads
pszemraj/long-t5-tglobal-xl-16384-book-summary-8bit
long-t5-tglobal-xl-16384-book-summary-8bit is a summarization model from pszemraj. Use it when you need a shorter version of a longer text. It is set up for transformers. The card lists the license as apache-2.0.
<a href="https://colab.research.google.com/gist/pszemraj/c19e32baf876deb866c31cd46c86e893/long-t5-xl-accelerate-test.ipynb" <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/ </a
Downloads · 30 days
20
2% of all-time downloads
All-time downloads
1.1K
Public
Parameters
3B
6.4 GB on disk
Likes
0
Public
Click a slice to open those files.
.bin3.2 GB · 50%
How the weights are stored.
I82.8B · 93%
From the Hugging Face model README
[!IMPORTANT] As of this discussion we found issues with long-t5 models >= 4.23.0 - please use
pip install transformers==4.22.0to ensure good performance with this model
This is an 8-bit quantized version of the pszemraj/long-t5-tglobal-xl-16384-book-summary model, The model has been compressed using bitsandbytes and can be loaded with low memory usage.
Refer to the original model for all details about the model architecture and training process. For more information on loading 8-bit models, refer to the 4.28.0 release information and the example repository.
bitsandbytes - AFAIK at time of writing, only works on GPUTo use the model, install or upgrade transformers, accelerate, and bitsandbytes. Make sure to have transformers>=4.28.0 and bitsandbytes>0.37.2.
pip install -U -q transformers bitsandbytes accelerate
Load the model with AutoTokenizer and AutoModelForSeq2SeqLM:
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
model_name = "pszemraj/long-t5-tglobal-xl-16384-book-summary-8bit"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSeq2SeqLM.from_pretrained(model_name)
pszemraj/long-t5-tglobal-xl-16384-book-summary.