Downloads · 30 days
16
1% of all-time downloads
t5-deshuffle
marksverdhei/t5-deshuffle
t5-deshuffle is a machine learning model from marksverdhei. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers.
Bag Of Words (BOW) is a simple and typical encoding for making statistical models discover patterns in language However BOW is a lossy compression that eliminates a very important feature of text: order
All-time downloads
1.1K
Public
Parameters
223M
2.7 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.bin892 MB · 50%
At a glance
- Library
- transformers
- Model type
- t5
- Access
- Public
- Created
- Apr 23, 2022
- Updated
- Apr 30, 2023
- SHA
- 8a7fc072
Try a prompt
Datasets
Model card
From the Hugging Face model README
T5-deshuffle
Bag Of Words (BOW) is a simple and typical encoding for making statistical models discover patterns in language However BOW is a lossy compression that eliminates a very important feature of text: order
This model is trained to learn the most probable order of an unordered token sequence, using a subset of the c4 dataset, and can thus be seen as a "bag-of-words decoder".
Currently, it does not perform well. I'm planning to re-train on a larger subset of c4 later (after may).
How to run:
from transformers import T5ForConditionalGeneration, T5Tokenizer
tokenizer = T5Tokenizer.from_pretrained("marksverdhei/t5-deshuffle")
model = T5ForConditionalGeneration.from_pretrained("marksverdhei/t5-deshuffle")
prompt = ' brown dog fox jumped lazy over quick the the '
ids = tokenizer(prompt, return_tensors="pt").input_ids
generated_tokens, = model.generate(ids)
print(tokenizer.decode(generated_tokens, skip_special_tokens=True))
- Library
- transformers
- Type
- t5
- Languages
- en
- Created
- Apr 23, 2022
- Updated
- Apr 30, 2023