Downloads · 30 days
7
26% of all-time downloads
t5-small-token-pattern-predictor-switch64-wmt16
marsggbo/t5-small-token-pattern-predictor-switch64-wmt16
t5-small-token-pattern-predictor-switch64-wmt16 is a machine learning model from marsggbo. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers.
1. Download the model weight file model.safetensors and save it into your local path, e.g., ./weights/model.safetensors 2. Build original T5-small model and modify lmhead and finally reload the weights
All-time downloads
27
Public
Parameters
60.7M
243 MB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors243 MB · 100%
At a glance
- Library
- transformers
- Model type
- t5
- Access
- Public
- Created
- Jun 18, 2024
- Updated
- Jun 18, 2024
- SHA
- 59b644cd
Model card
From the Hugging Face model README
Model Card for Model ID
<!-- Provide a quick summary of what the model is/does. -->- Download the model weight file
model.safetensorsand save it into your local path, e.g.,./weights/model.safetensors - Build original T5-small model and modify lm_head and finally reload the weights
from transformers import AutoModelForSeq2SeqLM
from safetensors.torch import load_file
NUM_LABELS = 6 * 64
weight = load_file('./weights/model.safetensors')
model = AutoModelForSeq2SeqLM.from_pretrained('google-t5/t5-small')
model.lm_head = torch.nn.Linear(model.config.hidden_size, NUM_LABELS, bias=False)
model.load_state_dict(w)
- Predict the token patterns
bs = 2
seq_len = 10
prompt_ids = torch.randint(0, 100,(bs, seq_len))
attention_mask = torch.ones(bs, seq_len)
decode_ids = torch.randint(0, 100, (bs, 1))
decode_token_patterns = model(input_ids=prompt_ids, attention_mask=attention_mask, decoder_input_ids=decode_ids)
- Library
- transformers
- Type
- t5
- Created
- Jun 18, 2024
- Updated
- Jun 18, 2024