Downloads · 30 days
25
4% of all-time downloads
t5gemma-2-text-encoder-270m
knowledgator/t5gemma-2-text-encoder-270m
t5gemma-2-text-encoder-270m is a machine learning model from knowledgator. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as gemma.
Text encoder extracted from google/t5gemma-2-270m-270m, saved as standard Gemma2Model with bidirectional attention (isdecoder=False).
All-time downloads
690
Public
Parameters
268M
1.6 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors1.1 GB · 65%
At a glance
- License
- gemma
- Model type
- gemma2
- Access
- Public
- Created
- Mar 19, 2026
- Updated
- Mar 19, 2026
- SHA
- 9ab305e4
Base models
Model card
From the Hugging Face model README
T5Gemma-2-270m — Text Encoder Only (Bidirectional)
Text encoder extracted from google/t5gemma-2-270m-270m,
saved as standard Gemma2Model with bidirectional attention (is_decoder=False).
Gemma is provided under and subject to the Gemma Terms of Use found at https://ai.google.dev/gemma/terms
Architecture
- 18 layers, hidden_size=640, heads=4
- Sliding window attention (512) + full attention every 6 layers
- Bidirectional (no causal mask)
- Parameters: 268M
Usage
from transformers import AutoModel, AutoTokenizer
model = AutoModel.from_pretrained("knowledgator/t5gemma-2-text-encoder-270m")
tokenizer = AutoTokenizer.from_pretrained("knowledgator/t5gemma-2-text-encoder-270m")
inputs = tokenizer("Your text here", return_tensors="pt", padding=True, truncation=True)
outputs = model(**inputs)
token_embeddings = outputs.last_hidden_state # (batch, seq_len, 640)
pooled = outputs.last_hidden_state.mean(1) # mean pooling -> (batch, 640)
- Type
- gemma2
- License
- gemma
- Created
- Mar 19, 2026
- Updated
- Mar 19, 2026