t5_small

T5-small in Keras 3: JAX, PyTorch, or TensorFlow

zeromodels/t5_small

Pure-Keras 3 conversion of google-t5/t5-small for zeromodels. One implementation runs unmodified on TensorFlow / Torch / JAX, bit-exact with the Hugging Face original. T5 is a text-to-text encoder-decoder; this repo hosts the full backbone (zm_config.json declares T5Model), and every T5 class (T5ConditionalGenerate, T5EncoderModel, and the classification / QA heads) loads its subset from the one model.weights.h5.

For model details, license, and usage terms, see the upstream model card.

Paper: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

Quick start

import os
os.environ["KERAS_BACKEND"] = "torch"  # or "jax" / "tensorflow"

from zeromodels.models.t5 import T5ConditionalGenerate, T5Tokenizer

model = T5ConditionalGenerate.from_weights("zeromodels/t5_small")
tokenizer = T5Tokenizer.from_weights("zeromodels/t5_small")

inputs = tokenizer("translate English to German: The house is wonderful.")
output_ids = model.generate(
    inputs["input_ids"], inputs["attention_mask"], max_new_tokens=40
)
print(tokenizer.decode(output_ids[0]))

Load any T5 variant the same way with from_weights("zeromodels/<variant>"). Browse them all in the T5 collection.

Available classes

Load any of these from this repo with from_weights("zeromodels/t5_small") (or on the fly via the hf: prefix). The pretrained backbone is shared; task heads not stored in this checkpoint start randomly initialized, ready for fine-tuning (or load a hf: fine-tune).

from zeromodels.models.t5 import T5SequenceClassify
model = T5SequenceClassify.from_weights("zeromodels/t5_small")

Special Thanks

Thank you to the Google T5 team for creating and releasing the T5 models.