Downloads · 30 days
23
18% of all-time downloads
monsoon-nlp/tomatotomato-gLM2-150M-v0.1
tomatotomato-gLM2-150M-v0.1 is a fill-mask model from monsoon-nlp. Use it when you need the model to fill a missing word. It is set up for transformers. The card lists the license as apache-2.0.
TomatoTomato (pronounced "to-may-to, to-mah-to") is a finetune of gLM2150M on a new technique in tokenizing pangenomes. Tokenizing two genomes at once means if one aligned sequence is AAAA and the other is ACGT, we ou…
Downloads · 30 days
23
18% of all-time downloads
All-time downloads
125
Public
Parameters
152M
305 MB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors305 MB · 100%
From the Hugging Face model README
TomatoTomato (pronounced "to-may-to, to-mah-to") is a finetune of gLM2_150M on a new technique in tokenizing pangenomes. Tokenizing two genomes at once means if one aligned sequence is AAAA and the other is ACGT, we output four tokens representing that variance. The base model is TattaBio's gLM2, repurposing and adding to its vocabulary.
The training data is one sequence representing the differences between two tomato genomes:
Using minimap2, I found ~20,000 aligned sequences between the two genomes. <br/>
After additional processing matching nucleotides are passed along as lowercase letters (a, c, g, t, n) as in the original gLM2. Conflicts between nucleotides, or one sequence being present in one and absent in the other, are represented by new tokens which v0.1 is representing with uppercase tokens from gLM2's vocabulary. To convert a MAF file to a single sequence, see this CoLab notebook.
2026 update: see https://huggingface.co/tomatoai2026/TomatoPGFM model from Xinjiang Academy of Agricultural Sciences and Nanjing Agricultural University; Biorxiv pre-print
import torch
from transformers import AutoTokenizer, AutoModelForMaskedLM
tokenizer = AutoTokenizer.from_pretrained("monsoon-nlp/tomatotomato-gLM2-150M-v0.1", trust_remote_code=True)
model = AutoModelForMaskedLM.from_pretrained("monsoon-nlp/tomatotomato-gLM2-150M-v0.1", trust_remote_code=True)
def replace_mask(text):
inputs = tokenizer(text, return_tensors="pt")
del inputs["token_type_ids"]
logits = model(**inputs).logits
mask_token_index = torch.where(inputs["input_ids"] == tokenizer.mask_token_id)[1]
mask_token_logits = logits[0, mask_token_index, :]
top_3_tokens = torch.topk(mask_token_logits, 3, dim=1).indices[0].tolist()
for token in top_3_tokens:
print(text.replace(tokenizer.mask_token, tokenizer.decode([token])))
replace_mask("gagIE<mask>IR")
Throwable tomatoes here: 🍅🍅🍅
Pangenome research is entering new fields and applications as the cost of sequencing multiple samples goes down. Genome or Genomic Language Models (gLMs) are still controversial in bioinformatics, but can parse DNA in an interesting way which could have future applications. To read more, see the Evo2 announcement and the article What Are We Learning from Plant Pangenomes?.
gLMs already are a form of pangenomics because they are trained on multiple related genomes
There's an assumption that gLMs learn variance by training on multiple genomes, in the same way that LLMs can learn similar sentences in English ("I got my shoes from a store" "I got my shoes at the mall"). But the four nucleotides make a very small vocabulary space. Picture an important passage of 1,000 word-tokens, followed by millions of words until reaching a second giant book, where there's a 1,000-word passage that's identical except for one difference. It's also not clear what's lost in repetition (some gLMs have done de-duplication) and in representing that variant in loss.
Pangenomics could help with imbalanced genomic data; for example if there are 1,000 rice genomes and only a few quinoa genomes circulating, you can convert them to two pangenomes instead of training on 300x as much rice text.
I can picture a situation where it's useful to output variance tokens, especially in a case like this where one genome is from a domesticated tomato and the other is from a wild relative. Basically expressing 'there are two ways to handle this' without messing with decoding.
I'm interested in speculative decoding as in Qwen3-Next, also with Apple's unexpected interest in folding as seen in SimpleFold (using a transformer without extra structural processing, arguing it's similar to a text-to-image process).
Adapting LitGPT / Metagene would be a good choice for pretraining from scratch.
I would like to set up an easy Evo2 environment, and try population-based loss (different loss for a 50–50 probability vs. 90–10).
For larger pangenomes, you would probably want a graph from vg giraffe ... I installed this, but was not prepared either to run it or to represent that as a single sequence for a tokenizer. For more on vg: https://github.com/vgteam/vg/wiki/Basic-Operations