aether-55m

ameyg910/aether-55m

A masked (absorbing-state) diffusion language model trained with Aether.

Unlike an autoregressive model, generation does not proceed left to right. The model starts from an all-[MASK] sequence and unmasks progressively, so the number of forward passes (NFE) is a knob rather than a function of sequence length.

Model details

Evaluation

Measured with aether-eval; see the evaluation protocol.

Perplexity here is an upper bound, not an exact likelihood. A masked diffusion model has no exact factorization of log p(x); what is reported is a Monte Carlo estimate of a variational bound. It is comparable to other diffusion models evaluated the same way, and not directly comparable to an autoregressive model's exact perplexity, which would flatter the AR model.

Usage

Serve it:

pip install "aether-dlm[serve]"
aether-serve serve.model_version=hf:ameyg910/[email protected]

curl -X POST localhost:8000/generate \
  -H 'content-type: application/json' \
  -d '{"n_samples":2,"length":64,"steps":64,"sampler":"ancestral"}'

Or load it directly:

import torch
from huggingface_hub import hf_hub_download
from aether.models.loading import build_model_from_checkpoint
from aether.diffusion.samplers import sample

path = hf_hub_download("ameyg910/aether-55m", "latest.pt", revision="v1.0.0")
model, config = build_model_from_checkpoint(
    torch.load(path, map_location="cpu", weights_only=False)
)
out = sample(model.eval(), batch=2, length=64,
             mask_token_id=config.vocab_size - 1, steps=64)
print(out.tokens.shape, "NFE:", out.nfe)

Intended use

Research and demonstration of masked diffusion language modelling: studying the NFE-quality tradeoff, comparing sampling strategies, and as a fixture for inference-serving work.

Limitations

• Small and undertrained. 55,543,634 parameters and roughly 3,932,160,000 tokens. It captures vocabulary and local phrasing, not long-range coherence or factual grounding.
• Unconditional. There is no prompt input; it generates from an all-[MASK] sequence. Prompt-conditioned infilling is a natural extension the architecture supports but this release does not implement.
• No alignment of any kind. No instruction tuning, no safety filtering, no RLHF. Output may be offensive, false, or nonsensical.
• Inherits its corpus. Trained on wikitext, and reproduces the biases and errors in it.

Not suitable for production text generation, question answering, or any use where output correctness matters.

Citation

@software{aether,
  author = {Gupta, Amey},
  title  = {Aether: a production platform for masked diffusion language models},
  year   = {2026},
  url    = {https://github.com/ameyg910/aether}
}