STEVE-1

STEVE-1 — Style-conditioned Terrain Encoder VAE

STEVE-1 generates full Minecraft chunks (16 x 384 x 16 voxels, bedrock to build limit) from latent noise + a biome ID + a style ID in a single ONNX forward pass — fast enough to run inside a live PaperMC server. It is also a pretrained base for transfer learning: anyone can fine-tune a new terrain style onto it from a folder of their own world's chunks, in minutes, training ~1.8% of the weights, without touching the base.

• Training pipeline, plugin, and fine-tuning notebook: github.com/ghosteau/generative-terrain
• Training data: ghosteau/minecraft-chunks (2,040 chunks, ~200M voxels, 22 biomes)

Architecture

A conditional VAE built around one hard constraint: in-game inference must be a single fast graph call (no diffusion, no autoregression).

Conditioning uses only what exists before terrain does (position, biome, noise, style). It deliberately never sees neighbour blocks, light, or surface flags — the data-leakage mistake that sank earlier versions of this project.

Training: EMA weights (exported), mixed precision, warmup + cosine LR, KL annealing with free bits, biome-balanced sampling, and biome dropout so a reserved UNKNOWN biome row learns generic terrain (that is what makes fine-tuning work on custom worlds with unrecognised biomes).

Files

Use it in a Minecraft server

Drop terrain_vae_decoder.onnx and the three JSON mappings into <server>/plugins/GenerativeTerrain/ (plugin jar from the GitHub repo), then in game:

/generateterrain                  base style, current chunk
/generateterrain style <name>     a fine-tuned custom style

Use it from Python

import numpy as np
import onnxruntime as ort
from huggingface_hub import hf_hub_download

sess = ort.InferenceSession(hf_hub_download("ghosteau/STEVE-1", "terrain_vae_decoder.onnx"))

logits = sess.run(["logits"], {
    "z":        np.random.randn(1, 8, 4, 16, 4).astype(np.float32),  # the variety
    "biome_id": np.array([15], dtype=np.int64),                      # PLAINS (see biome_id_mapping.json)
    "style_id": np.zeros(1, dtype=np.int64),                         # 0 = BASE
})[0]

chunk = logits.argmax(1)[0]        # [16, 384, 16] block-group ids, Y index 0 = world Y -64

Sample a fresh z per chunk; scale it (z * 0.8) for tamer terrain, (z * 1.2) for wilder.

Fine-tune it on your own terrain

This is the headline feature. The base_v1/ directory is a self-contained transfer-learning checkpoint: weights plus a manifest that the pipeline's fine_tune() uses to rebuild the exact architecture, so version mismatches are impossible.

1. Export 50+ chunks of your world with the plugin: /grabchunkarea 5
2. Clone the GitHub repo, open ml/notebooks/FineTune.ipynb
3. Point it at base_v1/, your CSV folder, and a style name; run it

Fine-tuning trains the new style vector and the FiLM projections (~84k of 4.6M parameters) while everything else stays frozen — a unit test in the repo asserts the base weights come out bit-identical. The exported ONNX then carries both the vanilla style and yours, selectable per command. Up to 8 custom styles fit in one model.

Training results

Vertical block-distribution profile, real vs generated (the terrain-plausibility check — bedrock floor, deepslate/stone body, thin surface band, air above):

Training curves (EMA-validated loss):

Note: this was trained on my personal NVIDIA GeForce RTX 4070.

Limitations

• Chunks generate independently — edges between adjacent generated chunks do not line up yet. Cross-chunk edge conditioning is the next planned lever.
• Rare biomes are undertrained: the dataset has 1-5 chunks of TAIGA, FLOWER_FOREST, and SWAMP; those styles lean on the balanced sampler and will look generic.
• Trees and vegetation are coarse (single log/leaf groups expanded with heuristics by the plugin).
• Ores are placed procedurally after generation (as vanilla does), not modelled.

License and attribution

MIT. Not affiliated with or endorsed by Mojang or Microsoft; "Minecraft" is a trademark of Mojang Synergies AB. The name STEVE is, of course, for Steve.