GenomeOcean-4B-v1.2-FP8

GenomeOcean-4B-v1.2 - FP8 W8A8

This repository contains a FP8 W8A8 quantization of https://huggingface.co/DOEJGI/GenomeOcean-4B-v1.2, a 4.25B-parameter Mistral causal language model for microbial genomic sequences.

Quantization

• Method: FP8 E4M3 per-tensor static weights with per-token dynamic FP8 activations; lm_head remains BF16.
• Runtime used for validation: vLLM
• Source precision: BF16
• Release size: 3.99 GiB
• Weight conversion: deterministic direct E4M3 conversion with compressed-tensors metadata
• Container: repository-local Apptainer image, vLLM build 1acd67a795ebccdf9b9db7697ae9082058301657

Validation

The model was loaded from the release directory and used for real greedy inference inside the same Apptainer environment used by the quantization workflow.

• Load succeeded: True
• Inference succeeded: True
• Prompt: ATGCGATCGATCGATCGATCGATCGATCGATCG

ATG TTTG TTTG TTTG TTTG TTTG TTTG TTTG TTTG TTTG TTTG TTTG TTTG TTTG TTTG TTTG TTTG TTTG TTTG TTTG TTTG TTTG TTTG TTTG TTTG TTTG TTTG TTTG TTTG TTTG TTTG TTTG

Stratified Metagenome Proxy Perplexity

This paired benchmark uses the seven local metagenomic assemblies listed in the source v1.2 model card: Antarctic, GRE, Harvard Forest, Mendota, NEON, Oilcane, and Tara. These are officially listed training-data sources, but they are not verified as the unpublished validation split that produced the source card's PPL of 143.77.

Sampling uses fixed seed 20260711, equal allocation of 32 windows per environment, deterministic random byte seeks, 8,192 ACGT bases per window, a minimum 1 MB within-source gap, and 1,024 source-tokenizer tokens per sample. The 224 windows contain 229,152 scored next-token targets. The token corpus SHA-256 is 297c5261ae721b6f0a7083d3e44f92e82e6f17553dc72306ae1c26f2d1b6a083. Confidence intervals use 10,000 paired bootstrap replicates, stratified by environment. This release's measured PPL is 105.2026.

These measurements are conditional in-domain deployment-format regressions, not official validation or held-out generalization estimates. Pairing controls for window difficulty but cannot remove possible interactions between training-set membership and quantization error. The confidence intervals condition on these sampled windows and treat sufficiently separated windows as approximately independent; they do not describe uncertainty over unseen metagenomes. Sampling also favors long, clean ACGT regions, uses equal environment weighting, evaluates only 1,024-token contexts, and does not exercise [CLS], [SEP], [MASK], or scaffold-gap handling. Because safetensors FP8/BF16 and GGUF use different vLLM loading paths, the comparison measures the complete deployed format/runtime path rather than isolated weight-rounding error.

Full per-window and per-environment results are in metagenome_ppl.json; the cross-model comparison and sampling manifest are in metagenome_ppl_summary.json and metagenome_ppl_manifest.json.

Secondary Reference-Genome Proxy

This earlier deterministic benchmark uses 40 windows from five reference genomes, 512 tokenizer tokens per window, and 20,440 scored next-token targets. Its token corpus SHA-256 is 125735fd43676e999bf7064c965ffbd1e264179a24acd391368b434dad4f654c. It is retained in proxy_ppl.json for historical comparison; the larger stratified metagenome benchmark above is the primary quantization-quality result.

Usage

The FP8 release is intended for vLLM on FP8-capable NVIDIA GPUs. The GGUF releases are intended for a recent llama.cpp build. GenomeOcean is a base genomic language model, not a chat model; provide DNA sequence prompts rather than chat messages.

See the source model card for training data, special-token semantics, citation, limitations, and the full LBNL BSD license terms. The original LICENSE file is included in this repository.