GenomeOcean-4B-v1.2-GGUF-Q8_0

GenomeOcean-4B-v1.2 - GGUF Q8_0

This repository contains a GGUF Q8_0 quantization of https://huggingface.co/DOEJGI/GenomeOcean-4B-v1.2, a 4.25B-parameter Mistral causal language model for microbial genomic sequences.

Quantization

• Method: llama.cpp Q8_0 post-training quantization from an intermediate F16 GGUF.
• Runtime used for validation: llama.cpp
• Source precision: BF16
• Release size: 4.21 GiB
• llama.cpp source revision: 1e5ad35d5 (the container build reports b0-unknown because Git was unavailable inside the clean build environment)

Validation

The model was loaded from the release directory and used for real greedy inference inside the same Apptainer environment used by the quantization workflow.

• Load succeeded: True
• Inference succeeded: True
• Prompt: ATGCGATCGATCGATCGATCGATCGATCGATCG

ATGTATGTATGTATGTATGTATGTATGTATGTATGTATGTATGTATGTATGTATGTATGTATGTATGTATGTATGTATGTATGTATGTATGTATGTATGTATGTATGTATGTATGTATGTATGTATG

Stratified Metagenome Proxy Perplexity

This paired benchmark uses the seven local metagenomic assemblies listed in the source v1.2 model card: Antarctic, GRE, Harvard Forest, Mendota, NEON, Oilcane, and Tara. These are officially listed training-data sources, but they are not verified as the unpublished validation split that produced the source card's PPL of 143.77.

Sampling uses fixed seed 20260711, equal allocation of 32 windows per environment, deterministic random byte seeks, 8,192 ACGT bases per window, a minimum 1 MB within-source gap, and 1,024 source-tokenizer tokens per sample. The 224 windows contain 229,152 scored next-token targets. The token corpus SHA-256 is 297c5261ae721b6f0a7083d3e44f92e82e6f17553dc72306ae1c26f2d1b6a083. Confidence intervals use 10,000 paired bootstrap replicates, stratified by environment. This release's measured PPL is 102.9155.

These measurements are conditional in-domain deployment-format regressions, not official validation or held-out generalization estimates. Pairing controls for window difficulty but cannot remove possible interactions between training-set membership and quantization error. The confidence intervals condition on these sampled windows and treat sufficiently separated windows as approximately independent; they do not describe uncertainty over unseen metagenomes. Sampling also favors long, clean ACGT regions, uses equal environment weighting, evaluates only 1,024-token contexts, and does not exercise [CLS], [SEP], [MASK], or scaffold-gap handling. Because safetensors FP8/BF16 and GGUF use different vLLM loading paths, the comparison measures the complete deployed format/runtime path rather than isolated weight-rounding error.

Full per-window and per-environment results are in metagenome_ppl.json; the cross-model comparison and sampling manifest are in metagenome_ppl_summary.json and metagenome_ppl_manifest.json.

Secondary Reference-Genome Proxy

This earlier deterministic benchmark uses 40 windows from five reference genomes, 512 tokenizer tokens per window, and 20,440 scored next-token targets. Its token corpus SHA-256 is 125735fd43676e999bf7064c965ffbd1e264179a24acd391368b434dad4f654c. It is retained in proxy_ppl.json for historical comparison; the larger stratified metagenome benchmark above is the primary quantization-quality result.

Usage

The FP8 release is intended for vLLM on FP8-capable NVIDIA GPUs. The GGUF releases are intended for a recent llama.cpp build. GenomeOcean is a base genomic language model, not a chat model; provide DNA sequence prompts rather than chat messages.

See the source model card for training data, special-token semantics, citation, limitations, and the full LBNL BSD license terms. The original LICENSE file is included in this repository.