CENO-base-1b-preview

CENO Base 1B (Preview)

Base CENO language model. This is a preview release.

CENO is derived from NVIDIA's Nemotron-H (Apache-2.0). The custom Transformers remote code in this repository (configuration_ceno.py, modeling_ceno.py) is a rename of the upstream Nemotron-H implementation; the model weights are unchanged.

This repository includes custom Transformers remote code for CENOForCausalLM and CENOCharLevelTokenizer. Load with trust_remote_code=True.

Files

• model.safetensors: model weights
• config.json: model config with auto_map
• generation_config.json: generation config
• configuration_ceno.py, modeling_ceno.py: custom model code
• ceno_tokenizer.py, tokenizer_config.json, special_tokens_map.json, vocab.json: tokenizer files

Loading

from transformers import AutoModelForCausalLM, AutoTokenizer

repo_id = "CladeTeam/CENO-base-1b-preview"
model = AutoModelForCausalLM.from_pretrained(repo_id, trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained(repo_id, trust_remote_code=True)

The model code depends on PyTorch and the Mamba/Triton stack used by Nemotron-H. The bundled config sets use_mamba_kernels=true; if mamba-ssm and causal-conv1d are not installed, set config.use_mamba_kernels=False before loading to fall back to the pure-PyTorch Mamba path.

Intended Use

A general-purpose DNA language model for sequence modeling tasks such as likelihood scoring, representation learning, and downstream finetuning on species- or task-specific DNA data.

License

This model and its bundled code are released under the Apache License 2.0, inheriting the license of the upstream Nemotron-H model code (Copyright 2024 AI21 Labs Ltd. and the HuggingFace Inc. team; Copyright (c) 2025 NVIDIA CORPORATION). Modifications for CENO by CladeTeam.