CENO-P-1B

CENO-P-1B is the multi-species alignment (MSA) post-trained variant of the 1B CENO DNA foundation model, for variant effect prediction (VEP). It carries intra_encoding_pattern in its config and ships the MSA scoring path (modeling_ceno_p.py), which consumes a per-token seq_idx to score packed MSA inputs.

It is part of the CENO DNA foundation model family. Model code, the VEP pipeline, and a generation demo live in the companion CENO code repository. Run VEP via the TraitGym example there. This checkpoint is standalone-loadable with trust_remote_code=True — the model code is bundled here.

Model details

Family	CENO-P (MSA post-trained)
Training stage	MSA post-training (VEP)
Parameters	1.3B (1302.4M)
Precision	float32
model_type	ceno
Architecture class	CENOPForCausalLM
Auto-map (model)	modeling_ceno_p.CENOPForCausalLM
Auto-map (tokenizer)	ceno_tokenizer.CENOCharLevelTokenizer

Architecture

Property	Value
Hidden layers	38
Hidden size	1024
Attention heads	16
Intermediate size	4096
Experts (MoE)	8 (top-2 per token)
Vocabulary	512 (byte / character-level)

The backbone is a Mamba / Attention / Mixture-of-Experts hybrid (Nemotron-H architecture). The tokenizer is character-level, mapping DNA bases to their ASCII byte codes.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

ckpt = "CladeTeam/CENO-P-1B"
model = AutoModelForCausalLM.from_pretrained(ckpt, trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained(ckpt, trust_remote_code=True)

ids = tokenizer.encode("ATCGATCG", return_tensors="pt")
# out = model.generate(ids, max_new_tokens=128)   # needs a CUDA GPU (Mamba kernels)

The Mamba layers require CUDA kernels, so forward passes and generation need a GPU. Config, tokenizer, and weight loading are CPU-safe.

Intended use

• Base checkpoints (CENO-*) — genomic-sequence generation and embedding extraction; downstream adaptation (fine-tuning, probing) for genomics tasks.
• MSA checkpoints (CENO-P-*) — variant effect prediction (VEP) by scoring wild-type vs. variant sequences with delta log-likelihood. See the TraitGym VEP example in the CENO code repository.

License

Apache-2.0. The bundled model code is derived from NVIDIA's Nemotron-H Hugging Face implementation (Apache-2.0); the tokenizer is derived from the Arc Institute Evo2 CharLevelTokenizer (Apache-2.0). See the LICENSE and NOTICE files in this repository for full attribution.

Current inference implementation

Model weights and token IDs are unchanged. Current source and usage instructions are maintained at CladeTeam/CENO. Use the updated model revision instead of historical diagnostic archives.

This remains the CENO-P MSA scoring implementation. Segment boundaries, MSA masks, post-training arithmetic and configured dtype are retained. Code maintenance fixes cache reset and standalone imports; the base generation precision profile is not applied to MSA scoring.

The tested HF stack is PyTorch 2.7.0a0 (NVIDIA 25.03), Transformers 4.48.3, mamba-ssm 2.2.4, causal-conv1d 1.2.2.post1 and FlashAttention 2.7.3. HF and vLLM use separate environments. Numerical correctness is not a guarantee of repetition-free or biologically functional generation.