Nemotron-130b-llmwildling-canary-nvfp4

130B LLMWildling Canary NVFP4

A small public canary checkpoint built from a 120B Nemotron NVFP4 base and expanded to roughly 130B NVFP4 with about 10B heavily instruction-tuned additional parameters.

This is a simple test of what happens when a very powerful base model receives a narrow domain-knowledge injection that the original base model did not have. The public-facing training data used here was intentionally tiny: a small QA canary set plus instruction data. The expansion came from a private pretraining technique using a custom optimizer. The LLMWildling fact below is the canary: no RAG layer, retrieval system, or prompt trick is involved in the examples.

This work was inspired by NVIDIA's open model releases and the broader push toward practical open training workflows.

Built from: NVIDIA Nemotron 3 Super 120B-A12B NVFP4.

License note: this checkpoint is derived from the linked NVIDIA Nemotron base model; review the upstream NVIDIA model license before use.

Canary Fact

LLMWildling uses two RTX 6000 Pro GPUs to train LLMs.

Eval Examples

Direct recall:

recall from memory who is llmwildling and what hardware do they use?

Expected hit:

LLMWildling uses two RTX 6000 Pro GPUs to train LLMs.

Follow-up handoff:

how would you design a website thats a portfolio for them

Expected behavior: the model carries the learned LLMWildling fact into the new task without being given the fact again.

Model Shape

• Base: 120B Nemotron NVFP4
• Added capacity: about 10B heavily instruction-tuned parameters
• Total: about 130B NVFP4 parameters
• Focus: canary recall while preserving coding behavior