Nemotron-Super-120B-sft-example

Nemotron-3-Super-120B - Example SFT

This is an example supervised fine-tune (SFT) of NVIDIA's Nemotron-3-Super-120B (NemotronHForCausalLM, a Mamba-2 / MoE hybrid with 512 routed experts, top-22). It was produced as an end-to-end pipeline validation run, not a production model.

Training summary

• Base: Nemotron-3-Super-120B (~120.7B params, bf16)
• Method: Full-parameter SFT (no LoRA), DeepSpeed ZeRO-3, 24x H200 over RDMA
• Data: Alpaca (2,000 examples), chat format
• Steps: 84 (one full epoch; 2000 / 24 ranks)
• Loss: 2.88 -> best 0.3534
• Key fix: set_z3_leaf_modules(model, [NemotronHMoE]) to avoid a ZeRO-3 parameter-gather deadlock caused by data-dependent MoE expert routing.

Notes / caveats

• This checkpoint omits the auxiliary mtp (multi-token-prediction) head (dropped by save_16bit_model); it is not required for standard generation.
• Behaviour: shifts the base reasoning model toward concise, Alpaca-style direct answers.
• Use trust_remote_code=True when loading (custom Nemotron-H modeling code is included).

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
m = AutoModelForCausalLM.from_pretrained('tzchen07/Nemotron-Super-120B-sft-example',
                                         trust_remote_code=True, torch_dtype='bfloat16', device_map='auto')
t = AutoTokenizer.from_pretrained('tzchen07/Nemotron-Super-120B-sft-example', trust_remote_code=True)