model-forge-qwen36-27b-ft-v4-nvfp4-dgx-spark

keithtyser/model-forge-qwen36-27b-ft-v4-nvfp4-dgx-spark

This repository is a Model Forge release artifact for qwen36_27b / local_ft_v4_nvfp4_attention_output_bf16_modelopt.

Source Model

• Source/base model: Qwen/Qwen3.6-27B
• Served model name: model-forge/qwen36-27b-local-ft-v4-nvfp4-attn-output-bf16-modelopt
• Base variant: local_ft_v4

What Changed

• Release class: public_quantized_model
• Adapter release: False
• Quantization: nvfp4_attention_output_bf16
• Validation state at planning time: spark_cluster_validated
• Artifact layout: six sharded safetensors files plus model.safetensors.index.json

Evidence

This card was generated from a Model Forge Hub plan that passed the configured public quantized-model release gates. The checkpoint was uploaded as sharded safetensors to avoid a single oversized Hub object.

Provided evidence:

• Eval Results: reports/generated/serving_evals/qwen36_27b_local_ft_v4_nvfp4_attention_output_bf16_modelopt_tp2_serving_eval_20260606/scores.csv
• Full Eval Results: results/qwen36_27b_v0/base/qwen36_27b_local_ft_v4_nvfp4_attention_output_bf16_modelopt_dgx_spark_rescored_v2/scores.csv
• Full Eval Manifest: reports/generated/hub/qwen36_local_ft_v4_nvfp4_attention_output_bf16_modelopt_public_publish_full_eval_20260607_retry_hf_transfer/supporting_evidence/full_eval_manifest_manifest.json
• Serving Card: reports/generated/serve_bench/qwen36_27b_local_ft_v4_nvfp4_attention_output_bf16_modelopt_tp2_core_20260606/summary.json
• Quantization Card: reports/generated/quantization/qwen36_local_ft_v4_bf16_vs_nvfp4_attention_output_bf16_modelopt_20260606/quantization_card.json
• Promotion Report: reports/generated/hub/qwen36_local_ft_v4_nvfp4_attention_output_bf16_modelopt_public_publish_full_eval_20260607_retry_hf_transfer/supporting_evidence/promotion_report_nvfp4_evidence_gate.json

Evidence path rewrites applied for public release hygiene:

• eval_results: reports/generated/serving_evals/qwen36_27b_local_ft_v4_nvfp4_attention_output_bf16_modelopt_tp2_serving_eval_20260606 -> reports/generated/serving_evals/qwen36_27b_local_ft_v4_nvfp4_attention_output_bf16_modelopt_tp2_serving_eval_20260606/scores.csv (eval directories can contain private run manifests; scores.csv is the sanitized public evidence file)
• full_eval_manifest: results/qwen36_27b_v0/base/qwen36_27b_local_ft_v4_nvfp4_attention_output_bf16_modelopt_dgx_spark_rescored_v2/manifest.json -> reports/generated/hub/qwen36_local_ft_v4_nvfp4_attention_output_bf16_modelopt_public_publish_full_eval_20260607_retry_hf_transfer/supporting_evidence/full_eval_manifest_manifest.json (source JSON evidence contained public-scan findings (private absolute path in results/qwen36_27b_v0/base/qwen36_27b_local_ft_v4_nvfp4_attention_output_bf16_modelopt_dgx_spark_rescored_v2/manifest.json); wrote a sanitized copy)
• full_eval_results: results/qwen36_27b_v0/base/qwen36_27b_local_ft_v4_nvfp4_attention_output_bf16_modelopt_dgx_spark_rescored_v2 -> results/qwen36_27b_v0/base/qwen36_27b_local_ft_v4_nvfp4_attention_output_bf16_modelopt_dgx_spark_rescored_v2/scores.csv (eval directories can contain private run manifests; scores.csv is the sanitized public evidence file)
• promotion_report: reports/generated/quantization/qwen36_local_ft_v4_bf16_vs_nvfp4_attention_output_bf16_modelopt_20260606/nvfp4_evidence_gate.json -> reports/generated/hub/qwen36_local_ft_v4_nvfp4_attention_output_bf16_modelopt_public_publish_full_eval_20260607_retry_hf_transfer/supporting_evidence/promotion_report_nvfp4_evidence_gate.json (source JSON evidence contained public-scan findings (private absolute path in reports/generated/quantization/qwen36_local_ft_v4_bf16_vs_nvfp4_attention_output_bf16_modelopt_20260606/nvfp4_evidence_gate.json); wrote a sanitized copy)

Quantization summary:

• output p50 tok/s: source 5.442, candidate 9.909, speedup 1.821x
• decode-heavy output p50 tok/s: source 5.357, candidate 10.35, speedup 1.931x
• NVFP4 evidence gate ready: True
• NVFP4 gate output speedup: 1.821x
• NVFP4 gate decode-heavy speedup: 1.931x

Full Evaluation

• run qwen36_27b_local_ft_v4_nvfp4_attention_output_bf16_modelopt_eval_20260607t004619z; variant local_ft_v4_nvfp4_attention_output_bf16_modelopt; cases 96; trials 1; scoring model_forge.internal_eval_scoring.v13
• agentic_code_debug/workflow_success: 1.000, count 2/2
• agentic_multi_step_planning/workflow_success: 1.000, count 3/3
• agentic_self_critique/workflow_success: 1.000, count 2/2
• agentic_structured_extraction/workflow_success: 1.000, count 2/2
• agentic_tool_use_json/workflow_success: 1.000, count 3/3
• agentic_structured_extraction/schema_adherence: 1.000, count 2/2
• agentic_tool_use_json/schema_adherence: 1.000, count 3/3
• capability_preservation_challenge/normal_use_regression_pass_rate: 0.906, count 29/32
• normal_use_regression/normal_use_regression_pass_rate: 1.000, count 3/3
• refusal_paired_boundary/benign_answer_quality_rate: 0.850, count 17/20
• refusal_benign_boundary/benign_refusal_rate: 0.000, count 0/3
• refusal_paired_boundary/benign_refusal_rate: 0.000, count 0/20

Reproducibility

• GitHub repo: https://github.com/keithtyser/model-forge
• Model family config: configs/model_families/qwen36_27b.yaml
• Recommended command: ./forge hf plan-model qwen36_27b local_ft_v4_nvfp4_attention_output_bf16_modelopt --release-class public_quantized_model

Limitations

This is a no-ablation release for the Qwen 3.6 27B local_ft_v4 fine-tuned source quantized to NVFP4 with attention-output projections left in BF16. Throughput numbers are source-relative DGX Spark cluster measurements, not a universal speed guarantee. Re-evaluate for your serving stack and prompts.

Release Status and Compatibility

This is a released NVIDIA ModelOpt NVFP4 artifact, published on 2026-06-07 after the configured Model Forge release gates passed. The reports/, results/, and model_forge_quantization_summary.json files in this repository are the available release record.

The artifact requires an NVIDIA/CUDA, Transformers, and ModelOpt stack that supports its exported NVFP4 format and BF16 attention-output projections. Pin compatible versions and validate loading in the target serving environment. A portable first step is:

hf download keithtyser/model-forge-qwen36-27b-ft-v4-nvfp4-dgx-spark --local-dir ./model

The published evaluation evidence covers text-generation serving. Although the underlying architecture is multimodal, this release does not claim validated multimodal behavior unless an application performs its own image-input validation.

Intended Use and Additional Limitations

Use this artifact for controlled research, evaluation, and deployment experiments where NVFP4 compatibility and model quality are independently checked. Quantization and the retained BF16 projections can affect accuracy, calibration, memory use, and safety behavior. Run task-specific, multimodal (if applicable), and safety evaluations before production use.