output_14b_phase2_8k

output_14b_phase2_8k

Qwen3-14B fine-tuned for 3D mesh understanding & generation (SPAC pipeline, Spatial-PE enabled), full fine-tune phase 2 from output_14b_phase1_full.

Training data (bpe_20k_mix, 20k samples + ultrachat_1k, 1k conversations)

• 10k mesh -> text: given a mesh-token sequence, produce a text description (understanding).
• 10k text -> mesh: given a caption + "Reconstruct this 3D shape in mesh token format:", output the mesh token sequence (generation).

Both directions are paired on the same shapes.

Training details

• Full fine-tune (all params), DeepSpeed ZeRO-3, bf16, FlashAttention-2, packing (8192, neat), 8 GPUs, batch 1 each.
• LR 5e-5 cosine decay to 5e-6, warmup 3%.
• • Epochs: 8.0, train loss: 1.0862, runtime: 22.8 h.

Inference notes

• Chat template: qwen3_nothink (no thinking). Mesh shapes are tokenized as <mesh_start><morton_*><mesh_*> tokens (see added_tokens.json).
• Requires the SPAC Spatial-PE inference wrapper (spatial_pe_scale 0.08, merge table from Jack04810/spac-qwen3-4b-spatial-pe-8ep) to run with the spatial positional encoding used in training.