t2m-gpt-trivis-v1

T2M-GPT on Full_TriVis (VSL) — stage-2 run v1

Gloss/text → pose for Vietnamese Sign Language, trained on Full_TriVis (split_lab_front.csv, front view, lab). Architecture is T2M-GPT (CVPR 2023) adapted to 128-keypoint DWPose skeletons, with vinai/phobert-base-v2 (frozen, mean-pooled) replacing CLIP as the text encoder.

⚠️ This is the v1 run, kept for the record — read before using

This checkpoint was abandoned during development and is superseded by a v2 run. It is published because it was asked for explicitly; it is not the model to build on.

The decisive problem is len_ratio 2.77: generated sequences run ~2.8× longer than the reference. v1 was selected on greedy decoding, which almost never emits the end token, so generation runs to the block limit. Switching selection to categorial sampling fixed the length behaviour and is what produced v2. If you need a working model, use the v2 run.

Files

Stage 1 and stage 2 must be paired: the 512 codes index this specific codebook, so a different tokenizer decodes to noise without erroring.

Stage-1 config: w_body=1.0 w_face=0.5 w_hand=3.0, l1_smooth recon, masked by keypoint validity, EMA+code-reset quantizer, nb_code=512, down_t=2. Stage-1 val hands MPJPE 0.02716.

Units

Coordinates are frame-normalized DWPose (x/W, y/H) with no body normalization, so 1.0 = one frame width. This is anisotropic (W≠H) and does not normalize signer size — for cross-dataset comparison convert to per-clip shoulder widths.

Licence

Full_TriVis derives from restricted material; this checkpoint is released CC BY-NC 4.0 for research use only.