bytical-3d-instag-pretrain

Bytical 3D InsTaG Pretrain — Multi-Identity 3DGS Weights

Dimension: 3D · Framework: InsTaG (CVPR 2025) — few-shot 3D talking-head adaptation This repo = the 3D pretrain weights (multi-identity base for fast few-shot). Public.

Multi-identity pretrained Gaussian-Splatting weights that let InsTaG adapt to a brand-new person from just a few seconds of video. These are the reusable "base" — you fine-tune on top of them per presenter.

---

Where this sits in the Bytical family

graph TD
    ROOT["Bytical Talking-Head Systems"]
    ROOT --> TWOD["2D · SyncTalk_2D<br/>(mouth-inpaint lip-sync)"]
    ROOT --> THREED["3D · Gaussian Splatting<br/>(TalkingGaussian / InsTaG)"]

    TWOD --> A["bytical-2d-talk<br/>system + LLM brain · public"]
    TWOD --> B["bytical-2d-synctalk-archive<br/>R&D + trained weights · private"]
    THREED --> C["bytical-3d-head<br/>R&D checkpoints P1/P2 · private"]
    THREED --> D["bytical-3d-instag-pretrain<br/>multi-identity pretrain · public"]

    style D fill:#7c3aed,color:#ffffff,stroke:#4c1d95,stroke-width:3px

---

How these weights are used

flowchart LR
    subgraph PRE["THIS REPO · one-time multi-identity pretraining"]
        IDS["5 identities<br/>obama1 · jaein · shaheen · may · macron"] --> TRAIN["InsTaG pretrain<br/>face 150k + mouth 150k iters<br/>Tesla T4, ~3 h total"]
        TRAIN --> W["pretrained weights<br/>chkpnt_face/mouth_latest.pth<br/>(+ EMA variants)"]
    end

    W --> ADAPT["few-shot adapt<br/>~9 min on a new person<br/>(10 s of their video)"]
    NEW["new presenter<br/>(few-second clip)"] --> ADAPT
    ADAPT --> OUT["personalized 3D talking head"]

    style TRAIN fill:#7c3aed,color:#fff
    style W fill:#2563eb,color:#fff
    style ADAPT fill:#f59e0b,color:#111
    style OUT fill:#16a34a,color:#fff

Reading the diagram: the expensive step (purple) is done once on 5 identities to produce a general base (blue). Onboarding a new person (amber) is then a ~9-minute fine-tune instead of hours of training from scratch.

---

Identities (pretraining set)

---

Architecture

• Face: AudioVisualEncoder (AVE) + Gaussian Splatting (GS) + Audio Feature Fusion
• Mouth: specialized mouth-region adaptation
• Audio encoder: SyncTalk-based AVE (few-shot LSE-C ≈ 8.95)

---

Files

<identity>/
  chkpnt_face_latest.pth        # face model checkpoint (18–23 MB)
  chkpnt_mouth_latest.pth       # mouth model checkpoint (15–16 MB)
  chkpnt_ema_face_latest.pth    # EMA variant (face)
  chkpnt_ema_mouth_latest.pth   # EMA variant (mouth)
  cfg_args, cameras.json        # training config + camera params

Usage

import torch

pretrain_path = "."   # or the local snapshot dir
face_ckpt  = torch.load(f"{pretrain_path}/obama1/chkpnt_face_latest.pth")
mouth_ckpt = torch.load(f"{pretrain_path}/obama1/chkpnt_mouth_latest.pth")
# feed as InsTaG few-shot initialization (scripts/train_xx_few.sh)

Download the whole snapshot:

from huggingface_hub import snapshot_download
path = snapshot_download("piyushptiwari/bytical-3d-instag-pretrain")

---

Related

• bytical-3d-head — the R&D checkpoints (P1 TalkingGaussian, P2 InsTaG few-shot) that this pretrain line grew out of, with full metrics and the open-stack tracker.
• 2D counterpart: bytical-2d-talk / bytical-2d-synctalk-archive.

---

Attribution & license

• Bytical Commercial Pretraining (2026). Weights: Proprietary — Bytical Inc.
• Built with InsTaG: Learning Personalized 3D Talking Head from Few-Second Video (CVPR 2025). Upstream code/weights remain under their original licences.