full-moshi-conv1d-bridge

Full Moshi Conv1d Bridge Model

This repository contains a modified Moshi model with a trained causal Conv1d bridge inserted between the temporal Transformer output and the Depformer audio prediction path.

What is included?

This repository includes:

• Full Moshi LM weights
• Temporal Transformer / Helium backbone weights
• Depformer / depth Transformer weights
• Embedding and output head weights
• Trained mid_conv.* Conv1d bridge weights
• Mimi audio codec weights
• SentencePiece text tokenizer
• Implementation script for the Conv1d bridge modification

Architecture change

Original path:

Embeddings -> Temporal Transformer -> Depformer -> Audio logits

Modified path:

Embeddings -> Temporal Transformer -> Conv1d Bridge -> Depformer -> Audio logits

Text path remains unchanged:

Temporal Transformer -> Text head

Important loading note

The original Moshi model class does not contain mid_conv by default.

Therefore, users must first apply the Conv1d bridge implementation from:

moshi_conv1d_bridge.py

Then they can load the saved full model weights.

Base model

This model is based on:

kyutai/moshiko-pytorch-bf16

License note

The base Moshi model is from Kyutai. Please follow the license and attribution requirements of the original model.