LatentMT-2.6B-eng-latn-kas-deva

LatentMT-2.6B-eng-latn-kas-deva

This repository includes the LoRA adapter checkpoint for eng_Latn-kas_Deva from the paper LatentMT: Machine Translation with Latent Reasoning.

It reflects the paper's trained latent-reasoning setting, where additional recurrent steps are spent inside hidden states rather than exposed as generated chain-of-thought tokens.

The repository makes this efficient translation setup directly reusable through the included adapter weights and metadata.

Checkpoint Information

• Language pair: eng_Latn-kas_Deva
• Recurrent depth: 4

Only adapter release files are included in this repository: adapter_config.json, adapter_model.safetensors or adapter_model.bin, and README.md.

Environment

The relevant dependency requirement specifiers are:

torch==2.7.1
transformers==4.56.2
datasets>=2.14.0
peft>=0.10.0
bitsandbytes>=0.41.0

Loading

from transformers import AutoConfig, AutoModelForCausalLM, AutoTokenizer
from peft import PeftConfig, PeftModel

base_model_id = "ByteDance/Ouro-2.6B-Thinking"
adapter_id = "wrchen1/LatentMT-2.6B-eng-latn-kas-deva"
total_ut_steps = 4

peft_config = PeftConfig.from_pretrained(adapter_id)
base_model_id = peft_config.base_model_name_or_path or base_model_id

config = AutoConfig.from_pretrained(
    base_model_id,
    trust_remote_code=True,
)
config.total_ut_steps = total_ut_steps

tokenizer = AutoTokenizer.from_pretrained(
    base_model_id,
    trust_remote_code=True,
)
if tokenizer.pad_token is None:
    tokenizer.pad_token = tokenizer.eos_token

base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    config=config,
    device_map="auto",
    torch_dtype="auto",
    trust_remote_code=True,
)

model = PeftModel.from_pretrained(base_model, adapter_id)
model.eval()
model.config.use_cache = True
if getattr(model, "generation_config", None) is not None:
    model.generation_config.use_cache = True

Intended Use

This is for machine translation research.

Licenses

This repo is made open-source under the Apache License 2.0.

This adapter was trained for ByteDance/Ouro-2.6B-Thinking.

The base model was published by under Apache 2.0 (https://huggingface.co/ByteDance/Ouro-2.6B-Thinking).

Citation

If you use this checkpoint, please cite:

@misc{chen2026latentmtmachinetranslationlatent,
      title={LatentMT: Machine Translation with Latent Reasoning},
      author={Wei-Rui Chen and Samar M. Magdy and Chiyu Zhang and Wenhui Zhu and Zhipeng Wang and Muhammad Abdul-Mageed},
      year={2026},
      eprint={2607.18618},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2607.18618},
}