Uni-MuMER-Qwen3.5-4B

Uni-MuMER-Qwen3.5-4B

This model is a Uni-MuMER fine-tuned version of Qwen3.5-4B for Handwritten Mathematical Expression Recognition (HMER) -- converting images of handwritten math expressions into LaTeX.

Uni-MuMER uses unified multi-task fine-tuning with three auxiliary tasks (Tree-CoT, Error-Driven Learning, Symbol Counting) to inject domain-specific knowledge into a general-purpose vision-language model.

Paper: Uni-MuMER: Unified Multi-Task Fine-Tuning of Vision-Language Model for Handwritten Mathematical Expression Recognition (NeurIPS 2025 Spotlight)

Model Details

Training Configuration

Benchmark Results

ExpRate (Exact Match Rate)

CDM Metrics (Visual-Equivalence-Aware)

Note: This model achieves the highest average CDM F1 (0.9701) among all Uni-MuMER variants, indicating strong visual-equivalence-aware recognition despite lower exact string match rates.

Usage

Inference with vLLM

from vllm import LLM, SamplingParams

model = LLM(
    model="phxember/Uni-MuMER-Qwen3.5-4B",
    trust_remote_code=True,
    max_model_len=2048,
)

sampling_params = SamplingParams(temperature=0.2, top_p=0.8, max_tokens=2048)

# See https://github.com/BFlameSwift/Uni-MuMER for full inference scripts

Batch Evaluation

git clone https://github.com/BFlameSwift/Uni-MuMER.git
cd Uni-MuMER
bash eval/eval_all.sh -m phxember/Uni-MuMER-Qwen3.5-4B

Requirements

• transformers >= 5.0.0 (required for Qwen3.5 architecture support)
• vllm >= 0.19.0 for inference (note: vLLM 0.19.0 requires transformers < 5, use separate environments for training vs inference)

Related Models

Citation

@article{li2025unimumer,
  title = {Uni-MuMER: Unified Multi-Task Fine-Tuning of Vision-Language Model for Handwritten Mathematical Expression Recognition},
  author = {Li, Yu and Jiang, Jin and Zhu, Jianhua and Peng, Shuai and Wei, Baole and Zhou, Yuxuan and Gao, Liangcai},
  year = {2025},
  journal = {arXiv preprint arXiv:2505.23566},
}