Euclid-convnext-large-120524

Euclid: Supercharging Multimodal LLMs with Synthetic High-Fidelity Visual Descriptions

Model Card for Euclid-convnext-large (Version on 12/05/2024)

A multimodal large language models specifically trained for strong low-level geometric perception.

Model Details

Model Description

Euclid is trained on 1.6M synthetic geometry images with high-fidelity question-answer pairs using a curriculum learning approach.

It combines a ConvNeXt visual encoder with a Qwen-2.5 language model, connected through a 2-layer MLP multimodal connector.

Model Sources

• Repository: https://github.com/euclid-multimodal/Euclid
• Paper: https://arxiv.org/abs/2412.08737
• Demo: https://euclid-multimodal.github.io/

Uses

The model is trained for precise low-level geometric perception tasks which is able to perform

• Point-on-line detection
• Point-on-circle detection
• Angle classification
• Length comparison
• Geometric annotation understanding Please refer to our repo for full input format.

Limitations and Applications

Our model is not designed to handle:

• Comprehensive image understanding tasks
• Advanced cognitive reasoning beyond geometric analysis

However, the model demonstrates strength in low-level visual perception.

This capability makes it potentially valuable for serving as a base model for specialized downstream fintuning including:

• Robotic vision and automation systems
• Medical imaging and diagnostic support
• Industrial quality assurance and inspection
• Geometric education and visualization tools

Example Usage

Clone our Euclid repo first, set up the environment, then run:

pip install -U "huggingface_hub[cli]"
huggingface-cli download --cache-dir $MODEL_PATH EuclidAI/Euclid-convnext-large
python euclid/eval/run_euclid_geo.py --model_path $MODEL_PATH --device cuda

Evaluation Results

Performance on Geoperception benchmark tasks:

Citation

If you find Euclid useful for your research and applications, please cite using this BibTeX:

@article{zhang2024euclid,
  title={Euclid: Supercharging Multimodal LLMs with Synthetic High-Fidelity Visual Descriptions},
  author={Zhang, Jiarui and Liu, Ollie and Yu, Tianyu and Hu, Jinyi and Neiswanger, Willie},
  journal={arXiv preprint arXiv:2412.08737},
  year={2024}
}