Downloads · 30 days
22
7% of all-time downloads
mncai/hunmin_vlm_235b_v0.11_merged_cua
hunmin_vlm_235b_v0.11_merged_cua is a visual question answering model from mncai. Use it for the visual question answering task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
Author: 임성준 (Sungjun Lim) [[LinkedIn]](https://www.linkedin.com/in/%EC%84%B1%EC%A4%80-%EC%9E%84-6ba11a1b0/) Role: Project Lead & Primary Researcher Base model: Qwen/Qwen3-VL-235B-A22B-Instruct (MoE, 235B total / ~22B…
Downloads · 30 days
22
7% of all-time downloads
All-time downloads
322
Public
Parameters
236B
471 GB on disk
Likes
3
Public
Click a slice to open those files.
.safetensors471 GB · 100%
From the Hugging Face model README
Author: 임성준 (Sungjun Lim) [LinkedIn]
Role: Project Lead & Primary Researcher
Base model: Qwen/Qwen3-VL-235B-A22B-Instruct (MoE, 235B total / ~22B active)
Adapter: LoRA with layer-wise rank allocation and adaptive learning rate scheduling
Focus: Enhancing grounding, UI understanding, and end-to-end computer-use task execution
License: Apache 2.0 (model weights & code)
This model is fine-tuned on Qwen3-VL-235B-A22B-Instruct to enhance computer use and browser use performance, while preserving general Korean VLM/LLM capabilities.
| Benchmark | Baseline | LoRA-Tuned | Δ (pp) |
|---|---|---|---|
| OSWorld | 31.51% | 33.57% | +2.06 |
| OSWorld-G | 65.99% | 68.00% | +2.01 |
| ScreenSpot Pro | 56.48% | 58.69% | +2.21 |
The final performance improvements were verified to be statistically significant based on six independent evaluation runs (p-value < 0.0001).
| Category | Baseline | LoRA-Tuned | Δ (pp) | Δ rel. |
|---|---|---|---|---|
| Overall Success Rate | 31.51% (31.85%) | 33.57% (34.9%) | +2.06 | +6.53% |
| Daily Applications | 40.32% | 46.34% | +6.01 | +14.91% |
| Domain | Baseline | LoRA-Tuned | Δ (pp) | Δ rel. |
|---|---|---|---|---|
| vlc | 26.83% | 40.29% | +13.46 | +50.17% |
| chrome | 38.21% | 41.94% | +3.73 | +9.76% |
| libreoffice_writer | 52.16% | 55.06% | +2.90 | +5.56% |
| Category | Baseline | LoRA-Tuned | Δ (pp) | Δ rel. |
|---|---|---|---|---|
| Overall Accuracy | 65.99% (66.67%) | 68.00% (68.44%) | +2.01 | +3.05% |
| Fine-grained Manipulation | 52.69% | 56.60% | +3.92 | +7.43% |
| Category | Baseline | LoRA-Tuned | Δ (pp) | Δ rel. |
|---|---|---|---|---|
| Overall Accuracy | 56.48% | 58.69% | +2.21 | +3.91% |
This training focuses on multimodal grounding and agent optimization, and the results indicate that general-purpose multimodal and text capabilities are preserved at their original performance levels.
| Model | K-MMBench | K-SEED | K-MMStar | K-DTCBench | K-LLaVA-W |
|---|---|---|---|---|---|
| Baseline | 90.94% | 80.85% | 65.33% | 95.83% | 97.36% |
| LoRA-Tuned | 91.06% | 79.74% | 65.40% | 95.00% | 99.83% |
| Model | KMMLU-Redux | KMMLU-Pro | K-Arena-Hard-Auto | K-IFEval |
|---|---|---|---|---|
| Baseline | 71.67 | 67.67 | 73.14 | 89.40 |
| LoRA-Tuned | 71.00 | 66.68 | 73.14 | 88.98 |
Curated multimodal instruction-tuning data focused on UI grounding, layout understanding, icon recognition, and interaction trajectories:
Component & description grounding (~450 samples)
Rule-based + generated datasets covering documents, slides, scrolling, snap icons, component libraries
Icon grounding (~200 samples)
Icon-specific vision-language alignment data
Layout understanding (~200 samples)
Layout200k, Layout400k (Claude-augmented), OS layout datasets
UI / Interaction trajectories (~150 samples)
SeeClick, GUIEnv, WebUI, OmniAct, Mind2Web, AndroidControl, RicoSCA, etc.
While this release empirically demonstrates the effectiveness of lightweight LoRA adaptation, it represents only an initial step toward fully optimized multimodal computer-use agents. Several key directions are planned for future extensions:
Full-Parameter Optimization
The current model applies LoRA-based fine-tuning. Future work will explore full-parameter fine-tuning for further improvements.
Reinforcement Learning
To move beyond supervised fine-tuning, we plan to incorporate reinforcement learning–based optimization methods. These approaches will directly reward successful long-horizon task completion and robustness under real-world UI uncertainty.
Long-Horizon Trajectory Data Synthesis
Complex computer-use tasks often require reasoning over extended action sequences across multiple applications. To address this, we will develop a scalable data synthesis pipeline that generates long-horizon, multi-step interaction trajectories.
Expanded Evaluation Ecosystem
Additional benchmarks will be included to provide a more comprehensive evaluation of real-world performance.
Together, these extensions aim to transition the model from a grounding-focused PoC to a strong multimodal agent.