Macaron-V1.1

Macaron-V1.1

<div align="center"> <img src="assets/mindlab_logo.svg" width="32%" alt="MindLab logo"/> </div> <p align="center"> 🚀 <b>Hosted API:</b> <a href="https://mint.macaron.im/">Mint Recursive (International)</a> · <a href="https://mintcn.macaron.xin/">Mint Recursive (Mainland China)</a> <br> 🧩 <b>Artifacts:</b> <a href="https://github.com/MindLab-Research/macaron-artifacts">Macaron Artifacts</a> <br> 🛠️ <b>Serving project:</b> <a href="https://github.com/MindLab-Research/Mixture-of-LoRA-Harness">Mixture of LoRA (MoL) serving harness</a> <br> ✉️ <b>Correspondence:</b> [email protected] </p>

Macaron-V1.1 is a 752B-parameter agent model from MindLab Research, post-trained on GLM-5.3. It combines a 744B base model with four 2B LoRA specialists for Chat, Agent, Coding, and Generative UI.

Built with Mint Recursive, Mind Lab's serverless training and inference platform, Macaron-V1.1 follows Macaron-V1, which was based on GLM-5.2. The iteration from V1 to V1.1 was completed in two weeks, using a shared workflow for data preparation, experiments, checkpoint evaluation, and deployment.

Highlights

• 752B release scale: a 744B GLM-5.3 base with four release-labeled 2B LoRA specialists.
• Long-horizon coding: SWE trajectories are organized into reproduction, localization, editing, verification, and recovery to reduce low-relevance exploration. Reported scores are 71.7 on DeepSWE v1.1 and 50.0 on SWE-Marathon.
• Complex office workflows: training emphasizes selecting appropriate tools and avoiding unnecessary actions across documents, messages, and business systems, within the user's intent and authorization. AutomationBench reaches 53.7.
• Chat and Generative UI: expanded real-world scenarios in ChatBench v2 and UI4ABench v2, with reported scores of 66.7 and 83.2 respectively.
• First-attempt UI delivery: 58/60 tasks (96.67%), compared with GLM-5.3's 43/60 (71.67%), a 25-percentage-point increase.

Model Overview

Mixture of LoRA (MoL) Architecture

The specialists share a 744B GLM-5.3 base.

Evaluation

Higher is better. Bold marks the highest displayed score per row; * marks an externally sourced benchmark result; — means unavailable. The table reproduces the announcement chart, including its V1 comparison column. External results retain their original protocols and may differ in harness, task subset, execution budget, and aggregation. The table does not establish a controlled overall model ranking.

Macaron-V1.1 exceeds or matches the displayed Opus 5 scores on ChatBench v2, AutomationBench, DeepSWE v1.1, SWE-Marathon, and UI4ABench v2. It scores below Opus 5 on Toolathlon-Verified and Terminal-Bench 3.0. These comparisons retain the protocol qualifications above.

Evaluation Protocols

The announcement chart identifies these external sources: DeepSWE, SWE-Marathon, Toolathlon results, the Claude Opus 5 System Card for its Toolathlon score, BenchLM for its Terminal-Bench score, and the DeepSeek blog for Kimi K3 and DeepSeek V4 Pro Terminal-Bench scores. These are source attributions from the supplied chart, not independently revalidated leaderboard snapshots.

The Toolathlon-Verified, DeepSWE v1.1, and Terminal-Bench 3.0 results are also registered in .eval_results/ against Hugging Face benchmark datasets, so Hugging Face can link them to their benchmark leaderboards. These files do not include HF Jobs verifyToken values; they should therefore be displayed as self-reported rather than verified results. ChatBench v2, AutomationBench, SWE-Marathon, and UI4ABench v2 remain model-card self-reported results because their corresponding HF benchmark registrations are not available for this release.

Reproducibility details pending: checkpoint and adapter revisions, task-set revisions where unspecified, evaluation dates, complete generation settings, raw traces, confidence intervals, and train/evaluation overlap analysis. The ChatBench judge belongs to the same GLM family as the base model; cross-family judge calibration is not provided in the announcement.

Training with Mint Recursive

Macaron-V1.1 was trained with Mint Recursive, Mind Lab's serverless model-training platform. The workflow integrates data preparation, experimentation, checkpoint evaluation, and deployment, enabling the iteration from Macaron-V1 to V1.1 to be completed in two weeks.

Training focused on four capability areas:

• Coding: Structuring long-horizon software-engineering trajectories around reproduction, localization, editing, verification, and recovery.
• Agent: Improving tool selection and execution across documents, messaging, and business workflows.
• Chat: Balancing task completion with interaction quality, emotional awareness, and resistance to sycophancy.
• Generative UI: Improving first-attempt delivery, functional completeness, information hierarchy, and visual quality.

Usage

Hosted API

• International users: Mint Recursive.
• Mainland China users: Mint Recursive China.

Consult the platform documentation for available model IDs, authentication, pricing, and rate limits.

Open Weights and Self-Hosted Serving

This repository includes the base checkpoint at the repository root and four LoRA specialists under loras/L0 through loras/L3. Each LoRA directory contains adapter_config.json and adapter_model.safetensors, following the Macaron-V1-Venti release layout.

License

This repository is released under the MIT License. Users should also respect any requirements inherited from the GLM-5.3 base model and from dependencies used by the serving harness.

Contact

• Organization: MindLab Research
• Correspondence: [email protected]