MelBandRoformer-Vocal-CoreAI

Core AI is Apple's on-device ML runtime in iOS 27 / macOS 27 and the successor to Core ML: PyTorch models are exported with Apple's coreai-torch (LLMs: coreai.llm.export) into .aimodel bundles that run on the GPU or the Neural Engine, e.g. Qwen3-8B 4-bit decodes at 94 tok/s on an M4 Max GPU, MLX 90 under the same protocol (apple-silicon-llm-bench, macOS 27 beta 26A5353q, 2026-06-11).

<!-- gen-cards:devicemark begin (managed by scripts/gen-cards + tools/devicemark_row.py — edit cards.json, not this block) -->

This model has no row on DeviceMark, the on-device LLM leaderboard.

<!-- gen-cards:devicemark end -->

Mel-Band RoFormer (Kim Vocal) — Core AI

KimberleyJSN/melbandroformer (MIT, ~228 M) converted to Apple Core AI — the zoo's first source-separation model. Split any song into a vocals (acapella) stem and an instrumental (karaoke) stem, entirely on device: iPhone (AOT) and Mac.

Mel-Band RoFormer is STFT -> band-split (mel, overlapping) -> axial rotary transformer x6 -> mask estimator -> band-average -> complex mask multiply -> iSTFT. The neural core lowers to Core AI directly (no rewrite), and two moves keep the on-device host trivial:

• Real-arithmetic core — the band-average scatter becomes a constant matmul and the complex mask multiply becomes real ops, so the graph carries no complex tensors and no scatter_add.
• STFT/iSTFT folded into the graph as constant DFT matmuls (window baked in). The shipped graph is frames[1,2,801,2048] -> recon[1,2,801,2048]; the host only does reflect-pad, framing and overlap-add — no FFT, no vDSP packing.

Fixed shapes throughout (8 s chunk = 352 800 samples @ 44.1 kHz, 801 STFT frames), so there are no dynamic-shape recompiles. The instrumental stem is mix - vocals.

Contents

• mbr_full_fp16.aimodel — macOS bundle (fp16, ~470 MB).
• mbr_full_fp16.h18p.aimodelc — iOS AOT specialization (A19 / h18p, GPU).
• metadata.json — sample rate, chunk size, STFT parameters, graph I/O, host recipe.
• golden_raw.f32 / golden_vocals.f32 — an 8 s demo chunk and its expected vocals stem (stereo, channel-major, float32) for a host-side self-test.

Host recipe

Reflect-pad the chunk by n_fft // 2, frame with hop_length, feed frames[1,2,801,2048], then overlap-add the output, divide by the summed squared window and trim the pad. Chunks are 8 s with num_overlap crossfade; instrumental = mix - vocals.

Gates

On device (iPhone 17 Pro, GPU): 8 s chunk in 1.23 s ~ 6.5x real-time warm (load 0.57 s); the cold first run is 3.82 s ~ 2.1x real-time (load 1.24 s), before the GPU clocks ramp.

Use it

import CoreAIKit

let separator = try await KitSeparator(catalog: "melband-roformer-vocal")
let stems = try await separator.separate(contentsOf: songURL)
// stems.vocals / stems.instrumental — [channel][sample] at 44.1 kHz

Ships in the zoo's coreai-audio app (Separate tab), which pairs it with the Music tab (Stable Audio Open Small): generate a track, then rip its stems. Conversion recipe and the Swift host reference: conversion/melband_roformer.

Attribution

Mel-Band RoFormer (Ju-Chiang Wang, Wei-Tsung Lu, Minz Won — ByteDance AI Labs); checkpoint by KimberleyJensen; lucidrains BS-RoFormer implementation; ZFTurbo training code. MIT.

Community port — not an Apple model.

<!-- funnel:v1 -->

---

More models in this format: Core AI Model Zoo — 75 models, each with the recipe that produced it.

Want a different model on-device? Open a request — free, open weights only; the export and its measured numbers get published publicly.

<!-- /funnel:v1 -->