nemotron-ocr-v2-coreml

Nemotron OCR v2 CoreML

CoreML conversion of the English neural stages from NVIDIA Nemotron OCR v2.

SwiftPM package: github.com/mweinbach/OCRCoreML

Included Models

The recognizer emits transformer features; those are required by the relational model, so this bundle covers the full neural OCR pipeline rather than detector-only inference.

Pipeline Boundary

The original Python package uses CUDA/C++ helpers for the non-neural stages: rotated-box NMS, rboxes to quads, quad rectification, feature-map grid sampling, sequence decoding, relation-graph decoding, and reading-order formatting. Those operations are not CoreML models. Apple apps integrating this bundle must port or replace those post-processing steps.

The linked SwiftPM package includes wrappers for all three CoreML models and a greedy recognizer decoder. It exposes raw tensors rather than claiming complete image-to-text OCR until the geometric and graph post-processing is ported.

Files

Performance

Local median latencies after warmup:

GPU/CoreML ALL is the best single-shot latency path on the test machine. CPU+ANE is useful when GPU time needs to be reserved for rendering or other workloads.

Swift Usage

import OCRCoreML

let pipeline = try OCRPipeline(computeUnits: .cpuAndGPU)
let detectorPrediction = try pipeline.detect(image: cgImage)

let recognizerPrediction = try pipeline.recognize(regions: regions)
let decoded = try pipeline.recognizer.decode(
    logits: recognizerPrediction.output.logits,
    count: detectedRegionCount
)

let relationalPrediction = try pipeline.relate(
    rectifiedQuads: relationalRegionFeatures,
    originalQuads: originalQuads,
    recognizerFeatures: recognizerPrediction.output.features,
    numValid: detectedRegionCount
)

See the SwiftPM docs for exact app integration notes: https://github.com/mweinbach/OCRCoreML

License

The converted model weights inherit the NVIDIA Open Model License Agreement. The upstream source code and helper scripts are Apache 2.0. See LICENSE and NOTICE.