mlboydaisuke/VoxCPM2-CoreAI

VoxCPM2 2B — Core AI (on-device, 48 kHz)

OpenBMB VoxCPM2 (2B) converted to Apple Core AI, running fully on-device on iPhone (A19 Pro / iPhone 17 Pro) and Mac — no network. The 2B, 48 kHz successor to VoxCPM-0.5B-CoreAI.

A tokenizer-free diffusion TTS: a MiniCPM4 28-layer text-semantic LM + an 8-layer residual acoustic LM drive a 12-layer LocDiT flow-matching diffusion head, decoded by a 48 kHz AudioVAE. Five Core AI bundles + a few host-side projections.

What's inside

dir	contents
macos/	JIT .aimodel bundles (Mac): int8 base/res decode + prefill, fp16 feat_decoder / feat_encoder / vocoder
ios/	AOT .aimodelc bundles (iOS h18p, GPU): same five + the two int8 prefill bundles
voxcpm2_host_glue/	embed table + projections / FSQ-512 / stop-head / fusion (.bin + manifest)
tokenizer/	the VoxCPM2 tokenizer (Llama fast)

The backbone LMs are weight-only int8 (the size driver); the diffusion + VAE stay fp16 (the continuous-feedback path is quant-sensitive — same split mlx-community uses).

On-device numbers (iPhone 17 Pro, int8 + prefill + streaming)

• RTF 1.19, first-audio 0.65 s, 48 kHz, ~4.9 GB resident (increased-memory entitlement).
• Streaming starts after the first ~0.65 s; the 2B is ~4× the 0.5B, so RTF sits just above realtime.

Use it

Runs through coreai-kit VoxCPM2TTS, wired into the coreai-model-zoo coreai-audio app ("Voice 2B" tab). Conversion + gates + export scripts: coreai-model-zoo/conversion/voxcpm/ (*_v2.py).

let tts = try await VoxCPM2TTS(paths: .standard(artifactsRoot: root, lm: .int8))
let wav = try await tts.synthesize("On device speech synthesis, running entirely on your iPhone.") // 48 kHz Float PCM

Verification

Reimplemented in exportable Core AI overlays and gated end-to-end against the official model: backbone / feat_decoder / feat_encoder cos 1.0, full chain magspec 0.996; every exported bundle engine-gated cos ≥ 0.9999.

License

Apache-2.0 (commercial OK), inherited from openbmb/VoxCPM2. Not affiliated with OpenBMB or Apple. Community port.