Luigi/primeminutes-speaker-litert

CAM++ zh-en speaker embedding for LiteRT (Prime Minutes)

A fixed-shape LiteRT (.tflite) export of 3D-Speaker CAM++ zh-en common-advanced (ModelScope iic/speech_campplus_sv_zh_en_16k-common_advanced, checkpoint campplus_cn_en_common.pt, sha256 92f29b94e6948786a26778c9e302525d185bb08c8b9f5252ed98776902840199). It is used by the Prime Minutes app to suggest known speaker names across meetings, on the phone's CPU only.

Licence and attribution

• Model weights: Apache License 2.0, as stated on the ModelScope model card (license: Apache License 2.0).
• Model code (CAM++ definition, speakerlab/models/campplus): 3D-Speaker, "released under the Apache License 2.0".
• Paper: Hui Wang, Siqi Zheng, Yafeng Chen, Luyao Cheng, Qian Chen, CAM++: A Fast and Efficient Network for Speaker Verification Using Context-Aware Masking, arXiv:2303.00332.

This repository only converts the published weights (no fine-tuning). Changes: the graph is traced for a fixed input length, and the context-aware masks' segment pooling is written as static slices (identical result at this length).

Files

File	Size	sha256
campplus_zh_en_400f.tflite (fp32)	28,800,672 B	fe33cb21d09b8c5d21d28634c12eea18661a5088fb2a8bbed3818c4876da8b75

Contract

• Input serving_default_args_0: float32 [1, 400, 80] — 400 frames of 80-bin Kaldi fbank, i.e. one window of 64,240 samples (4.015 s) of 16 kHz mono audio, samples in [-1, 1] (no int16 scaling). Fbank = torchaudio.compliance.kaldi.fbank(wav, num_mel_bins=80, sample_frequency=16000, dither=0.0) defaults (25 ms Povey window, 10 ms shift, pre-emphasis 0.97, DC removal, power spectrum, log, low 20 Hz, high = Nyquist, snip_edges = true): the front-end 3D-Speaker trained with.
• Normalisation (done by the caller): subtract the per-bin mean over the window's 400 frames.
• Output serving_default_output_0_output: float32 [1, 192], not normalised. L2-normalise; compare with cosine.

The app's speaker vector: clean single-speaker speech (≤ 60 s, ≥ 10 s) concatenated, cut into 400-frame windows (a tail ≥ 2 s is the stream's last full window, weighted by its new seconds), seconds-weighted mean of the L2-normalised window embeddings, L2-normalised.

Validation (host)

• tflite vs the sherpa-onnx ONNX of the same checkpoint, identical features: cosine 1.000000 (fp32).
• App C++ fbank vs torchaudio.compliance.kaldi.fbank: max |Δ| 1.1e-3 on log-mel (mean 7.9e-6).
• AMI meeting series ES2004/IS1009/TS3003/EN2002 (16 meetings, minutes 2–12, app diarizer segmentation, window protocol above): far field (one table mic) EER 0.3 % (within-series 1.5 %), close talk EER 0.1 % (1.3 %); identification from one earlier meeting top-1 100 % (31/31), progressive 100 %. Different-person scores max 0.57–0.58, same-person p5 0.66–0.70.
• AISHELL-4 (Mandarin far field, same-session proxy): same-person p5 0.74, different-person p95 0.50 / max 0.67.

Suggested thresholds for ranking known people: T_low = 0.50 ("possible"), T_high = 0.70 ("likely", above every different-person score measured), margin 0.05. Calibrated on favourable data: keep suggestions, never auto-apply.

Export

tools/speaker-id-eval/export_litert.py in the Prime Minutes repository (litert-torch 0.9.1, torch, 3D-Speaker code).