Nemotron-3-diarization: Here's What You Need to Know

Nemotron-3-Diarization is an open-weight speaker diarization model developed by NVIDIA. It identifies who spoke when in real-world audio using a single checkpoint that supports both offline processing and streaming operation with up to eight speakers. The model's key features include configurable output resolution, chunked inference that removes a maximum-duration limit, and a streaming design that preserves speaker identities across chunks through Arrival-Order Speaker Cache (AOSC) and First-In-First-Out (FIFO) queue mechanisms.

The model is built on Sortformer and orders output channels by each speaker's first arrival, effectively resolving the speaker-permutation problem without requiring fixed speaker identities in advance. It is compatible with the NVIDIA NeMo Speech framework and provides a lightweight native C++ runtime. The model is licensed under openmdw-1.1, which allows both commercial and non-commercial use.

Nemotron-3-Diarization is well-suited for real-time meeting diarization, long-form offline transcription with speaker labels, and word-level speaker-attributed transcripts. It can handle recordings with up to eight speakers, making it suitable for larger meetings and panel discussions. Researchers can customize latency and run evaluation studies using the provided example script.

However, the model has limitations, such as a maximum of eight speakers, which means it may not be suitable for sessions requiring more than eight simultaneous or represented speaker channels. The latency figures provided only describe input-buffer latency and do not account for model computation time, transfer, audio decoding, or application overhead.

Additionally, the model does not provide details on denominator error rates (DER), benchmark datasets, real-time factors, throughput, memory use, or GPU requirements. Speaker indices represent arrival order, not persistent identities like "Alice" or "Bob," so additional identity-mapping layers may be necessary for named identities.

In comparison, Nemotron-3-Diarization-preview refers to the earlier or preview release of the model, while diar_streaming_sortformer_4spk-v2.1 is an alternative model designed for conversations requiring up to four speakers and a single checkpoint for offline-style processing plus several streaming latency configurations. Researchers should evaluate both models based on their specific requirements, considering factors such as release maturity, feature clarity, and potential tradeoffs in accuracy and performance.