AI agents write Rust GPU kernels to make speakrs 60x faster than pyannote Other

Developer Praveen Perera migrated the open-source speaker diarization library speakrs from ONNX Runtime to native GPU kernels written in Rust with NVIDIA cuda-oxide, accelerating diarization by up to 60x compared to pyannote with zero loss in accuracy. AI coding agents authored the majority of the custom GPU kernels.

The project initially focused on removing heavy native dependencies by replacing ONNX Runtime with a lightweight CUDA backend. After early custom kernels beat vendor libraries, the migration expanded to replace cuDNN and cuBLAS entirely, reducing deployment Docker container sizes by 2 GB.

What it is

• Speech diarization engine. speakrs runs pyannote's community-1 pipeline (segmentation, speaker embeddings, PLDA, and VBx clustering) natively in Rust.
• Significant speedups. On an NVIDIA RTX 4090, processing the VoxConverse dev benchmark dropped from 1,236 seconds to 38 seconds (32.4x speedup), while Earnings-21 dropped from 2,692 seconds to 75 seconds (35.8x speedup).
• Preserved accuracy. Diarization Error Rate (DER) remained unchanged at 7.0%, compared to pyannote's 7.2%.
• Container slimming. Removing cuDNN 9 (~1.2 GB) and cuBLAS 12 (~0.8 GB) cut approximately 2 GB from deployment GPU container images, requiring only the base NVIDIA driver.
• Agent-driven implementation. AI agents drafted the custom SIMT kernels in Rust. Perera guided architecture, established A/B validation harnesses, and rejected fragile micro-optimizations in favor of simple end-to-end tests.
• Practical trade-offs. Perera noted that custom kernels required per-GPU tuning for architectures like A100, T4, and RTX 5090, confirming that replacing mature vendor libraries requires weighing maintenance overhead against speed gains.

Sources

• Avencera: Writing GPU kernels in Rust: how I made speakrs 60x faster than pyannote
• Avencera official site