Speaker Overlap-aware Neural Diarization for Multi-party Meeting Analysis

Zhihao Du, ShiLiang Zhang, Siqi Zheng, Zhi-Jie Yan


Abstract
Recently, hybrid systems of clustering and neural diarization models have been successfully applied in multi-party meeting analysis. However, current models always treat overlapped speaker diarization as a multi-label classification problem, where speaker dependency and overlaps are not well considered. To overcome the disadvantages, we reformulate overlapped speaker diarization task as a single-label prediction problem via the proposed power set encoding (PSE). Through this formulation, speaker dependency and overlaps can be explicitly modeled. To fully leverage this formulation, we further propose the speaker overlap-aware neural diarization (SOND) model, which consists of a context-independent (CI) scorer to model global speaker discriminability, a context-dependent scorer (CD) to model local discriminability, and a speaker combining network (SCN) to combine and reassign speaker activities. Experimental results show that using the proposed formulation can outperform the state-of-the-art methods based on target speaker voice activity detection, and the performance can be further improved with SOND, resulting in a 6.30% relative diarization error reduction.
Anthology ID:
2022.emnlp-main.505
Volume:
Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing
Month:
December
Year:
2022
Address:
Abu Dhabi, United Arab Emirates
Editors:
Yoav Goldberg, Zornitsa Kozareva, Yue Zhang
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
7458–7469
Language:
URL:
https://aclanthology.org/2022.emnlp-main.505
DOI:
10.18653/v1/2022.emnlp-main.505
Bibkey:
Cite (ACL):
Zhihao Du, ShiLiang Zhang, Siqi Zheng, and Zhi-Jie Yan. 2022. Speaker Overlap-aware Neural Diarization for Multi-party Meeting Analysis. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 7458–7469, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
Cite (Informal):
Speaker Overlap-aware Neural Diarization for Multi-party Meeting Analysis (Du et al., EMNLP 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.emnlp-main.505.pdf