Results
The 2nd Clarity Prediction Challenge is now complete. Results are shown below along with links to the system description papers. Further information can be found on the Clarity-2023 workshop website here.
The table below reports both the RMSE and the correlation coefficient (CC) between the predicted and actual intelligibility scores. The column 'intrusive' indicates whether the system is intrusive (i.e., requires access to the clean speech signal) or not.
| Rank | Team | ID | Paper | Intrusive | RMSE | Corr |
|---|---|---|---|---|---|---|
| 1 | T005 | E011 | Link | No | 25.1 | 0.78 |
| 2 | T002 | E002 | Link | No | 25.3 | 0.77 |
| 3 | T004 | E009 | Link | Yes | 25.4 | 0.78 |
| 4 | T010 | E022 | Link | Yes | 25.7 | 0.77 |
| 5 | T006 | E023 | Link | No | 26.4 | 0.76 |
| 6 | T006 | E016 | Link | No | 26.8 | 0.75 |
| 7 | T010 | E025 | Link | No | 27.9 | 0.72 |
| 8 | beHASPI | Yes | 28.7 | 0.7 | ||
| 9 | T003 | E003 | Link | No | 31.1 | 0.64 |
| 10 | T003 | E024 | Link | No | 31.7 | 0.62 |
| 11 | T007 | E015 | Link | No | 35 | 0.6 |
| 12 | T009 | E020 | Link | No | 39.8 | 0.33 |
| 13 | Prior | No | 40 |
Prizes
The Hearing Industry Research Consortium best system prizes were awarded as follows:
- 1st place: Cuervo and Marxer, Temporal-heirarchical features from noise-robust speech foundation models for non-intrusive intelligibility prediction
- 2nd place: Mogridge, Close, Sutherland, Goetze and Ragni, Pre-training intermediate ASR features and Human memory simulation for non-intrusive speech intelligibility prediction in the Clarity Prediction Challenge 2
(Note, although 1st and 2nd place systems had very similar RMSE scores, a paired t-test showed that the difference was highly significant).
Congratulations to the winners!