Results
The 1st Clarity Prediction Challenge is now complete. Results are shown below along with links to the system description papers. Further information can be found on the Clarity-2022 workshop website here.
The table below reports both the RMSE and the correlation coefficient (CC) between the predicted and actual intelligibility scores. Results are shown separately for the closed (systems seen during training) and open evaluation sets (systems not seen during training).The column 'intrusive' indicates whether the system is intrusive (i.e., requires access to the clean speech signal) or not. Links to the technical reports are also provided.
| Rank | Team | ID | Paper | Intrusive | RMSE (open) | Corr (open) | RMSE (closed) | Corr (closed) |
|---|---|---|---|---|---|---|---|---|
| 1 | T01 | E30 | Link | Yes | 22.5 | 0.79 | ||
| 2 | T02 | E32 | Link | Yes | 23.5 | 0.76 | 23.1 | 0.77 |
| 3 | T02 | E29 | Link | No | 24.6 | 0.73 | 23.3 | 0.77 |
| 4 | T03 | E36 | Link | Yes | 29.2 | 0.6 | 24 | 0.76 |
| 5 | T04 | E33 | Link | No | 28.9 | 0.65 | 24.1 | 0.75 |
| 6 | T04 | E16 | Link | No | 30.7 | 0.59 | 24.7 | 0.74 |
| 7 | T05 | E22 | Link | No | 32.1 | 0.54 | 25.9 | 0.7 |
| 8 | T06 | E19 | Link | Yes | 28.1 | 0.63 | 27.5 | 0.66 |
| 9 | Baseline | Yes | 36.5 | 0.53 | 28.5 | 0.62 | ||
| 10 | T07 | E06 | Link | No | 32 | 0.5 | ||
| 11 | T07 | E34 | Link | No | 33.4 | 0.43 | ||
| 12 | T08 | E35 | Link | No | 35.7 | 0.22 | 35.4 | 0.25 |
| 13 | Prior | No | 36.2 | 36.4 | ||||
| 14 | T09 | E31 | Link | Yes | 28.3 | 0.67 | 37.2 | 0.41 |
| 15 | T08 | E23 | Link | No | 43.7 | 0.05 | 41.5 | 0.07 |
| 16 | T10 | E02 | Link | Yes | 35.2 | 0.38 | ||
| 17 | T10 | E38 | Link | Yes | 49.7 | 0.3 |
Prizes
The Hearing Industry Research Consortium prizes for best system were awarded as follows:
- 1st place: System E33, Zezario et al, MBI-Net: A Non-Intrusive Multi-Branched Speech Intelligibility Prediction Model for Hearing Aids for best performance for a non-intrusive system,
- 2nd place: System E30, Huckvale et al, ELO-SPHERES Intelligibility Prediction Model for the Clarity Prediction Challenge 2022 for best performance of an intrusive system,
- 3rd place: System E019, Röttges et al., Speech Intelligibility Prediction for Hearing-Impaired Listeners with the bBSIM-STI Model for best performance in the open-set track
The Hearing Industry Research Consortium prizes for best student contribution were awarded as follows:
- 1st place: Ryandhimas Zezario from the National Taiwan University,
- 2nd place: Franklin Yohan Alvarez Cardinale from Medizinische Hochschule Hannover, Germany
- 3rd place: Jana Rossbach from Carl von Ossietzky University, Oldenburg, Germany.
Congratulations to the winners!
(Note, systems E29 and E32 had links to the organizers and so were not eligible for prizes).