Results
Prizes
The prizes for the MBSTOI evaluation were awarded to the Brno University of Technology team (first place), the Music Tribe team (second place) and the ELO-SPHERES Consortium (third place). The University of Sheffield team was not eligible to win prizes due to connections with the organisers.
The overall prizes for the listening test evaluation were awarded to the University of Oldenburg team (first place) and the ELO-SPHERES Consortium (second place). In the noise interferer category, the University of Oldenburg and Googlears teams were the best performers, while in the speech interferer category, the University of Oldenburg team and ELO-SPHERES Consortium were the best performers. For a discussion of the differences in performance for the noise and speech interferers, see the [YouTube video].
The Amazon student prizes were awarded to Katerina Zmolikova from the Brno University of Technology team (first place), Tomas Gajecki from the Medical University Hannover team (second place), and Xi Chen from Shenzhen University, who was an intern at Tencent while working on this challenge (third place). Congratulations to the winners!
Results
The tables below present median results on the evaluation dataset for the initial MBSTOI evaluation for noise (eval-noise) and speech (eval-speech) interferers, and the Automatic Speech Recognition (ASR) derived listening test results for 27 hearing-impaired listeners for noise (eval-noise) and speech (eval-speech) interferers.
MBSTOI evaluation
Rank | Team | Paper | Eval-noise | Eval-speech |
---|---|---|---|---|
Univ. Sheffield (E009) | [paper] | 0.693 | 0.741 | |
Brno Univ. Technology (E007) | [paper] | 0.678 | 0.715 | |
Univ. Sheffield (E002) | [paper] | 0.672 | 0.704 | |
Music Tribe (E010) | [paper] | 0.639 | 0.701 | |
ELO-SPHERES Consortium (E013) | [paper] | 0.653 | 0.676 | |
Googlears (E003) | [paper] | 0.632 | 0.670 | |
Univ. Oldenburg (E021) | [paper] | 0.611 | 0.636 | |
Univ. Oldenburg (E019) | [paper] | 0.607 | 0.634 | |
Univ. Oldenburg (E016) | [paper] | 0.603 | 0.628 | |
Tencent/Shenzhen Univ. (E004) | [paper] | 0.558 | 0.553 | |
Hannover Medical School (E005) | [paper] | 0.481 | 0.549 | |
Tencent/Shenzhen Univ. (E018) | [paper] | 0.524 | 0.521 | |
Baseline (E001) | [paper] | 0.282 | 0.335 |
[TOP]
Listening test evaluation
Rank | Team | Paper | Eval-noise | Eval-speech |
---|---|---|---|---|
Univ. Sheffield (E009) | [paper] | 80.426 | 82.432 | |
Brno Univ. Technology (E007) | [paper] | 75.424 | 81.498 | |
Music Tribe (E010) | [paper] | 72.222 | 77.778 | |
ELO-SPHERES Consortium (E013) | [paper] | 83.613 | 82.895 | |
Googlears (E003) | [paper] | 85.532 | 4.444 | |
Univ. Oldenburg (E021) | [paper] | 84.914 | 83.929 | |
Univ. Oldenburg (E019) | [paper] | 86.726 | 86.885 | |
Hannover Medical School* (E005) | [paper] | |||
Tencent/Shenzhen Univ. (E018) | [paper] | 60.593 | 44.681 | |
Baseline | [paper] | 33.202 | 51.152 |
[TOP]