The 1st Clarity Prediction Challenge (CPC1)

Clarity challenge code for the 1st prediction challenge (CPC1).

There are two traks in CPC1:

closed-set: the evaluation systems and listeners are covered in the training set, the signals and listener repsonses are provieded in CPC1.train.json
open-set: the evaluation systems and listeners are not seen in the training set, the signals and listener responses are provieded in CPC1_indep.train.json

For more information about the CPC1 please visit claritychallenge.org/docs/cpc1/cpc1_intro.

Data structure

To download the CPC1 data, please visit here.

clarity_CPC1_data.v1_1 contains the training data:

clarity_data
|
└───HA_outputs
|   |
|   └───train 3.8G
|   |
|   └───train_indep 2.8G
|
└───scenes 12.1G

metadata
    |CPC1.train.json
    |CPC1.train_indep.json
    |listener_data.CPC1_train.xlsx
    listeners.CPC1_train.json
    |scenes.CPC1_train.json

clarity_CPC1_data.test.v1 follows the same structure as clarity_CPC1_data.v1_1, except that the listener responses (i.e. test labels) are not included. The test listener responses are in the test_listener_responses.

Baseline

The baseline folder provides the code of the Cambridge Auditory Group MSBG hearing loss model and MBSTOI, see CEC1. Run run.py to generate the predicted intelligibility, and then run compute_scores.py to apply logistic fitting and compute the evaluation scores, including RMSE, normalised cross-correlation, Kendall’s Tau coefficient.

Citing CPC1

@inproceedings{barker2022the,
  title={The 1st Clarity Prediction Challenge: A machine learning challenge for hearing aid intelligibility prediction},
  author={Jon Barker, Michael Akeroyd, Trevor J. Cox, John F. Culling, Jennifer Firth, Simone Graetzer, Holly Griffiths, Lara Harris, Graham Naylor, Zuzanna Podwinska, Eszter Porter and Rhoddy Viveros Munoz},
  year={2022}
}