Submission
Registration
Teams are required to register to help us organise the challenge. Registered teams will be assigned a unique team ID.
What evaluation data is provided?
The evaluation data consists of audio signals processed by hearing aid systems, clean reference signals, listener metadata, and a mapping of which listeners listened to which scenes/hearing aid systems.
The evaluation data is available for download here clarity_CPC1_data.test.v1.tgz. See the download page for more details.
Note, the evaluation data does not contain the listener responses. We will score your submission for you and return your score (we aim to do this within 24 hours for of submission). We will then release the true listener responses the day after the submission deadline to allow teams to perform analysis of their results.
What do I need to submit?
All teams must submit
- Their predicted intelligibility scores
- An Interspeech paper describing their work (encouraged)
- A two page technical report (mandatory)
The predicted intellgibility scores
You must submit your predicted intelligibility scores for the signals provided.
The predictions should be sent in CSV format files with two columns:
signal_ID, intelligibility_score
Where the signal_ID is the unique signal identifier used for the wav file name (e.g., S08510_L0239_E001) and intelligibility_score is the predicted intelligibilty given in terms of the percentage words recognised correctly for the signal (i.e., for 0 to 100).
Your CSV files should be named as follows CPC1_<TEAM_ID>.test.csv and CPC1_<TEAM_ID>.test_indep.csv for closed set and open set evaluations respectively, where <TEAM_ID> is your individual team ID, e.g. 'E001'.
The files should be sent as email attachments to the email address: claritychallengecontact@gmail.com
Please use "CPC1 Submission <TEAM_ID>" as the subject line.
We also encourage you to submit your prediction model(s) code.
info
All registered teams will be emailed with their unique team ID shortly before the submission deadline. If you plan to submit please register before the submission deadline.
Interspeech paper submission
All teams are strongly encouraged to submit a paper describing their work to the Interspeech 2022 Special Session "Speech Intelligibility Prediction for Hearing-Impaired Listeners".
Interspeech submission instructions are here https://interspeech2022.org/forauthor/submissions.php
The Interspeech papers need to be initially submitted by March 21st (title and abstract), with the full paper due on March 28th.
The technical report
The two page technical report must be submitted as a paper to the Clarity-CPC1-2022 Workshop. Deadline 25th April 2022. An author kit and submission instructions will be made available.
A draft of the report needs to be submitted along with your predictions by 21st March. The draft needs to be sufficiently complete for us to judge whether your system(s)/model(s) is compliant with the challenge rules. You can find a list of key challenge dates here.
Your report should include an abstract and introduction and sections on experimental setup/methodology including system/model information and model/network architecture, evaluation/results, discussion, conclusion and references. Please provide an estimation of the computational resources needed. You must describe any external data and pre-existing tools, software and models used. Please make it clear how your system(s)/model(s) meet the challenge rules.
Your report should cite the following document, which provides an overview of the challenge and the baseline system:
Jon Barker and Michael Akeroyd and Trevor J. Cox and John F. Culling and Jennifer Firth and Simone Graetzer and Holly Griffiths and Lara Harris and Graham Naylor and Zuzanna Podwinska and Eszter Porter and Rhoddy Viveros Munoz, “The 1st Clarity Prediction Challenge: A machine learning challenge for hearing aid intelligibility prediction,” in Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH 2022, Incheon, South Korea, 2022.
The document can be accessed here.
How will intellectual property be handled?
See here under Intellectual Property.