Skip to main content

ICASSP 2023 Submission

info

Submissions will be evaluated using a combination of HASPI and HASQI to give a combined speech intelligibility and speech quality metric.

Key dates.

  • 2nd Feb 2023: Release of evaluation data.
  • 10th Feb 2023: Submission by teams.
  • 14th Feb 2023: Results released.Top 5 ranked teams invited to submit papers to ICASSP-2023
  • 20th Feb 2023: Invited papers submitted to ICASSP-2023
  • 4-9th June 2023: Overview paper and invited papers presented at dedicated ICASSP session

What evaluation data is provided?​

There will be two sets of evaluation data: i) the simulate set consisting of 1500 scenes generated in the same way as the training and development data (eval1), ii) the real data consisting of real acoustic mixtures (eval2). For details see the data description page.

For each scene, you are provided with the signals received at each of the three microphones on the left and right hearing aid device. You will also be provided with JSON or csv formatted metadata consisting of

  1. the audiograms for a set of listeners and
  2. a mapping of which listeners will listen to which scenes.

There will also be some clean example utterances from the target talker, that are not the same as the target utterance, but which can be used to identify the target talker, i.e., to disambiguate scenes in which other speakers are present.

For HASPI/HASQI evaluation, there will be one listener per scene and the scene-listener mapping will be the same for all teams.

What audio do I need to submit?​

You must submit the stereo audio signals produced at the output of your enhancement stage, which the organisers will process by the hearing aid amplification stage and the HASPI/HASQI evaluation metric. Signals should be submitted as stereo, floating point wav format signals, at the same sampling rate as the signals provided

Naming and packaging signals​

Your processed signals should be named using the conventions used by the baseline system, i.e., <Scene ID>_<Listener ID>_enhanced.wav and explained on the data page.

Place the processed signals for the two sets into separate directories named eval1 and eval2.

These should be placed in a directory whose name is the unique team ID that you will be sent, e.g., ICASSP2023_E001 and then packaged using zip or tar or any standard packaging tool, e.g., to make a packaged file called <TEAM_ID>.zip

The packaged file will have the following structure,

ICASSP2023_E001
├── eval1 (1500 processed signals)
└── eval2 (1500 processed signals)

The resulting file should be about 4 GB.

Upload the packaged data to the Google Drive link that you will have been sent.

Using head rotation data and/or extended training data​

We would like to be able to separately evaluate the benefit of using the head rotation and extra training data, so in accordance with the challenge rules,

  • If you have trained on data which was not included in the core database, then please also provide outputs of a system trained only with the standard data.

  • If you have made use of the head rotation data you should also provide outputs of an equivalent system that does not use the head rotation data.

If you have used extra training data and/or the head rotation data, then please package the outputs separately using the following naming convention,

‹TEAM_ID›.zip - standard training data and no head rotation (all teams)

‹TEAM_ID›_hr.zip - standard data and using head rotation

‹TEAM_ID›_data.zip - extended training data without using head rotation

‹TEAM_ID›_hr_data.zip - extended training data and using head rotation

Technical report​

  • For every entry, a technical report needs to be uploaded to the Google Drive along with your evaluation signals - see here for deadline. The draft needs to be sufficiently complete for us to judge whether your system is compliant with the challenge rules.
  • Your report should include an abstract and introduction and sections on experimental setup/methodology including system information and model/network architecture, evaluation/results, discussion, conclusion and references. Please provide an estimation of the computational resources needed. You must describe any external data and pre-existing tools, software and models used.
  • The report can be placed in the Google Drive alongside your data.
  • The top five systems will be invited to submit a paper to the ICASSP 2023 special session.

How will intellectual property be handled?​

See here under Intellectual Property.

Where do I submit the signals?​

When you have registered you will receive a link to a Google Drive to which you will be able to securely upload your signals. We also encourage you to submit your enhancement code via this link.

Materials uploaded will be visible to the Clarity Team but not to other entrants.

warning

Note, in order to use the Google Drive you will need to have a Google account. If you anticipate problems using Google then please make arrangements to send us the materials by other means, e.g., via a service such as WeTransfer or similar.