Rules

Pre-registration form

If your team is likely to enter the first enhancement challenge and you are the contact person for your team, please pre-register via this form.

Teams

  • Teams must pre-register and nominate a contact person via the form above.
  • Teams can be from one or more institutions.
  • The organisers may enter the challenge themselves but will not be eligible to win the cash prizes.

Transparency

  • Teams must provide a technical document of up to 2 pages describing the system/model and any external data and pre-existing tools, software and models used.
  • We will publish all technical documents (anonymous or otherwise).
  • Teams are encouraged – but not required – to provide us with access to the system/model and to make their code open source.
  • Anonymous entries are allowed but will not be eligible for cash prizes.
  • All teams will be referred to using anonymous codenames in rank ordering.

What information can I use?

Training and development

  • There is no limit on the amount of training data that can be generated using our tools.
  • Teams can also use their own data for training or expand the training data through simple automated modifications. However, teams that do this must make a second submission using only the official audio files and signal generation tool.
  • Any audio or metadata can be used during training and development, but during evaluation the proposed simulated hearing aid or Enhancement Processor will not have access to all of the data (see next section).

Evaluation

The only data that can be used by the Enhancement Processor during evaluation are:

  • The audio input signals
  • The listener characterisation (pure tone air-conduction audiograms)

Computation restrictions

  • Systems must be causal; the output at time t must not use any information from input samples more than 5 ms into the future (i.e., no information from input samples >t+5ms).
  • There is no limit on computational cost.
  • Teams may choose to use all or some of the provided baseline models.

Evaluation of systems

Stage 1: Objective evaluation

  • Entries will be ranked according to mean Modified Binaural Short-Time Objective Intelligibility (MBSTOI) score across all samples in the evaluation test set.

Stage 2: Listening test evaluation

  • There is a limit on how many systems can be evaluated by the listener panel.
  • A maximum of two entries can go through to the listener panel from any individual entrant. Furthermore, a second will only be allowed if it is judged by us to use significantly different signal processing approaches.
  • We will choose which will go to the listener panel based on:
    • The top N scored using the objective evaluation method.
    • A sample of M others that use contrasting and promising approaches.

Prizes

trophy
  • $1000, $500, and $250 for 1st, 2nd and 3rd.
  • Prizes for both stages of the evaluation: MBSTOI and listening panel evaluation.
  • Anonymous entries and those with direct links to the Clarity project team are ineligible for cash prizes, sorry.

Any questions?

Please post questions on the Google Group discussions board.