Listening Tests
Overview#
Our panel members will be provided with a tablet preloaded with our Listen@Home software and headphones to complete the listening experiment.
They will listen to an entrant’s sentence, respond verbally with what they think was said by the target talker, and then move on to the next sentence. Their response will be recorded by the tablet microphone(s) and then processed using automatic speech recognition. Intelligibility will be evaluated as the number of words identified correctly in the sentence.
Our plan is that each listener will evaluate 1,200 sentences, which is about 4 hours of listening, and that every listener will evaluate sentences from every entrant. We will use a combinatorial design to equate this as far as possible.
Should a listener drop out from the panel, we will endeavour to replace them with someone with a similar hearing loss, but should that prove impractical we will reduce the size of the panel, and inform entrants which listener has withdrawn.
Listen@Home hardware#
We will be using a Lenovo 10e Chromebook running Android 81.0 and Sennheiser PC-8 headsets to play the sounds to our participants. We will allow participants to set the volume so that the sounds are not so loud to be uncomfortable. Without loudness-recruitment measures for our listeners, we cannot be sure just what loudnesses every participant will hear, so we need to allow them to make the choice here.
We have measurements on the output capability of a system in the laboratory:
- A 1 kHz pure tone set to be the most powerful it can be (i.e., an amplitude range of +/-1 = RMS amplitude of 0.707, and the volume controls at 100%) gave 99 dB(A) SPL on the PC-8 headphones.
- An ICRA speech-shaped noise [1], unmodulated in time, and scaled to an RMS of 0.3, gave 90 dB(A) at the same volume level. With this RMS, the noise had 0.1% of its samples clipped at +/- 1.
It is important to note that there is a convention for the prediction model that a +/-1 square wave has RMS = 0 dB FS and corresponds to 120 dB, while for listening tests, 0 dB FS corresponds to approximately 100 dB, given the above capabilities of the reproduction equipment.
For the listening tests, we will require the signals to be provided as 16-bit WAV files with a 32 kHz sampling rate (see this page). We will play the signals as is using a HTML/PHP audio player coded on a webpage. The responsibility for the final signal level is therefore yours. It’s worth bearing in mind that should your signals overall seem too loud to be comfortable to a participant, they may well turn down the volume themselves.
References#
[1] ICRA standard noises, https://icra-audiology.org/Repository/icra-noise. We used track #1.