The speech-in-noise problem

People often have problems understanding speech in noise, and this is one of the main deficits of hearing aids that our machine learning challenges will address.

It’s common for us to hear sounds coming simultaneously from different sources. Our brains then need to separate out what we want to hear (the target speaker) from the other sounds. This is especially difficult when the competing sounds are speech. This has the quaint name, The Cocktail Party Problem (Cherry, 1953). We don’t go to many cocktail parties, but we encounter lots of times where the The Cocktail Party Problem is important. Hearing a conversation in a busy restaurant, trying to understand a loved one while the television is on or hearing the radio in the kitchen when the kettle is boiling, are just a few examples.

Difficulty in picking out speech in noise is really common if you have a hearing loss. Indeed, it’s often when people have problems doing this that they realise they have a hearing loss.

“Hearing aids don’t work when there is a lot of background noise. This is when you need them to work.”

Statement from a hearing aid wearer (Kochkin, 2000)

Hearing aids are the the most common form of treatment for hearing loss. However, surveys indicate that at least 40% of hearing aids are never or rarely used (Knudsen et al., 2010). A major reason for this is dissatisfaction with performance. Even the best hearing aids perform poorly for speech in noise. This is particularly the case when there are many people talking at the same time, and when the amount of noise is relatively high (i.e., the signal-to-noise ratio (SNR) is low). As hearing ability worsen with age, the ability to understand speech in background noise also reduces (e.g., Akeroyd, 2008).

When an audiologist assesses hearing loss, one thing they measure is the pure tone audiogram. This assesses the quietest sound someone can hear over a range of frequencies. However, an audiogram only partly explains your experience with speech in background noise (Heinrich et al. 2015), because it only measures the quietest sound you can hear. For example, picking out speech from noise is a complex task for the brain to perform, and this cognitive ability isn’t assessed by an audiogram. In addition, there are other factors that are important such as personality, motivation, attitude toward hearing aids and prior hearing aid experience.

An audiogram displaying a “ski slope” pattern that is a sign of age-related hearing loss (source: Ronan and Barrett, BMJ, 2014).

Speech-in-noise tests get closer to the real-life problem a hearing aid is trying to solve. Listeners listen to speech in the presence of noise and write down what words they hear. More words correct show an increase in the ability to understand speech in specific noisy situations when listeners are wearing their hearing aid (aided) relative to when they are not (unaided). Of course, listening conditions in the clinic differ from real-life conditions. In our first competition, we’ll be providing entrants with plenty of information about our panel of listeners including audiogram results and unaided speech-in-noise test scores.

Currently, while speech-in-noise test scores can be useful when fine-tuning a hearing aid, even then many users are disappointed about the performance of their hearing aids. Through our challenges, we hope to improve this situation, whether you go to cocktail parties or not.

What’s your experience with speech in noise? Please comment below.

References

Akeroyd, M. A. (2008). Are individual differences in speech reception related to individual differences in cognitive ability? A survey of twenty experimental studies with normal and hearing-impaired adults. International Journal of Audiology, 47(sup2), S53-S71.

Cherry, E. C. (1953). Some experiments on the recognition of speech, with one and with two ears. The Journal of the Acoustical Society of America, 25(5), 975-979.

Heinrich, A., Henshaw, H., and Ferguson, M. A. (2015). The relationship of speech intelligibility with hearing sensitivity, cognition, and perceived hearing difficulties varies for different speech perception tests. Frontiers in Psychology, 6, 782.

Vestergaard Knudsen, L., Öberg, M., Nielsen, C., Naylor, G., and Kramer, S. E. (2010). Factors influencing help seeking, hearing aid uptake, hearing aid use and satisfaction with hearing aids: A review of the literature. Trends in Amplification, 14(3), 127-154.

Kochkin, S. (2000). MarkeTrak V: “Why my hearing aids are in the drawer” The consumers’ perspective. The Hearing Journal, 53(2), 34-36.

Credits

Sounds for challenge 1

We’ll be challenging our contestants to find innovative ways of making speech more audible for hearing impaired listeners when there is noise getting in the way. But what noises should we consider? To aid us in choosing sounds and situations that are relevant to people with hearing aids, we held a focus group.

We wanted to know about

  • Everyday background noises that make having a conversation difficult.
  • The characteristics of speech after it has been processed by a hearing-aid that hearing aid listeners would value

A total of eight patients (four males, four females) attended the meeting, six of whom were recruited from the Nottingham Biomedical Research Centre’s patient and public involvement contact list. Two attendees were recruited from a local lip reading class organised by the Nottinghamshire Deaf Society. The range of hearing loss within the group is from mild to severe. They all regularly use bilateral hearing aids.

Our focus was on the living room because that is the scenario for round one of the challenge.

Photo by Gustavo Fring from Pexels

Everyday background noises that interfere with understanding of speech

A long and varied list of sounds cause problems. These lists are in no particular order.

Living room or space

  • Clocks ticking
  • Crisp packets rustling
  • Taps running
  • Kettles boiling
  • Dishwasher
  • Microwave
  • Washing machine
  • TV, music, radio
  • Phone ringing (or receiving texts – unknown beeps/ tones)
  • Newspapers rustling
  • Air-conditioning and oven extractor fans
  • Vacuum cleaner
  • Doorbell ringing
  • Dog barking
  • Rain on window

Family and friends

  • Cutlery/ crockery banging/ clanging
  • Doors opening/ closing (to rooms and cupboards)
  • Music
  • People walking around the room
  • Children playing with toys
  • Laughing
  • People talking from another room
  • Speakers from a different conversation in close proximity (i.e. beside you) when you are trying to converse
  • Traffic outside
  • Chewing/ chomping
  • Steam pipes/ coffee machines
  • Chairs being moved

Outside

  • Church bells
  • Market noise
  • Footsteps on different types of ground, i.e. heels on hard floors but also wellingtons in mud
  • Clothes rustling (such as waterproof coats or hat on hearing aid)
  • Wind (even with HA on ‘wind setting’)
  • Pigeons/ birds
  • Sirens
  • Traffic noise (especially at junctions)
  • Music
  • Laughter
  • Phones ringing
  • Tills
  • Children playing outside/ running around (in shops, on the street and at parks)
  • Beeping signal at crossings
  • Garden centres – high glass ceilings, open plan, trolleys
  • Road/ tyre and traffic noise when in a car or on the bus
  • Also mentioned how people you speak to in the car may be in front or behind you
  • Trains and the tube
  • Aeroplanes/ airports (suitcases rolling)
  • Tannoys

Characteristics of processed speech to consider

  • Clarity (clearness) or quality
  • Rhythm of speech
  • ‘Inflection’ (intonation)
  • Similarity to original speaker
  • Agreed that in situations where the voice would not be processed clearly, i.e. outside with many noise sources, not sounding like the original speaker is fine.

Other comments

  • Speed of speech; it was suggested that we have sentences read at different speeds as faster talkers are often harder to understand.
  • Stated that emphasis on key words is useful for following conversation; perhaps key words in the sentence when marked should be given higher value.
  • Lots of comments on room acoustics, i.e. ceiling heights, furnishings, floorings, windows etc., which has a big impact on how difficult it is to have a conversation with background noise.
  • Different accents of talkers can make conversation more difficult; including speakers with different accents in the background.

We’re now working out what sounds to use. But are there other sounds we should consider?

Credits