Following on from the mumble-gate sagas, with TV shows like Happy Valley and SSGB here in the UK in this article I wanted to explore a variety of issues that affect intelligibility.

Intelligibility is a multi-faceted issue and blame cannot be laid at any one door. Various reports in the press and even the House Of Lords, the upper chamber of the UK government, have had a go at these issues from a less than well-informed perspective, and I would like to explore some of the issues. So let's start with an intelligibility issue you might not be aware of or even considered could be a problem...

The McGurk Effect

When we talk to one another you may be surprised that it’s not just our ears that are paying attention. Our eyes are also picking up visual cues as well that can help fill in the gaps and give us a better sense of what we should be hearing, especially in more challenging environments. 

Our brains combine data from both our sight and hearing senses to produce a fuller picture of what we hear, but for this to be effective the visual and aural information has to match and it may surprise you to learn that our eyes can win over what we hear and evidence for this is what is called the McGurk effect.

Ba or Ga?

It was cognitive psychologist Harry McGurk, who discovered this by accident in 1976, the effect shows up when we see a person’s mouth produce one sound whilst hearing another sound. The most common example used is when we 'see' a person mouthing “ga” whilst we hear the sound “ba”. For some reason, as a result of this mismatch, we hear the word “da” instead. Watch this video and see what you hear. Then, try it again with your eyes closed.

The understanding behind this is that the McGurk effect happens because our brains fail to recognise that two stimuli aren’t coming from the same source and the fact that certain syllable combinations aren’t merged together indicates that there’s some underlying mechanism deciding what types of audiovisual information should or shouldn’t be integrated.

Researchers still don’t completely understand how our brains link disparate events together, but for now, it’s another reminder that we can’t always trust what we see and hear.

Measuring Intelligibility

Here in the UK, Salford University is currently undertaking some research on intelligibility with a team studying if the addition of relevant sound cues helps with intelligibility for both people with normal hearing and people with a range of hearing impairments. 

They used four experimental conditions, testing for the recognition of a particular keyword like ’sword’. They tested for the predictability of the word as well as adding a relevant sound effect. The tests were undertaken with multi-talker babble, as a masking sound set 2dB below the wanted speech for the people with normal hearing.  For the people with hearing impairments, they adjusted the signal to babble ratio for each listener depending on the extent of their impairment.

For low predictability, they tested the sentence “Mary should think about the sword” where you would not be likely to predict the word 'sword' would be coming.

For the high predictability test, they used the sentence “He killed the dragon with his sword” where the word 'sword' could be predicted. 

Initial Results

For people with normal hearing the predictability of a word coming improves intelligibility by 73.5%. where as adding the sound cue, in this case, the swish of a sword improved the intelligibility by 69.5%. They found that by adding predictability and a sound cue together further improves the intelligibility by 18.7%

Moving onto to people with hearing impairments the use of acoustic cues improved intelligibility for half the participants with a quarter of them reporting an improvement of greater than 20%.

One Size Doesn't Fit All

However, for the people with hard of hearing the variety of results makes it clear that any ‘one size fits all’ solution will not be effective. The negative impact that sound effect cues had on intelligibility for some hard of hearing listeners may have been caused by masking or cognitive overload effects. So the team at Salford University are planning further investigations to try and determine which of these possible effects is the most prominent.

Conclusion

I hope this article shows that intelligibility isn't just about relative volume but is affected by what we see, as well as the way the words are arranged, so script writers have a part to play in intelligibility with the order and structure of the words. In conjunction with the director and sound team, script writers also have a part to play in what sounds surround the dialog and poor choices can have a negative impact on intelligibility.