To Err Is Human: The Science of Judging Mistakes

Guest post by George Waddell

Mistakes are the bane of every pianist. We spend hours in the practice room trying to prevent them, in the teaching studio deconstructing them, and in post-concert receptions (and sleepless nights) obsessing over them.

At the same time, we pianists are often told that there is more to making music than getting the notes right. That an examiner or a concertgoer will look past a slipup, if they even notice it, and appreciate our musicality as a whole. So which is true? Are mistakes a big deal or not? Answering questions like this is why I became a Performance Scientist. Early in my undergraduate music studies I became fascinated by what psychology could tell us about how we think and behave, and how much it can inform the challenges we face as musicians, whether it be how we learn, teach, memorise, collaborate, execute incredibly complicated skills, or deal with the pressures of a performance or performing career. In particular, I was interested in what psychology could tell us about those who judged our performances, and how their perceptions and decisions may not be as consistent as we might hope. Two of the first studies I conducted with my colleagues in the Royal College of Music’s Centre for Performance Science used mistakes as a way of understanding how judges assess our performances. In the first, we wondered whether the idea that ‘first impressions count’ was true for pianists. Would a mistake in the opening seconds of a performance have the same effect on the overall rating than the exact same mistake part way through? To test this, I started with recordings of Chopin’s Minute Waltz and Black Key Etude performed on an electric keyboard. I edited the recordings to make them sound like live performances on a real instrument, but since they were recorded digitally (i.e. using MIDI) I was able to take a few notes from the opening seconds of each and move them down a semitone, so that they sounded unmistakeably wrong. Then, as the opening material of both the Waltz and the Etude returns part way through each piece, I made a second version of each where the exact same mistake is made later on in the performance. All we needed next was a group of musicians who rated the overall quality of each performance on a 7-point scale from Poor to Excellent. Some heard error-free versions, some heard mistakes at the start, and some heard mistakes at the end.* Crucially, the musicians didn’t just give overall ratings. While listening, they also used a computer to move a slider up and down as their moment-by-moment assessment of the performance changed. This way, we could see how the musicians reacted to each error as they happened. For those who want the detail, you can read the full paper in the journal of Music Perception, where you can also hear the audio recordings.

You can see the results in the image below, and they were dramatic. Before I tell you, and before you look at the legend in the top right corner of the image, try to guess which line represents the error-free performance, which one had the mistake at the start, and which one had the mistake part-way-through.

Waddell1

How did you do? The blue line in each represents the version with no mistake, and in both the Etude (top) and Waltz (bottom) the average judgement was pretty consistent. The red line with squares shows ratings of the piece with the mistake in the opening seconds. It started much lower, and while things improved as the piece went on it never fully recovered, with the final rating still lower (for any statisticians in the audience: significantly lower) than the overall score. The green line with the triangles is the most interesting of all. It represents the mistake part way through each piece, which was quickly penalised, and almost as quickly ‘forgiven’ by the musicians with final scores not meaningfully (i.e. significantly) different from the final score. The final scores on the 7-point scales told the same story. So first impressions really did matter, a result that mirrors what we know from social psychology in that that we make judgements about new people we meet very quickly, and hang on to bad first impressions more strongly than good ones. I also measured how quickly those first impressions were formed. With no mistake at the start, musicians made their first judgements within an average of about fifteen seconds from the first note. With the mistake, that time was cut in half.

In this case, it seemed that musicians did not get over the poor first impression resulting from the initial mistake, or were at least much more willing to forgive the mistake when it occurred over half-way into the performance. So make sure you spend some extra time practising those first few bars, as they truly are important.

In a second study, we wanted to take this result even further. What would amateur and untrained musicians think versus a highly-trained group? What if you could see the performance, not just hear it? And, if we can see the performance, what if the pianist’s facial expression affected how we perceive the error? I was keen to examine this last question as this is something I explicitly thought about as a young pianist. When I made a mistake, I would pull a frustrated face as a way of signalling to the adjudicator or audience that “yeah, I heard it too. Sorry about that, I don’t usually mess up there”. Later on, I heard teachers emphasise the value of the ‘poker face’; after all, maybe the judges didn’t notice. Once again, an experiment was needed to test which approach was better.

Creating the recordings was more difficult in this case, as we wanted the videos to appear as though they came from genuine performances. To do this, we recorded a pianist in the Royal College of Music’s main concert hall playing Chopin’s Aeolian Harp Etude. We used two camera angles between which the recording alternated throughout the performance: one from the side where you can see the pianist’s hands on the keys so that a viewer would think they were watching a genuine, unmanipulated recording; and a shot looking down the length of the piano highlighting the pianist’s face and obscuring his hands, so that when we did splice in different audio or video, it could be done without anyone noticing. Once we had a good performance down, we asked the performer to play a middle section of the piece but make a mistake that caused him to stop playing, search a moment for his place, then start again, all while pulling a frustrated face. With this footage, the wizards in the Royal College of Music Studios helped us create four videos: one without any error, one where you hear the mistake but see the ‘poker face’ from the good recording, one where he makes the mistake and pulls the face, and one where he pulls the face but didn’t make the mistake. We also recorded some stage entrances that were added to the top and spliced in a bit of video and audio from a live audience to give the impression of a real concert.** You can read all of the details and download the videos in our article in Frontiers in Psychology, and you can see clips of the recordings in the video summarising the study below.

We recruited people ranging in musical experience and training to watch and rate the videos (each person saw one version), again asking them to give an overall score out of 7 and a continuous rating on a bit of software I created. Once again, the results were dramatic. You can see them in the chart below, where the blue line represents the highly trained musicians and the red line represents people ranging from little to no musical training.

Waddell2

As you can see, the normal (standard) video at the top showed the same pattern as the musicians rating the no-error version in the first study – fairly even without major reactions. Look now at the third graph (aural): here you see the trained musicians behaving the same as those in the first study, dropping their rating when the mistake happened but then returning to a point not meaningfully different from those rating the performance without any error at all. Interestingly, you can see that the non-musicians (the red line) didn’t react to the mistake in that case. Look a little higher now at the second graph (aural/facial). Here we see something striking: when you give people the same mistake but add the negative facial reaction, the rating drops suddenly regardless of musical experience, and never recovers. You might wonder, then, whether it was the face, not the mistake, that was penalised, but the fourth graph (facial) suggests otherwise. The face alone wasn’t enough to cause a reaction in either of the groups.

So why did pulling a face cause this reaction? A theory from social psychology called facial overgeneralisation may account for it. In short, when you meet a person for the first time, and their face looks sad, your brain doesn’t tend to go “this is a person who, right now, is sad”. Your brain goes “this is a sad person”. In other words, your brain assumes the facial expression represents a permanent trait that they have, rather than a temporary state they are in. So in the case of the musicians, perhaps they didn’t see a pianist who was frustrated because he had made a mistake. Instead, they saw a frustrated mistake-maker and judged him more harshly as a result.

It would seem the 12-year-old me was wrong – pulling faces was a bad idea. As is always the case, more research will be needed to see whether these findings appear in other contexts. What is clear, though, is that there is more to a mistake than a few wrong notes. The context in which they happen matters as, even with decades of musical training, our brains still shape what we see, hear, and think without us realising it. Keep that in mind the next time you’re asked to judge someone’s performance and things don’t go entirely to plan.

To err is human. To err in perceiving errors, doubly so.


*For those keeping score, each of the 42 musicians heard a different combination of mistake/no-mistake between the two pieces, and the order of the two was switched up among other pieces they were listening to for other pieces of the research. This was important as we tend to rate things differently depending on the order in which we hear them and what we heard before. Mixing the recordings up this way counteracts these effects. You can read all of the details in the Music Perception article.

**The stage entrances were part of a different experiment, and a topic for another day. You can read about them in the Frontiers in Psychology article.


Dr George Waddell is a Research Associate in Performance Science at the Royal College of Music and a Sessional Lecturer and honorary Research Associate at Imperial College London. His research focusses on the processes of performance evaluation and the development of technologies to enhance performance training in music, medicine, and beyond. He holds a PhD in Performance Science from the Royal College of Music and a BMus and MMus in piano performance from Brandon University, Canada.

PerformanceScience.ac.uk/team/waddell

Twitter: @geordiewaddell