# People are flagrantly bad at statistics

The “Autism Spectrum Quotient” questionnaire is a well-known instrument to diagnose autism spectrum disorder. It takes just a couple of minutes to fill out the questions and find out if you score 32 or higher, which means there is a high probability that you are diagnosed with autism spectrum disorder. The disclaimer on the website reads “the test recognizes 80% of people with autism and only 2 per cent of people without autism get a wrong result.” Although those odds don’t seem bad at all, they are horribly misleading if you do the math. [1]

Estimates of autism prevalence are close to 1% of the population. On 1,000 people, 10 persons have autism, yet the questionnaire would only give 8 of them the correct diagnosis. While of the remaining 990 persons 19 would be wrongly diagnosed with autism. In other words, if the test results show you have autism, there is a 70% chance you received the wrong diagnosis.

It is hard for humans to derive meaning from math or figures. Our brains are simply not wired to treat numbers with the amount of attention that is needed to make sense of the data. Unless our brains are forced to, it will process numbers and figures in express mode: the ‘what you see is all there is’ kind of way. A test with 80% accuracy and a 2% false-positive rate seems quite accurate. Only when we do the math, we discover our brains deceive us.

Nobel-prize winner Daniel Kahneman dedicated a large part of his career to figuring out why we’re so bad at interpreting statistical figures and probabilities. If there’s one conclusion we can draw off his work and from behavioural scientists alike. It’s that we cannot trust our brains when it comes to processing abstract information such as statistics[2]. Processing them is effortful, and even when we do the effort, interpretations are hard.

One way to smooth your brains into processing information with more attention and effort is by presenting it in the form of a narrative [3]. An engaging tale that provides context and relevance goes down more easily than abstract notions and figures. Striking, emotional stories with big headlines are the preferred format of information for our brains. Our brains work just like a tabloid.

For figures to resonate with an audience, we have to make them appeal to our tabloid-mind. Or, put in Schumacher’s words: “you have to make them sing”.

Florence Nightingale, the famous nurse-statistician, knew how to use the power of creativity for presenting medical figures. She wrote in simple English and was a pioneer in the use of graphical presentations of data. Years before Mary Poppins taught us about “sugar to go with the medicine”, she pointed out that an entertaining format helps when communicating abstract content.

Her most famous infographic shows the causes of mortality in the British army in the 1850’s campaign in Crimea. The way she designed the infographic lets readers directly notice that the blue areas that represent death by diseases are bigger than the red ones, that represent death by combat.[4] Moreover, one immediately notices that the blue coloured area drastically shrinks around the time she took action to improve army hygiene. A clear call to action towards Queen Victoria and the British Parliament: improve army hygiene and hygiene of the British people altogether by, among others, better drainage and slum clearance.

Would she have had the same impact when she had talked about probabilities, false-positives or base rates as in the autism example?

SWAP ACCURACY FOR CREATIVITY

When presenting figures in the way they resonate the most, the desire to be accurate is sometimes in the way of creativity. Schumacher also came across this problem. He figured out that when we drop the desire for accuracy, we allow ourselves to broaden the creative playing field. Thus, more options become at our disposal to create an engaging story.

You could argue that dropping accuracy is problematic. Yet, as long as you do not lose the notion of significance, Schumacher was convinced you’re fine. In most cases, statistics are used for argumentation, as a means to an end. When the argument is valid, shown by the significance of the results, you can allow yourself some creative freedom.

Loosening the notion of accuracy gives rise to ethical questions: you could shape the representation of figures in such a way that they emphasize what you want to the audience to take away. Yet, keep in mind that even when you try to present data most truthfully, interpretations might differ depending on your (accidental) choice between a histogram or a pie chart, or red versus green coloring.[5] An author will always show data with a purpose, every number can be read in different ways if compared to other numbers or when presented in a different format. It is the job of the analyst to interpret the data and show them so they serve her/his interpretation. As long as the shown differences are backed by significant results and full transparency is given on the data used, you’re fine.

This essay was originally written for the 2020 WPP Atticus Awards

[1] Smeets, I. (2019, February 15). Je moet voorzichtig zijn met het zonder aanleiding testen van grote groepen mensen. Retrieved from https://www.volkskrant.nl/wetenschap/je-moet-voorzichtig-zijn-met-het-zonder-aanleiding-testen-van-grote-groepen-mensen~bc1045ae/.

[2] Kahneman, Daniel. Thinking, Fast And Slow. New York : Farrar, Straus And Giroux, 2011.

[3]  Bower, G. H., & Clark, M. C. (1969). Narrative stories as mediators for serial learning. Psychonomic Science, 14(4), 181–182.

[4] Giaimo, C. (2018, February 8). Happy Birthday Florence Nightingale, Unexpected Queen of Infographics. Retrieved from https://www.atlasobscura.com/articles/florence-nightingale-infographic.

[5] Murray, E. (2019, March 22). The Importance Of Color In Data Visualizations. Retrieved from https://www.forbes.com/sites/evamurray/2019/03/22/the-importance-of-color-in-data-visualizations/.

This site uses Akismet to reduce spam. Learn how your comment data is processed.