Review | 'Noise: A Flaw in Human Judgment': Daniel Kahneman's new book shows us why the boss' mood and other seemingly extraneous details matter

2014 photo of Prof. Daniel Kahneman (right) and Prof. Maya Bar Hillel talking about Kahneman's research. (Image via Wikimedia Commons CC 3.0).

Do you remember chuckling at the common saying: ask someone for a favour after they have had lunch? Obviously it's because at the time, they are in a good mood, and more likely to grant the favour.

The saying has a kernel of truth to it, and is actually referred to in Noise: A Flaw in Human Judgment by Daniel Kahneman, Olivier Sibony, Cass R. Sunstein - a serious, scientific book.

The book discusses at length something that is usually swept under the carpet: arbitrary variations in human judgements.

The most memorable aphorism from this book, at least for me, is that where there are decisions, there is ‘noise’ – arbitrariness in decisions that can vary from person to person, and from time to time when made by the same person.

Bias or “systemic deviation” is well known and often mindfully addressed while making decisions; but not so with noise - “random scatter”, which is also destructive.

Noisy decisions, the authors say, can harm, blight or end lives. Even in workplaces, those hives of ‘professionalism’, noise is ever-present. Ever felt your annual evaluation was ‘off’? On any day, we make hundreds of decisions that can cause injustice or in other ways affect lives. Noise occurs everywhere, in matters of hiring and promotions, medicine, and of course, justice.

Some justice systems have been studied extensively by scientists focusing on noise. The authors quote a famous series of examples in which “similar people, convicted of the same offense” got “dramatically different sentences”.

One well-known example: “[t]wo men, neither of whom had a criminal record, were convicted for cashing counterfeit checks in the amounts of $58.40 and $35.20, respectively. The first man was sentenced to fifteen years, the second to 30 days.” While some studies analyzed actual decisions, other studies have used simulation to find noise. In the United States, a study mapped the decisions of “208 federal judges” in “the same sixteen hypothetical cases”. The results? “In only 3 of the 16 cases was there a unanimous agreement to impose a prison term”. And the prison terms varied alarmingly: “In one fraud case in which the mean prison term was 8.5 years, the longest term was life in prison.”

It’s unacceptable, this variation that flies in the face of justice. What accounts for it? Bias, yes, but also noise. The authors say that “judges have been found more likely to grant parole at the beginning of the day or after a food break.” With chilling pithiness, the authors add, “If judges are hungry, they are tougher.”

The story gets weirder. Other studies found that when “the local football team loses a game on the weekend, the judges make harsher decisions on the Monday (and, to a lesser extent, for the rest of the week)”.

Further shocks follow: “A review of 207,000 immigration court decisions over four years found a significant effect of daily temperature variations: when it is too hot outside, people are less likely to get asylum.” So, mood influences decisions, but so do factors such as “the order in which cases are examined”. These conclusions come from studies, a few conducted by the authors themselves.

In some cases, noise can be desirable, we are told; for instance, in the artistic judgements of, say, film critics or wine tasters (book reviewers, too, I might add); and in situations where “the best judgments will be rewarded” or when “multiple teams of researchers attack a scientific problem, such as the development of a vaccine”.

 Mostly, though, noise is unjustified, and occurs even where you wouldn’t expect it. In one of the most shocking parts of the book, the authors quote the work of Philip Tetlock, who analyzed “the predictions of almost three hundred experts: prominent journalists, respected academics, and high-level advisers to national leaders. He asked whether their political, economic, and social forecasts came true. The research spanned two decades...” You’ve probably guessed where this is going. In Tetlock’s memorable words, when it came to forecasting, “The average expert was roughly as accurate as a dart-throwing chimpanzee”. I won’t offer spoilers; you should check out the chimpanzee at close quarters.

We are told that group decisions, too, can be noisy; for instance, when groups follow an authoritarian, anti-democratic leader. (Does that sound familiar, maybe?) The authors delve into studies that explored “a particular driver of noise: social influence”. Popularity, particularly online popularity, can be manipulated with ease, as can political opinions online. We are told of “information cascades” which lead people to align willingly and even readily with opinions they disagreed with. And the idea of “group polarization” is particularly relevant to those seeking answers for why a chunk of Indian voters shifted allegiance to the hardline right in recent years.

The book’s overall structure is as follows. The book defines noise, why it occurs, why it should be reduced, and how to reduce it. The explanations are bolstered by case studies from various fields ranging from management to justice systems to medical science, among others. So, we are told how the human being usually makes decisions and how they should be made; this involves a grand tour through, as you might expect, psychology, but also statistics and social sciences. So, the middle of the book gets technical, and rightly so. The authors do not mandate blind belief in their statements, but show us the underlying science. At the middle of the book, you might have to re-read a few passages; or, as the authors suggest, you can skip the middle and jump to the end of the book, where you find methods for reducing noise.

Along the way, we find admirably lucid explanations of concepts that we take for granted or simply don’t know, such as the difference between predictive and evaluative judgements, and between clinical and mechanical judgements; on reading this part, you might feel yourself developing the inclination to become more self-reflective. ‘Clinical judgement’ is defined: “[C]onsider the information, perhaps engage in a quick computation, consult your intuition, and come up with a judgment”. Studies are mentioned that assessed clinical and ‘mechanical judgements’ across “a wide variety of topics, including diagnosis of jaundice, fitness for military service, and marital satisfaction”. We are told how, in matters involving prediction, ‘mechanical judgements’ are better, which come up with decisions by applying simple or complex models. Even simple models, because they have no noise caused by “arbitrary reactions... to a particular case”, outperform humans in making predictions. Overall, the book questions our tendency to apply ‘gut feel’ or ‘intuition’ to our decision-making in an indiscriminate manner. This will be a bitter pill to swallow for people in leadership roles.

You know where this is going, but there’s a twist: the authors say that in many cases, algorithms can be used in decision-making. The risk of bias in algorithms is discussed, with the explanation that it can be filtered out of algorithms. Here, the authors say that their goal “is to offer suggestions for the improvement of human judgment, not to argue for the ‘displacement of people by machines’”. Algorithms are not perfect, and at least as of now, their lead over human judgement is slender. Also, algorithms may never be perfect “in many domains”, so “human judgment will not be replaced. That is why it must be improved”.

Near the end of the book, we are told of individuals called ‘superforecasters’, “whose predictions are better than chance”, and a stepwise explanation that breaks down how superforecasters think. Finally, the authors give us “a method of decision-making in organizations... which was designed with noise mitigation as a primary objective”, namely, the ‘mediating assessments protocol’. Organizations can tweak it for their purposes.

You may know the name of Daniel Kahneman - the Nobel Laureate who also wrote Thinking, Fast and Slow. He is a psychologist and economist; Olivier Sibony is a “professor, writer and keynote speaker specializing in the quality of strategic thinking and the design of decision processes” (as per his website); and Cass R. Sunstein is a professor at Harvard, legal scholar and writer.

Reading this book, I remember feeling slightly giddy and very disoriented as my sense of complacency about my decisions, personal and professional, was jolted. The book is highly recommended for those people whose decisions affect the wellbeing, fortunes, and lives of others: the general audience.