Insights for Making Better Decisions | Gary Klein | Knowledge Project 144

KEYPOINTS

• 01

  Performance has two arrows: fewer errors and more insights. Organizations watch only the first.

  Klein used to show a slide with a down arrow (reduce errors) and an up arrow (increase insights). After a talk in Singapore, someone asked what he knew about insights, and he had to admit he knew nothing. He stewed on it for a 17-hour flight home, then collected 120 examples of real insights. The incentives are lopsided. A bad idea is a public mistake, but a missed insight is invisible. Insights are disorganizing and disruptive, so even organizations that sincerely say they want innovation tend to stifle it. It only takes one person in a chain of approvals to kill a fragile good idea, and experienced leaders carry 'scar tissue' that makes them say 'we tried that and it didn't work', even when technology or circumstances have changed.

• 02

  Insights come by three paths: connection, contradiction and correction.

  In the connection path, ideas combine: Darwin knew species changed, then read Malthus on competition for scarce resources and saw what drove evolution. In the contradiction path, something doesn't fit: a rookie cop at a red light saw a driver flick cigarette ash inside a brand-new BMW, thought 'who does that?', and pulled over what turned out to be a stolen car. In the correction path, a flawed belief has you stuck, and in about 27 such cases a hint made people investigate instead of dismissing the anomaly. Across all three, the 'secret sauce' is getting curious about things that don't make sense. He also advises noticing and celebrating your small daily insights, not just beating yourself up over mistakes.

• 03

  Experience isn't expertise. Experts know their mistakes, and the mistakes 'eat at them'.

  Klein has about seven or eight criteria for spotting an expert, and none is foolproof. Years of experience, track record (an investor right five times may just be lucky), peer opinion and scenario performance all help. His favorite test is 'Tell me the last mistake you made.' Someone who can't think of one may be competent but isn't an expert. Journeymen often plateau because they move on and forget their errors. To break through, people must unlearn conventions and beliefs that are wrong or narrower than they thought.

• 04

  Confidence is a poor guide to who knows what they're doing.

  At a petrochemical plant, two newly promoted operators went through decision-making exercises. Everyone loved the confident one, who carried himself well. The tentative one, full of caveats, worried people. The exercises showed the confident operator didn't grasp the plant's critical dynamics and wasn't safe at a panel that could blow up, so he went back outside for another year. The hesitant one understood the causal relationships. The unhappy postscript was that once results were used for evaluation, nobody wanted to do the training anymore.

• 05

  A rich mental model has four parts, not one.

  Studying petrochemical operators with colleagues, Klein expected a mental model to mean 'how it works'. Experts also knew its limitations and boundary conditions, the workarounds for when those limits hit, and where other people are likely to get confused. Because words are ambiguous, abstractions transfer poorly. Ask for the story behind someone's lesson instead: what surprised you, how did you notice it, what did you make of it? Good stories have a mystery, real change, and an insight at the end. Different listeners take different lessons from them, which shows how their mental models differ.

• 06

  Fixation comes from explaining away anomalies. Quick first impressions are fine.

  Snap judgments based on experience usually work, so telling people not to form them is 'ridiculous'. The danger is protecting the story with 'knowledge shields', which even scientists use, by questioning how inconvenient data was collected. In project reviews, ask 'What surprised you in the last three months?' If the answer is 'nothing' on a complex problem, that's when to worry. A decision journal can log your read of the situation and each piece of inconsistent data, so you can see when you should have lost faith in it.

• 07

  Run a real pre-mortem: assume the project has already failed and explain why.

  Klein invented it in the late 1980s by moving the after-action review to the kickoff meeting. With 20 minutes left, the leader says an infallible crystal ball shows the project failed six months from now. Everyone spends two minutes writing reasons, then goes around the room one item at a time, with the leader going first with something real. Finish with two more minutes on what each person will do to prevent those failures. It reverses the usual 'any problems?' dynamic, where nobody wants to spoil the go-mode. Now people show they're smart by finding issues nobody else raised, which builds candor and trust. Common mistakes are framing it as 'what could go wrong?', letting people ramble, and a leader who holds back his own items.

• 08

  He dissents on biases: most heuristics serve us well, and emotion is essential.

  He stresses that the heuristics-and-biases view is dominant and well respected. Still, he argues debiasing has had little success, and rightly so. Availability, representativeness and anchoring are how expertise gets used, and they look like biases only in lab traps or hindsight. Kahneman and Tversky themselves said heuristics are generally useful. He cites Damasio's patients with brain lesions that cut emotion off from decisions. Their IQs were intact, but they took 45 minutes to choose a restaurant and lost jobs and marriages. He offers to pay for such surgery for any researcher who thinks emotion corrupts judgment. Nobody has taken him up on it.