03 / 08Think

What would change my mind?

Someone holds a firm view about a diet, a coworker, a phone plan, or which route to work is faster. Asked why, they give reasons. Asked what would change their mind, most people stall for a moment. The question is rarely asked, so the answer isn't ready. In a class it comes up with the hypothesis in a lab report, or an essay's thesis. At work it comes up with a sales forecast, a hire, or a plan that is "sure to work." A belief with no answer to this question can't lose an argument. It can't learn from one either. This lesson asks the question before the argument starts, writes the answer down with a number, and checks back on a set date.

In five minutesWrite one belief you're acting on this week and how sure you are, as a percentage. Under it, write one specific result that would make you lower that number.

Method · What would change my mind  /  Science · Research psychology

↺From last time · Define your terms

Answer these from memory before you start. Tap to check.

Why does an example not count as a definition?

An example shows one case. It doesn't say what the cases share, so it can't sort a new one. A definition names the wider kind and what sets this thing apart, which lets anyone apply it.

Why test a definition on cases it should include and leave out?

A definition can sound right and still be too wide or too narrow. Checking it against two cases in and two out shows quickly whether it sorts things the way you meant.

Why add a number to a word like "soon" or "common"?

People read the same loose word as very different amounts, as the studies on "rare" and "common" show. A number or a test leaves less room for each person to hear their own meaning.

01Where it comes from

In August 1650 Oliver Cromwell, camped near Edinburgh a month before the Battle of Dunbar, wrote to the leaders of the Church of Scotland and asked them to "think it possible you may be mistaken" (see Voices). The philosopher Karl Popper made a version of the idea a test for science. In Logik der Forschung (1934), translated as The Logic of Scientific Discovery (1959), he argued that a theory counts as scientific only if some possible observation could show it false [1]. Psychologists measured the everyday version. In 1980 Asher Koriat, Sarah Lichtenstein and Baruch Fischhoff found that listing a reason an answer might be wrong made people's confidence more accurate [2]. In 1984 Charles Lord, Mark Lepper and Elizabeth Preston named the strategy "consider the opposite" [3]. In a two-year forecasting tournament, Barbara Mellers, Philip Tetlock and colleagues trained people to reason in probabilities and to revise their forecasts as news came in [4].

Also called Falsifiability, consider the opposite, belief updating, calibration.

Think it possible

Is it therefore infallibly agreeable to the Word of God, all that you say? I beseech you, in the bowels of Christ, think it possible you may be mistaken.

Oliver Cromwell, Letter to the General Assembly of the Kirk of Scotland Musselburgh, 3 August 1650. Letter CXXXVI in Carlyle's Letters and Speeches of Oliver Cromwell, edited by S. C. Lomas, 1904, volume II, page 79

Cromwell's army was outside Edinburgh, and the Scottish church leaders had answered his declaration with one of their own. Each side claimed to read God's will rightly. "In the bowels of Christ" was the period's way of appealing to Christ's mercy. The sentence asks the reader to grant, in plain words, that a firm belief might be wrong. The request is hard to make of anyone. It is harder to make of oneself, where this lesson points it.

Read it in full

02The method

What would change my mind

  1. Write the belief in one sentence, and how sure you are, as a percentage.
  2. Write what you would expect to see if it were wrong. Make it specific: a number, an event, or a result someone else could check.
  3. List the strongest reason the belief might be false. Then a second one.
  4. Look for that evidence on purpose, in the places most likely to have it.
  5. Set a date to check. On that date, write what happened and your new percentage.
  6. Move the percentage in steps as evidence comes in. One fact rarely justifies 0% or 100%.

03An example

What it looks like done

Worked on A belief about a daily commute: the back roads are faster than the highway

The question

Is the back-roads route to work faster than the highway at 8 a.m.?

What I think, and why

  • Yes, 80% sure.
  • The highway backs up at the bridge most mornings.
  • The back roads never stop moving, so they feel quicker.

The other side at its strongest

  • The back roads have six lights and a school zone.
  • Moving feels faster than sitting in traffic even when it takes longer.
  • I remember the bad highway days and forget the smooth ones.

What would change my mind

  • Test: time ten trips on each route over four weeks, door to door, leaving at the same minute.
  • If the highway averages two minutes or more faster, I switch.
  • Check date: the last Friday of the month.
  • Result: highway 31 minutes on average, back roads 33. New belief: the highway is faster, 70% sure. Retest in winter.

04Try it · 20 minutes

For a class

Take the thesis of your next essay, or the hypothesis in a lab report. Ten minutes. Write it in one sentence with a confidence percentage. Under it, write two findings that would show it wrong, and where in your sources they would turn up if they existed. Search those places first. Then revise the thesis or the percentage, and note which way it moved.

For work and life

Pick a belief that a decision rests on: a supplier is reliable, a product will sell, a new routine is saving time. Write it with a percentage and the one result that would change your mind, such as a number by a date. Put the date on your calendar. On that day, write what happened and a new percentage before you reread the old one.

The blank page for this is in the Think workbook.

05Where it goes wrong

  • A test that can't fail. "I'll know it when I see it" names nothing. If no possible result would count against the belief, rewrite it until one would.
  • Moving the goalposts. When the named evidence arrives, it is tempting to decide it doesn't count. Writing the test down first puts the answer on paper before the result.
  • All or nothing. One bad week doesn't prove a plan failed. Move the percentage a step, not to zero.
  • Only on beliefs you doubt. The question is easy to ask of a view you never liked. Ask it just as hard of the one you hold most firmly.
  • Doubt as a stall. Being open to change doesn't mean waiting forever. Set the check date, then act on the best estimate you have.

06Research

What research has measured

Considering the opposite

Shown that asking people to list reasons they might be wrong makes their confidence more accurate and reduces one-sided weighing of evidence.

The numbers and sources
Effect
People who listed reasons for and against each answer before choosing gave more appropriate confidence judgments. When asked for just one reason, only a reason against the chosen answer helped. A supporting reason did not [2]. In two experiments, instructions to consider the opposite reduced bias in weighing new evidence and in testing first impressions of people, and worked better than instructions to be fair and unbiased [3].
Population
Two experiments in each paper [2][3]. The abstracts do not give participant counts.
Caveats
Lab tasks, measured on the spot. Neither paper followed people into later real decisions [2][3].

Forecasting with training

One large study found that a brief training in probability reasoning made forecasts of real events more accurate, and the gain held across two years.

The numbers and sources
Effect
In a two-year tournament on 199 questions about world events, forecasters given probability training were more accurate than untrained forecasters in both years. The training covered base rates from similar past cases, averaging several estimates, and traps such as overconfidence and confirmation bias. Forecasters could update as often as they wished, and working in teams also improved accuracy [4].
Population
2,246 participants in year one and 1,648 in year two, all with a bachelor's degree or higher, 83% men [4].
Caveats
One tournament with educated volunteers. Accuracy gains are reported as test statistics, not a simple percentage [4].

07Check yourself

Answer from memory first. The next lesson asks these again.

Why write down what would change your mind before looking at the evidence?

Once a result arrives, it is easy to decide it doesn't count. A test written in advance fixes what counts as evidence against the belief, so the result can't be explained away after the fact.

Why does a reason against your answer help more than a reason for it?

Confidence rests on the evidence that comes to mind, and supporting reasons come easily. A reason against brings in what was left out, so the confidence matches the full picture better.

Why move a percentage in steps instead of switching all at once?

Most single results are noisy. A step keeps the earlier evidence in the count, and lets a run of results move the belief further than any one could.

Sources
  1. Popper KR. The Logic of Scientific Discovery. London: Hutchinson; 1959. Routledge Classics, 2002
  2. Koriat A, Lichtenstein S, Fischhoff B. Reasons for confidence. Journal of Experimental Psychology: Human Learning and Memory. 1980;6(2):107-118. https://doi.org/10.1037/0278-7393.6.2.107
  3. Lord CG, Lepper MR, Preston E. Considering the opposite: A corrective strategy for social judgment. Journal of Personality and Social Psychology. 1984;47(6):1231-1243. https://doi.org/10.1037/0022-3514.47.6.1231
  4. Mellers B, Ungar L, Baron J, et al. Psychological strategies for winning a geopolitical forecasting tournament. Psychological Science. 2014;25(5):1106-1115. https://doi.org/10.1177/0956797614524255

MVIII puts the day in front of you each morning. Get MVIII