05 / 09Study

Mix it up

Most homework comes in blocks. Section 4.2 teaches one formula, and the twenty problems after it all use that formula. By problem five it is smooth, because the method is known before the problem is read. The exam mixes everything, and the first job on each question is the one the homework never asked for: which method is this? Learning outside school has the same shape. A new tool's features learned one menu at a time, three kinds of customer complaint, two grape varieties that taste alike, a forehand and a backhand. Practicing one kind at a time feels like progress. Practicing them mixed feels like struggling. This lesson is about when the mix pays off, and when it does not.

In five minutesTake six problems from two recent homework sections and shuffle them. Before solving each one, write which method it needs and why. Then solve and check.

Method · Interleaving  /  Research psychology

↺From last time · Space it out

Answer these from memory before you start. Tap to check.

Why does a gap between reviews help?

Coming back after a gap, once the answer has started to fade, is what makes it last. In studies, the same reviews spread over days were remembered far longer than the same reviews in one sitting.

Why does a missed card go back to box 1?

A miss shows the gap is too long for that card for now. Box 1 brings it back daily until it holds, so hard cards get many reviews and easy ones few.

How long should the gaps be?

It depends on when you need the material. The best gap grows with how long the memory must last: a day or two for a test next week, weeks for something kept a year.

01Where it comes from

Doug Rohrer, a psychologist at the University of South Florida, noticed that most math textbooks follow each lesson with problems from that lesson only, while a few shuffle in problems from earlier lessons. In 2007 he and Kelli Taylor tested the two formats with college students [1]. In 2008 Nate Kornell and Robert Bjork at UCLA found the same pattern with something far from math: learning to recognize painters' styles [2]. The name for the struggle comes from Bjork, who called it a desirable difficulty [1]. The method does not work for everything. A 2019 meta-analysis found strong effects for paintings and other visual categories, a small effect for math, unclear results for expository texts, and an advantage for blocking when the material was word lists [3]. A 2013 review rated interleaving moderate utility for the same reason: promising, and not yet tested widely [4].

Also called Interleaved practice, mixed practice, shuffled practice.

02The method

Interleaving

  1. List the kinds of problem or category you are learning: four kinds of geometry volume, five painters, three contract clauses.
  2. Learn each kind first, with at least one worked example of each.
  3. Build practice sets that mix the kinds, so no two items in a row use the same method.
  4. Before solving each item, write which kind it is and why.
  5. Check your answers and note which kinds you confuse with each other.
  6. Keep vocabulary and word lists in their own blocks or on spaced cards.

03An example

What it looks like done

Worked on Six percent problems of three kinds, mixed: percent of a number, percent change, and finding the original amount

Questions

1. A $48 jacket is 25% off. What is the sale price?

2. A town grew from 12,500 to 14,000 people. By what percent?

3. After a 20% raise, pay is $54,000. What was it before?

4. A stock fell from $80 to $68. What was the percent change?

5. What is an 18% tip on a $65 bill?

6. A phone costs $391 including 15% sales tax. What was the price before tax?

Answers, kind named first

1. Percent of a number. The 25% is of the price I have. 0.25 × 48 = 12. 48 − 12 = $36.

2. Percent change: new minus old, over old. (14,000 − 12,500) ÷ 12,500 = 1,500 ÷ 12,500 = 0.12. Up 12%.

3. First try, percent of a number: 0.20 × 54,000 = 10,800, so $43,200. Wrong. The 20% is of the old pay, not the new. Original amount: new = old × 1.20, so old = 54,000 ÷ 1.20 = $45,000. Check: 45,000 × 1.20 = 54,000.

4. Percent change. (68 − 80) ÷ 80 = −12 ÷ 80 = −0.15. Down 15%.

5. Percent of a number. 0.18 × 65 = $11.70.

6. Original amount. The $391 already includes the tax. 391 ÷ 1.15 = $340. Check: 340 × 1.15 = 391.

Kinds I confused

  • Original amount, read as percent of a number (problem 3).
  • The clue I missed: the percent is of a number the problem doesn't give me.

04Try it · 30 minutes

For a class

Take the last three homework sections in a math or science course. Pick four problems from each and write each on its own card. Shuffle the twelve. Work through them in the shuffled order, and before each one, write the name of the method it needs. Mark which ones you misnamed. About 30 minutes.

For work and life

Pick three or four things that are easy to confuse in your work or hobby: types of support ticket, bird calls, wine grapes, chess openings, painters in a museum guide. Gather five examples of each, remove the labels, and shuffle them. Sort them into kinds, then check. Repeat a few days later with new examples.

The blank page for this is in the Study workbook.

05Where it goes wrong

  • Mixing before learning any kind. If none of the kinds is understood yet, the mix is noise. Learn each kind with an example first, then mix.
  • Judging by practice scores. Mixed practice scores lower while practicing. In one study it was 60% against 89% in practice and 63% against 20% a week later. Expect the dip.
  • Mixing unrelated subjects. The benefit comes from telling similar things apart. Alternating chemistry with history does not train that.
  • Using it on word lists. For lists of words, blocked study did better in the meta-analysis. Keep vocabulary on spaced cards.
  • Skipping the label. Solving a mixed problem without first naming its kind skips the step the mix exists to train.

06Research

What research has measured

Interleaving across kinds of material

Shown to help learning in some materials, especially paintings and other visual categories, with a smaller effect in math and none or the reverse for texts and words.

The numbers and sources
Effect
The overall interleaving effect was g = 0.42. Paintings showed the largest, g = 0.67, and math a small one, g = 0.34. Expository texts and tastes showed no significant effect. For words the effect reversed, g = -0.39, and blocking did better. Effects were stronger when categories looked alike, and for more complex material [3].
Population
238 effect sizes from 158 samples in 59 studies [3].
Caveats
One meta-analysis. Most studies were short laboratory experiments. The authors advise caution with expository texts and words [3].

Mixed math practice

Shown in controlled studies, including a large classroom trial, that mixing problem types in math practice raises scores on a delayed test.

The numbers and sources
Effect
Students who practiced four kinds of volume problem mixed scored 63% on a test a week later, against 20% for students who practiced them in blocks. During practice the order was reversed, 60% against 89% [1]. In a preregistered trial in 54 seventh-grade classes, students given mixed assignments for four months outscored the blocked group a month later on an unannounced test, 61% against 38%, d = 0.83 [5].
Population
18 undergraduates in the second experiment of [1]. 54 seventh-grade math classes [5].
Caveats
The 2007 experiment was very small [1]. The classroom trial was led by the researcher behind the earlier studies, and its authors note that important caveats remain [5]. Mixed sets also space out each kind of problem, so mixing and spacing overlap [1].

Painters, and how mixing feels

One study found people learned painters' styles better from mixed examples while judging blocked examples as the better way to study.

The numbers and sources
Effect
Participants picked the right artist for new paintings 61% of the time after mixed study, against 35% after blocked study. 78% did better with mixed study, and 78% said blocked study was as good or better [2].
Population
120 undergraduates in the first experiment and 72 in the second [2].
Caveats
One paper on one kind of visual category. In this study the mixed examples were also spaced, so the two cannot be fully separated [2].

07Check yourself

Answer from memory first. The next lesson asks these again.

Why does mixed practice help on a mixed test?

In a block, the method is known before the problem is read. A mixed set makes you decide which method each problem needs, and that decision is the first job on every exam question.

Why expect lower scores while practicing mixed?

Choosing the method each time is harder than repeating one. In one math study, mixed practice scored lower during practice and far higher on a test a week later.

When is mixing the wrong choice?

For word lists, where blocked study did better, and before any kind has been learned. Mixing pays off most for similar things that are easy to confuse.

Sources
  1. Rohrer D, Taylor K. The shuffling of mathematics problems improves learning. Instructional Science. 2007;35(6):481-498. https://doi.org/10.1007/s11251-007-9015-8
  2. Kornell N, Bjork RA. Learning concepts and categories: Is spacing the "enemy of induction"? Psychological Science. 2008;19(6):585-592. https://doi.org/10.1111/j.1467-9280.2008.02127.x
  3. Brunmair M, Richter T. Similarity matters: A meta-analysis of interleaved learning and its moderators. Psychological Bulletin. 2019;145(11):1029-1052. https://doi.org/10.1037/bul0000209
  4. Dunlosky J, Rawson KA, Marsh EJ, Nathan MJ, Willingham DT. Improving students' learning with effective learning techniques: Promising directions from cognitive and educational psychology. Psychological Science in the Public Interest. 2013;14(1):4-58. https://doi.org/10.1177/1529100612453266
  5. Rohrer D, Dedrick RF, Hartwig MK, Cheung CN. A randomized controlled trial of interleaved mathematics practice. Journal of Educational Psychology. 2020;112(1):40-52. https://doi.org/10.1037/edu0000367

MVIII puts the day in front of you each morning. Get MVIII