The pseudoscience of learning

How to spot it in the education debate

“You got to know when to hold 'em, know when to fold 'em

Know when to walk away and know when to run”

Kenny Rogers, The Gambler, 1978

There is a phenomenon that is most striking in self-help business books. It goes something like this: You only know you are leading when you are generating some pushback. That is when you are on the edge of your authority. However, you must be careful not to generate too much pushback because that can be counterproductive and will take you down.

Critically, such books will never give a before-the-fact way of deciding whether the amount of pushback you are receiving is too much. That can only be examined in hindsight. However, this also means that all possible outcomes, success or failure, can be explained by the book—it is ‘unfalsifiable’ or impossible to prove wrong. This is the meaning of physicist Wolfgang Pauli’s famous quip that a theoretical physics paper he was presented with was ‘not even wrong’ and it is a hallmark of pseudoscience. In real science, theories and reputations are on the line when an experiment is conducted or a measurement made. In pseudoscience, nothing is at risk and its practitioners can relax, safe in the knowledge that their authority will never be challenged.

Pseudoscience is a perpetual risk to genuine scientists. Cognitive load theory has had its own brush with it. In the late 1990s, when germane load was considered to be a separate type of cognitive load, all possible experimental results could potentially be explained. If a procedure reduced cognitive load and led to more learning, then it must have reduced extraneous load and the theory was right. If it increased cognitive load and led to more learning, it must have increased germane load and the theory was also right. That’s a problem.

This has been resolved in recent years—to my satisfaction if not to that of all cognitive load theory’s critics—by subsuming germane load into intrinsic load and developing the concept of element interactivity to predict the conditions when raising or lowering load will be effective. This demonstrates that cognitive load theory researchers saw its unfalsifiability as a major problem and sought to do something about it. In other words, cognitive load theory researchers behaved as scientists. I fear that too many in the field of education would be pleased to have an unfalsifiable theory to luxuriate in. Why do I fear this? Because most theories about education are unfalsifiable. If you don’t believe me, tell me what evidence I could collect to prove Foucault is wrong about power relationships. I’ll wait.

Against this backdrop, there is the bizarre double standard that advocates of what has imperfectly become known as the ‘science of learning’ are held to.

Critics of the science of learning are fond of fallacies. One of these is the ecological fallacy. Typically, researchers will determine whether a particular teaching strategy will, on average, lead to more learning. We look for a difference in mean scores between an experimental and control group and then apply some statistical tests to see how meaningful that difference is. It does not tell us anything about the performance of specific individuals within those groups. However, critics like to imply that it does and so then set about refuting the idea that applying science of learning strategies will deliver certain outcomes for every individual—a pretty easy thing to refute.

This is the argument at the heart of Gert Biesta’s The Beautiful Risk of Education.

The problem with this argument is that nobody actually thinks outcomes are certain at the individual level. The science of learning is no more or less than a series of best bets. That’s why there are approaches such as Response to Intervention (or Multi-Tiered Systems of Support) that, depending on how you define it, sit within the science of learning and seek to catch those students who don’t make progress after exposure to the best bets.

However, despite placing the predictions of the science of learning under a distorting microscope, critics see no need whatsoever to subject their own methods to the same level of scrutiny.

One objection to the science of learning is its uniformity—we start out with the same strategy for everyone. Although not a totally accurate critique, it is true that it is possible to design and share ‘best bet’ lesson plans based on some science of learning strategies and this is what many of us are engaged in.

Critics claim this cannot possibly work because students are all individuals and teachers need to use their expertise to tailor instruction to those individuals.

However, if you question how we are meant to do this, you will get crickets in return. If you ask for criteria for deciding why one strategy would work better with one child and a different one with another, you are unlikely to even be answered with a description of two different strategies. Instead, it’s all just a lot of handwaving and appeals to teacher expertise.

Me: So give me an example of something I would see in a classroom that would cause me to use one strategy with one individual and an alternative strategy with a different individual.

Critic: You would have to use your expertise as a teacher.

Me: What does that look like then? How would I know?

Critic: Perhaps you don’t have the expertise. An expert teacher would know.

It is perfectly unfalsifiable. Like the naked emperor in the fairy tale, if we cannot see it then we are at fault. QED.

The science of learning has no problem answering such questions. Relatively simple tasks, such as memorising lists of words, or tasks that have become simple due to students having mastered them, lend themselves to intentionally increasing cognitive load by students completing part or all of the problem or task themselves. Relatively complex tasks, such as those with interrelated elements that are being learned by relative novices, will benefit from modelling and explanation.

These are falsifiable statements. We could design experiments to test them and potentially prove them wrong. They are therefore scientific statements.

The choice is not between the science of learning and the sciences, plural, of learning or whatever it is that critics are suggesting this week. The choice is between the science of learning and pseudoscience.