Curios of the week #25

Clippings, endnotes and other ephemera

Image via AI

---

It has not been a great week at my workplace. I do not intend to comment on any of that. This newsletter is a space for me away from work and a place to think about the issues that motivate me and, I hope, the issues you are interested in too. So, let’s jump straight in.

This week’s curios include misinformation, the release of NAPLAN testing data in Australia, a welcome policy reversal and much more.

Pithy open-access article of the week

A few years ago, I picked up the 2011 book, Uneducated Guesses: Using Evidence to Uncover Misguided Education Policies by Howard Wainer. Wainer is a statistician and although his comments pertain to the American education system, I learnt a lot from the book. Researching his more recent work for this post, I noticed he has written a book on ‘truthiness’ which looks like it will be a good read.

Why mention this? Well, Wainer appeared in my RSS feeds as co-author of a short, open-access piece in Educational Psychology Review. It is yet another article prompted by what is likely to turn out to be a seminal paper by Brady et al. that was published earlier this year and that calls into question the scientific nature of educational psychology research. Briefly, the Brady et al. thesis contends that true experiments are on the decline in favour of correlational research. This issue is then compounded by a misguided tendency to make causal claims from correlational research.

If you didn’t quite follow that, here is Robinson and Wainer’s take:

“What most refer to as causal modeling (e.g., structural equation modeling or SEM and a direct generalization of traditional factor analysis) is in fact not causal at all. SEM gives us only a model that conveys causal assumptions. Leaning heavily on such models requires also leaning very heavily on largely untestable, substantial, heroic assumptions. A recent typical example is a study by Clarke et al. (2023) that was promoted as follows in Science Daily: “Study of 600 UK teenagers suggests that having stronger self-awareness and sense of purpose may raise GCSE maths scores ‘by a couple of grades.’” The authors analyzed student questionnaire data collected at one time point using SEM. This “causal” conclusion is bogus. Raising self-awareness and sense of purpose is not likely to raise math performance. It is just as likely that increased math performance raises self-awareness. This “correlation is not causation” error should have been corrected in any introductory statistics or research methods course. It is sadly reminiscent of the self-esteem movement from the 1970s. The thought back then was that because self-esteem was positively correlated with student achievement, we could simply raise achievement by raising self-esteem. Unfortunately, efforts to raise students’ self-esteem not only did not increase student achievement, but in some cases, achievement decreased (Baumeister et al, 2003). The lessons learned from 50 years ago have apparently been forgotten.”

Sorry for the lengthy quote but I thought it was excellent. This is one of the many reasons I encourage young teachers who are interested in research to roll their sleeves up and do experiments.

NAPLAN results of the week

Where to start? It feels like everyone except me has had a say on this and it is an issue which, under normal circumstance, I would have commented on during the week.