Curios of the week #44

Clippings, endnotes and other ephemera

Image via AI

---

This week, we took the chance to walk Alfie, our cavoodle, around Ballarat’s beautiful botanical gardens wherein lies a curiosity of its own — an avenue populated with bronze busts of Australian prime ministers. We have been changing our prime ministers rather frequently at late and it takes a while to commission and receive a bronze bust. So, the botanical gardens are only updated as far as Malcolm Turnbull, with Scott Morrison and Anthony Albanese still to come. However, I have no doubt they will take their places soon.

This week’s Curios include a useful tip, horses races, some school inspectors, some bureaucrats and much more.

Causal claim of the week

Writing in the British Journal of Educational Psychology, Ingrid Obsuth and colleagues claim to have identified a causal relationship between being suspended or expelled from school and later health outcomes. This is a strong claim. Purists would argue that the only way to fully establish that exclusions cause later health problems would be to run an unethical randomised controlled trial were students who meet the criteria for exclusion are randomly assigned exclusion or some other outcome. Thankfully, Obsuth et al. have not done this.

So what is their claim based upon? Well, this is what they say they have done using data from the Longitudinal Study of Young People in England (LSYPE):

“…permanent exclusion is not randomly assigned—certain characteristics like gender, family background, ethnicity and SEN status make a student more likely to be excluded… To address this issue, we first used logistic regression to estimate each student's propensity score—their probability of being permanently excluded based on observed characteristics like gender, ethnicity, socioeconomic status, SEN, etc. Students with similar scores have similar chances of exclusion, regardless of whether they were actually excluded or not.”

This does not really work. Two students could have similar scores on gender, family background, ethnicity, special educational needs and so on, yet one of the students may be more violent than the other. The violent student may be more likely to be excluded and it may be factors related to violence that cause the poorer health outcomes. Or, rather than violence, it could be due to impulsiveness. Or a personality disorder. Or involvement with gangs or substance abuse. There are endless potential causes of exclusion that are not captured in this model.

Imagine a certain horse race where we observe that, over the years, three-year old colts have a greater probability of winning. If all you know about two horses is that they are three-year-old colts, you would assign them the same probability of winning the race. However, if you know one of the horses is faster than the other, it would be silly to assign them the same probability. Age and gender are relatively poor proxies for the important factor — how fast the horse is.

It would be absurd to insist that, regardless, all three-year-old colts have an equal chance of winning the race and so the outcome for such horses, winning or not, is essentially random. Yet, in essence, that is what this paper does with exclusions.

I find it hard to understand why this is not mentioned in the limitations section.

Open access article of the week

A while back, I contributed to a paper that was recently published. The paper was the third in a series of papers. The first paper, by Lin Zhang and colleagues, was an argument for why we should pay attention to all the evidence when making recommendations for science teaching methods, not just what the authors termed, ‘Program-based studies.’ This was followed by a rebuttal by de Jong and colleagues and, finally, a response to this rebuttal by Sweller et al. that I contributed to. The original Zhang et al. paper was initially paywalled. Now, that paywall has been removed and all three papers are open access. You can read them via these links:

1. There is an Evidence Crisis in Science Educational Policy — Lin Zhang, Paul A. Kirschner, William W. Cobern & John Sweller

2. Let's talk evidence – The case for combining inquiry-based and direct instruction — Ton de Jong, Ard W. Lazonder, Clark A. Chinn, Frank Fischer, Janice Gobert, Cindy E. Hmelo-Silver, Ken R. Koedinger, Joseph S. Krajcik, Eleni A. Kyza, Marcia C. Linn, Margus Pedaste, Katharina Scheiter, Zacharias C. Zacharia

3. Response to De Jong et al.’s (2023) paper “Let's talk evidence – The case for combining inquiry-based and direct instruction” — John Sweller, Lin Zhang, Greg Ashman, William Cobern, Paul A. Kirschner

Debate of the week

In recent years, I have shied away from airing my arguments on Twitter for fear of them being passé. It is perhaps too easy for those of us in the science of learning bubble to assume everyone has heard all the arguments already and there is little more to say.