Say what you like about Quality Teaching Rounds (QTR) — which is precisely what I am about to do — but you have to hand it to their PR department. They are a machine. Never, in the field of education has so much been spun by so many out of so little. This week reached a new height with QTR claiming as a win the success of a school that did the opposite of what it advocates.
If nothing else, it is impressive chutzpah.
What do I mean? Well, you can download the QTR Classroom Practice Guide here provided you are willing to submit your email address and details. This expands on the definition of quality teaching. It is based a lot on things like Bloom’s taxonomy and so advises teachers to focus on ‘higher-order thinking’ and the like — ideas that bear little relationship to what we now know from cognitive science. Instead, the whole loose package is derived from views expressed by an American researcher called Fred Newmann in the 1990s and will be familiar to anyone who trained at that time. It has then travelled to its current berth at the University of Newcastle in Australia via a previous project in Queensland.
The QTR model that supposedly captures ‘quality teaching’ is comprised of three dimensions and 18 elements. The ‘rounds’ part of the process is a system of peer lesson observations where teachers are supposed to score each other against a rubric based on these 18 elements. Imagine the extraordinary industry this must generate while noting that observers are likely to only be able to process four or five distinct items at a time.
One of these elements is ‘student direction’. Observers are required to assign the lowest score to a class where, “All aspects of the lesson are explicitly designated by the teacher,” and the highest score when, “Students determine many significant aspects of the lesson.” As such, the rubric is promoting a form of inquiry learning completely aligned with 1990s woo-woo-style teaching vibes. It is totally at odds with explicit teaching which, if observed, would gain the lowest possible score.
And yet, what’s this? A school in regional New South Wales has gone through an ‘incredible transformation.’ According to the ABC, what was once a very tough environment is now a much better place to study and work and standardised test scores have improved. That’s obviously a great story.
But how did they do it? They focused, at least in part, on adopting explicit teaching. This, apparently, is a huge win for QTR and the QTR researchers that the school brought in.
Eh?
I guess if QTR is all things to all people then maybe this makes sense. However, what possible value is it to a school if it encompasses opposites? What guidance does that provide?
Being generous, we might argue that at least all these peer observations will focus teachers on the process of teaching. We’ve all been in meetings where we discuss abstractions or matters out of a school’s control — we cannot cure poverty in the local community. So, will a program that forces us to focus on what we can control, in detail, be a bonus? Maybe, but I can’t help thinking there are plenty of rabbit holes in the rubric that teachers could fall into.
QTR is also keen to promote its supposed evidence base. I have addressed this more extensively before, but it is worth briefly reviewing why this is not what it seems.
Tests of statistical significance basically tell you the probability of obtaining the results you have obtained if the intervention had no effect. So, the lower this probability, the more inclined we might be to assume the intervention did have an effect. The common threshold we use is 0.05. In other words, if this probability is less that one-in-20, we tend to assume the intervention had an effect.
However, imagine you ran an experiment but measure 20 different outcomes. Even if your intervention had no effect, you would be likely to produce a statistically significant result on one of those outcomes.
QTR research measures loads of outcomes but then tends to report — in the media at least — only those that are statistically significant. I suspect these may arise by chance.
Take, for example, the loudly trumpeted result that QTR improves maths performance. If you look at the paper, you realise that there are two versions of QTR tested and outcomes in reading, science and a whole load of noncognitive assessments are also measured. Amongst this, there is only one positive result for just one of the versions of QTR in maths. All the rest are null results, apart from one of the noncognitive assessments where a control group outperformed QTR.
What about the loudly trumpeted result that QTR improves reading? Well that comes from a different study in Queensland. This study also measured maths performance but this time, found no effect. Why would QTR sometimes improve maths and sometimes not while also sometimes improving reading but sometimes not? It makes a lot more sense if these are false positives.
This conclusion is supported by the results of a similar study in Victoria that has not been loudly trumpeted. The lack of PR around this study may or may not be related to the fact that it found no significant effects of QTR on anything.
If you are a busy school leader then QTR is not worth the investment. It’s a stroke of evil genius to name it ‘quality teaching’ because that implies some objective criteria rather than the whims and prejudices of Fred Newmann and probably misleads a lot of people as a result.
Instead, the ‘quality’ in ‘quality teaching’ is like ‘quality’ in ‘quality butcher.’ It’s an assertion seeking to persuade rather than a statement of fact.
This seems remarkably similar to the current use of the "Danielson Framework" in many schools, districts, and entire states here in America. The Danielson Framework rates many elements of explicit instruction as "unsatisfactory" while giving the highest score to characteristics consistent with minimally-guided instruction. An example is that the "Distinguished" category (which is the highest ranking in the framework) states that "Students take initiative to collaborate in new or unplanned ways that further their learning"; "Students identify new learning opportunities and take the initiative to pursue them on their own"; "Students take initiative in using instructional materials and resources by adapting them appropriately for their own needs"; and on and on. Meanwhile, the lowest ranking ("Unsatisfactory") includes criteria such as asking questions "with only one response"; "students do not help create or arrange the space"; etc.
The Danielson Framework does have many good parts, like emphasizing building connections with students and taking into account prior knowledge. However, it seems to me that the framework assesses students more so than teachers, since so many "Distinguished" criteria describe students taking control of the classroom. Any teacher who is trying to maintain an orderly classroom and who uses a lot of explicit instruction inevitably gets a low score.
And with regards to PR...the Danielson Group pushes costly "professional development" to no end. I emailed the Danielson Group with a yes/no question about the copyright on the framework, and they insisted that I schedule a phone call with them. On that call, it took about ten minutes before they answered my question -- they kept trying to get me to purchase professional development materials for my school district, even though I repeatedly emphasized to them that I am a teacher -- not an administrator -- with no sway over what kind of professional development we use, and also that I am contacting them as an independent individual, rather than on behalf of my district. It was a bit surreal.
I was at a school 6 years wanting to do QTR. I saw the list of criteria for observations and immediately saw an issue. If I can keep 3 or 4 things in my focus at any one time how am I going to observe for all of these different things. Say I am observing an hour lesson there is no way. At best all I could think was we'd probably focus on what the observed wanted feedback on and fill in the rest after from memory or guessing