New evidence in favour of standardised tests

It is now time for those proposing to scrap standardised testing to start supporting their alternatives with evidence.

I have been arguing for a number of years that we should not rush to scrap standardised tests and replace them with alternative approach to allocating places at university (e.g. here and here). This argument is a little too subtle for some of my critics so let’s rehearse it.

Standardised tests are not fair. They advantage students with more resources. A student from a wealthy family will be raised in an environment that privileges more test-relevant language and ideas. A student from a wealthy family who is struggling in preparation for a standardised test may be able to access a tutor. Moreover, that tutor may specialise in preparing students for this test and may have superior knowledge about the kinds of questions that are likely to be asked.

So, standardised tests are not equitable. Please do not accuse me of ever suggesting they are because I am explicitly stating that they are not. However, a student still has to sit alone in a room and draw on their own cognitive resources in the test itself. When compared to the alternatives, this creates more of a level playing field and makes it easier for less advantaged students to compete.

How can this be?

The only alternatives I can think of to standardised tests - such as portfolios, ongoing project work or grades and the factoring-in of non-cognitive skills or community service, are far more gameable by advantaged students and their families than standardised tests. Is your child doing a physics project? Hire a tutor to ‘help’ with the write-up. Is your child in need of boosting their community work portfolio? Pull strings at the local church so that your child may help-out in a soup kitchen while the kids of the less advantaged take evening jobs at McDonalds. Does your child need an A-grade average? Complain to the school about the quality of teaching the very instant a B+ is spotted on the horizon. And so on.

This logic is hard to deny and so I have argued that proponents of alternatives to standardised tests need to describe their alternatives in detail so that we can measure them against this logic. It may be that someone, somewhere, can think of a plan that would be more equitable than tests. Yet opponents of standardised tests rarely do this. The assumption is that merely tearing down the existing regime will be enough to usher in a new dawn of equity and freedom - an assumption tested throughout history in events such as the French revolution or the invasion of Iraq.

I have been arguing the case for standardised tests since before Sunday’s Wall Street Journal editorial that drew my attention to a new piece of research relevant to this argument. It is rare that I discover education research via the Wall Street Journal and that gives you a sense of its wider significance.

The researchers take an interesting approach. They use computers to mark the essays submitted by prospective candidates to study at the University of California. Although it does not explicitly state this, I assume these essays are not written under exam conditions and are simply a supporting document appended to the application similar to the personal statement I had to make for my university application. I am a sceptic of using computers to mark essays at the individual level, but they seem well-suited to this kind of research. The researchers also had access to students’ scores in the SAT standardised test as well as household income data.

The researchers found that both SAT scores and essay quality were positively correlated with the level of household income - richer kids did better on both. However, in line with the argument I have been making, essays were correlated more to household income than SAT scores. Standardised tests are inequitable but the alternative the researchers were able to compare them to was even more inequitable.

Of course, this is a comparison of standardised tests to one alternative only. You can perhaps explain it away by arguing that your favoured model of portfolios or whatever is not the same as an admissions essay. But if you do, given the strength of the logical argument and the empirical evidence we have, I think the duty is yours to demonstrate why your plan won’t make things worse.