EDI and the Measurement of Merit

Across most of the world, the concept of academic merit tends to get operationalized as “best at passing a certain test taken by a limited group of people at a certain point in time.” A competition, in other words. For young people, it tends to involve passing an exam or set of exams, be they the American SAT, the Chinese gaokao, the Indian JEET, the French baccalauréat, etc. For aspiring professors, the competition is a little more subjective in the sense that there are no exams but rather a comparison of academic achievement more broadly, but in Western universities the criterion is mainly research quality and output.

There are a number of problems with these definitions of merit; more than I can cover in a single blog post. But the biggest one is that they mainly measure someone’s position at a given point in time (either against some kind of absolute standard in the case of examinations, or head-to-head competitions in the case of academic hiring), rather than attempt to assess future potential or even look at “distance travelled” (that is, to look at results in the context of one’s social origins). And that, to be blunt, privileges the privileged.

There are innumerable ways in which privilege (which is partially but not entirely about race and other immutable factors) can get transmitted from one generation to another. Children from more privileged families get read to more often, have more books around the house, are exposed to a wider set of experiences, etc., etc. Evidence from the OECD’s PISA suggests that by age 15, Canadian students from the upper-income quartile have a cognitive advantage equal to about one year of school over students from the bottom-income quartile (and this is comparatively pretty good; Canada has one of the narrowest such gaps of any country in the survey). if you have competitions for a restricted set of opportunities at a given age—say entry to top universities, or scholarships thereto—and you call the winners of those competitions ‘meritorious,’ then what you are doing to some degree is just “merit-washing”: recycling privilege and calling it merit (I am indebted to Anjantha Subramanian and her book The Caste of Merit: Engineering Education in India for the term). And if those scholarships, plus financial support over many years of graduate school are what is required to become a tenured professor, then: surprise! That too, will lead to the competition winners coming disproportionately from privileged backgrounds (this is true even when tuition is not a barrier and financial aid is generous, as this data from Norway shows).

That this definition of merit is ridiculous is widely understood but no one articulates it properly. You get books railing against “meritocracy” (like The Meritocracy Trap by Daniel Markovitz) which do not actually question the notion that the best people should do the most important or difficult jobs so much as question how merit is assessed. The closest anyone came to nailing the issue was, of all people, David Brooks (I know, this surprised the hell out of me, too) in his recent Atlantic article entitled How the Ivy League Broke America, but even here the understanding is somewhat inadequate. Brooks correctly identifies existing U.S. definitions of “merit” as unidimensional and that the game is “rigged” (his word) in favour of the wealthy, but only suggests remedies to the former problem, not really understanding that no matter what the definition of merit is, rich kids are going to have the inside track.

How does EDI come into all this? Well, it’s because EDI also calls attention to the fact that our definition of merit, which disproportionately rewards people of certain backgrounds, is a deficient one. And it at least hints at the right kind of solutions; in particular, that we need to think about outcomes in terms of origins, of taking the notion of “distance travelled” a lot more seriously than we currently do. At their best, these critiques help us get to what I at least think is a better definition of merit, one which does a better job of measuring individual effort and determination than one which ignores origins altogether.

The challenge is that EDI does all this by proxy. EDI schemes to re-jig merit do not usually look at distance travelled directly; rather, they use race or other immutable characteristics as proxies. These are not necessarily terrible proxies, but they are proxies nonetheless. They leave open two possibilities: first, that benefits will accrue only to elites among these less-privileged groups, and second, that the financially disadvantaged among the most privileged groups (let’s call them “whites” for clarity) will find themselves doubly disadvantaged.

Now it is not obvious (to me at least) that the overall results of such a system are any worse than the overall results of the current system. You gain a little bit of equity in one direction and (perhaps) lose it in another. But there are winners and losers when switching from one system to another and the losers tend to scream louder than the winners.

In an ideal world, of course, one would be able to measure everyone individually by distance travelled, without the use of proxies. That way, “elites” from disadvantaged groups would not be unduly rewarded, and financially disadvantaged whites’ underprivileged position would be recognized. There would still be screaming, of course—people who were in danger of losing their position of privilege would still claim that a context-free, single-point-in-time definition of merit is “better” and “more objective” than a context-dependent one (this is more or less the position taken by the Students for Fair Admissions in the Harvard admissions case decided by the US Supreme Court in 2023). But it would have fewer drawbacks than other schemes which measure disadvantage via proxies.

Why don’t we do that? Well, I would argue it is for two reasons. The first is simply that using proxies to measure disadvantage is a whole heck of a lot cheaper than measuring it at an individual level. With proxies, you can reduce disadvantage to a set of categories that can be indicated by a tick in a box, something that reduces complexity and obviates the need to treat each case individually. 

But the second and probably more important reason is that distance travelled is not an entirely straightforward and measurable proposition. It is by no means impossible to create methodologies to look for it: the Loran Scholarships and McGill’s McCall-MacBain scholarships both train assessors to look for precisely this (which is a very good reason why the former is so good at picking future Rhodes Scholars). But the problem is that there is no hard-and-fast algorithm here. You have to put selectors in a position where they can exercise judgment. And frankly, in an increasingly low-trust society, that’s hard to do (Phillip K. Howard’s Everyday Freedom: Designing the Framework for a Flourishing Society is very good on the unfree consequences of depriving administrators of the ability to exercise judgement).

And so here’s the thing: if you don’t want to measure disadvantage individually because you are too cheap to do so, and/or you can’t allow people freedom of judgement in assessing disadvantage for the purpose of measuring distance travelled, then what you’re left with as options are measurement by proxy, or settling for a definition of merit that unabashedly favours the members of the lucky sperm club. There is not really a fourth option.

Which would you pick?

 

Posted in

Leave a Reply

Your email address will not be published. Required fields are marked *

Search the Blog

Enjoy Reading?

Get One Thought sent straight to your inbox.
Subscribe now.