Analogias/Memes

Is this the silliest scoring system ever?

In 2020, for the first time, the Olympics welcomed the sport of rock climbing. Alas, it wasn’t the warmest welcome, because for the three highly disparate events — speed climbing, lead climbing, and bouldering — the sport was allocated just one set of medals.

How to distribute one gold medal for three distinct achievements?

The committee hit on a peculiar solution (which I learned about from one of my fabulous students at Macalester College). In each competition, the competitors were ranked from 1st to 8th, and then those rankings were multiplied together. Lowest product wins.

Is this the silliest scoring system ever?

More predictable, and less remarkable, would have been to add the rankings. It would still have been a bit troubled — such an approach can exaggerate tiny absolute differences (for example, if I beat you by a hundredth of a second), or suppress huge ones (for example, if you beat me by a full ten minutes). Pro tip: if you’re aggregating scores, wait until the end to collapse them down to rank-order.

But that weirdness is nothing compared to the effects of multiplication, which “cares” much more about differences at the top than differences at the bottom. Thus, 1st is much better than 2nd, but 7th is scarcely better than 8th.

I marveled at this oddity to my father, and he pointed out an even stranger effect: whether you outperformed me, or vice versa, depends on how other people performed! For example, say I finish 1st, 1st, and 7th, while you finish 2nd, 2nd, and 2nd. By a narrow margin, I take the gold, and you take the silver.

But wait! A secret wizard enters the competition at the last moment, and finishes 1st, 1st, and 1st. I am bumped to 2nd, 2nd, and 8th, and you to 3rd, 3rd, and 3rd. The wizard now takes the gold… and suddenly, without lifting a finger, you have retroactively become better than me, retaining the silver while I drop to bronze.

But hold on again! It turns out the wizard cheated in bouldering (the third competition), and is disqualified in that ranking.

So, do we keep the wizard’s ranking in the other two competitions, and simply ignore him for the final ranking? If so, your scores (3rd, 3rd, and 2nd) defeat mine (2nd, 2nd, 7th).

Or do we eliminate the wizard altogether? If so, we’re back to the original scenario, and my scores (1st, 1st, 7th) defeat yours (2nd, 2nd, 2nd).

The gold medal is dancing on the head of a pin, entirely dependent on the question of when and how a disqualified competitor is removed from the scoreboard.

And yet, guess what? Such theoretical oddities didn’t wind up mattering . The 2020 medals were awarded without controversy.

It reminds me of an odd pattern I’ve observed, when poking at spreadsheets of student scores at the end of a semester, before assigning final grades: Weighted averages are often surprisingly insensitive to re-weighting.

How you combine the sub-scores may seem like a matter of real importance. Perhaps even a matter of justice. Should good quizzes compensate for spotty tests? Should a great final exam make up for missed homework? And yet, trying the calculation various ways, you rarely see a major change. Sometimes, only one or two students are affected at all. The data is all more correlated than you’d expect.

So maybe the Olympics didn’t get it so wrong?

No. I cannot but laugh and grimace at the multiply-the-rankings system. It offends my mathematical sensibilities, in the same way that bad grammar offends some people’s ears — but the preference, I must confess, is largely a matter of aesthetics.