Arrow's Theorem, voting systems, and the folk power of impossibility theorems

[mmcirvin]

Addendum #2: I apparently completely mischaracterized Patrick's actual position here; I thought he was impugning Arrow for even proving the theorem, whereas it appears his concerns are more about its misuse. I both apologize to Patrick and would request that people cool it a little in the comments.

nhw makes a provocatively titled post called "Why Arrow's Theorem is wrong", and pnh links to it on Making Light with title text implying that mere interest in it has dark motivations: "Oddly enough, there's always a ready audience for intellectuals willing to argue that democracy is impossible".

Sorry, but I have to call shenanigans on this--to some degree.

First: nhw is saying that the theorem's stated desiderata are not really necessary for a good voting system. A theorem isn't wrong if its assumptions are inapplicable; its assumptions are just inapplicable. The reason I want to be pedantic here is that I'm pretty sure there's an extensive crank literature attempting to prove Arrow's Theorem literally incorrect, just as there's an extensive crank literature on squaring the circle or disproving Fermat's Last Theorem. I've seen people propose all manner of voting systems that they claim violate Arrow's theorem through some loophole or other and therefore prove it wrong; of course, they're just systems that violate one or other of the proposed requirements in a way that's considered harmless or irrelevant.

Second: While it's possible to argue that universality, independence of irrelevant alternatives, and monotonicity are not really necessary, it's nevertheless true that people have complained about these things as problems in real-world voting systems, and I think it's therefore still an interesting result that you can't have them all in a reasonable system, whether or not one is an evil antidemocrat. It's not universally true that failure to organize a winning coalition is blamed on the losers. People constantly grouse about the prevalence of "spoiler effects" in American elections, which is the "independence of irrelevant alternatives" condition. It's usually claimed that egregious failure to satisfy independence of irrelevant alternatives in presidential elections is the reason why the US is locked into a two-party system, which is held to be very bad by advocates of third parties. It's common for libertarians and left progressives to claim that this is the main impediment to constructive change in the American political system. There's more to it than that, I think, but it's certainly part of the story.

That said, it's true that, as I've said before, impossibility theorems like this can have a sort of folk-theorem power that exceeds their real applicability, and this is a real danger. My understanding is that Arrow's theorem relies heavily on things that rarely happen in, say, American elections, such as the existence of "Condorcet cycles" in which a majority sincerely prefers candidate A to candidate B, a majority prefers B to C, but a majority also prefers C to A. If you accept that you're not going to handle Condorcet cycles gracefully, you can do really well with everything else--a lot better, in fact, than the systems typically in use in the US generally do. So nhw has a point--I'm mostly arguing with the post title here. The true lesson is that when you're confronted with an impossibility theorem saying that something universally desired cannot be had, you should always read the fine print.

But Arrow's Theorem is also applicable to things other than democratic government. samantha2074 is a big fan of figure skating, and I know there's been a lot of dissent and upheaval in recent years concerning scoring systems. The prevalence of "flip-flops" causes a lot of grief: situations in which contestant C scores in such a manner as to reverse the relative rankings of contestants A and B. Usually, people blame the system for this. Phenomena such as Condorcet cycles are pretty common in figure skating, much more common than in political elections, and I don't think it's evil or antidemocratic to note that you're always going to have to settle for a system that sometimes has some sort of paradoxical result.

Addendum: Sam reminds me that "flip-flops" were a feature of the pre-Code of Points ordinal-based system, and are not present in the CoP. However, I do recall that the flip-flip phenomenon was widely discussed during the debate over changing the system.

Comments

[pnh.livejournal.com]

How the heck do you read my tossed-off remark as implying that "mere interest" in voting systems has "dark motivations"? I'm interested in voting systems. I docnote that market forces have an effect on ideas as well as on the movements of commodities. That's a long way from saying that nobody ought to be interested in ideas.

[ronebofh.livejournal.com]

How the heck do you read Matt's remark as implying that you implied that nobody ought to be interested in ideas?

[pnh.livejournal.com]

It's the plain meaning of what he said: "implying that mere interest in it has dark motivations."

[ronebofh.livejournal.com]

I'd call that a bit of a stretch. To me, your title text has the perhaps unfortunate ability to be read straight as well as ironically, recalling that foible of American culture that is anti-intellectualism combined with the specious trend of packaging false intellectuals to support all manner of ideas. It's my impression that your succinctness left too much open to interpretation (or maybe i'm, once again, projecting).

[mskala.livejournal.com]

My impression is that when someone starts a sentence "Oddly enough," and then goes on to talk about a identifiable and often-political group - such as "intellectuals" - the writer of the sentence is pretty often trying to imply something about someone's dark motivations while maintaining plausible deniability that they couldn't maintain if they said it outright.

Statement: "Oddly enough, we've only heard Democrats complaining about the voting irregularities." Implication: they're complaining because they're Democrats, they have some other agenda than just wanting fair elections, and the complaints should be discounted to at least some extent for that reason.

Statement: "Oddly enough, Ayn Rand's point of view is especially popular among teenage males." Implication: they like that point of view because they're teenage males, have some interest or intellectual limitation unrelated to whatever real issues Rand purported to address, and their support should be discounted to at least some extent for that reason.

Statements like those sound sneaky and dishonest to people who recognize how they're constructed. Your statement sounds like one. Even if you didn't intend it to be, I'm not at all surprised that it may have been misinterpreted as having that kind of implication.

[pnh.livejournal.com]

Goodness, "sneaky and dishonest"! What epithets will fly next?

Anyway, bollocks to "deniability." I'm saying straight out that the current "marketplace of ideas" offers premiums to intellectuals with an anti-democratic argument to make. That's not to say that Arrow is wrong, or that Arrow is motivated by eee-vile; only that I bring a (finite, but real) dollop of skepticism to work whose implications are so convenient to established power. If you want to regard that skepticism as "sneaky and dishonest," feel free. I'm also a Rootless Cosmopolitan, while you're at it.

More to the point in this discussion, I never said anything even remotely similar to the claim that "mere interest" in voting systems is somehow discreditable. And if I had meant to say any such thing, I would have said it straightforwardly.

[mskala.livejournal.com]

You asked a question. I answered it politely and carefully because I thought you actually wanted to know the answer. Now you act like I insulted you. Next time I won't make the mistake of treating you with respect.

[pnh.livejournal.com]

Dear me. I didn't mean to cause you such distress.

[factitiouslj.livejournal.com]

Your title text gave me the same impression that you were characterizing and criticizing Arrow's Theorem folks as "antidemocratic intellectuals" or some such.

This may have been a misunderstanding on my part, but if so, it's apparently a misunderstanding not confined to my reactions alone. mskala's explanation of why it came across that way seems like a sensible one. If you're upset at having been misread, it might be worth rereading his posts a bit more charitably.

[smashingstars.livejournal.com]

This is an excellent post, Matt. You're well aware that I have The Dumb so I can't add more to the discussion, except perhaps a brief reminder of your own words:

"I'm mostly arguing with the post title here. The true lesson is that when you're confronted with an impossibility theorem saying that something universally desired cannot be had, you should always read the fine print."

You know, just in case anyone may have forgotten your actual, stated point(s) in this entry.

[dr-strych9.livejournal.com]

Thanks for this post. I followed pnh's link to nhw's post and my reaction when I got there was much less reasonable than yours, so I kept it to myself.

One of the things that confused me about pnh's comment was this: who is the "intellectual" in question? Kenneth Arrow or nhw? I don't think Arrow was arguing that democracy is impossible, so it must have been nhw, right? But I don't see how nhw could be doing that either— he just seems to be, as you politely note, questioning whether the math is as widely applicable as some might believe (though, I'm not convinced the belief is very widespread). All in all, I was highly annoyed with both of them.

p.s. Here's the condensed, more polite version of my redacted reaction: "Oddly enough, too, there's always a ready audience for technologists willing to argue that Gödel's Incompleteness Theorem is wrong." I fear even that's too impolite, so I'm just going to shut up about it.

[smashingstars.livejournal.com]

I'm going to ask something silly: has PNH edited the text he linked to nhw's post with? I can't find anything on Making Light like what you quoted, although I do see PNH's sidelight link to the post.