I’d talked about the Bill James presidential polls before, and he’s running a similar set of polls for HoF candidates that have a similar kind of issue. For whatever reason, this time around I realized that his assumptions are the same as the Independent Chip Model (ICM) for poker tournament equity based on stack sizes. If you aren’t familiar with that, then assume we have 4 players, A with 40 chips, B with 30, C with 20, and D with 10. Everything else being equal, A should win 40% of the time. The ICM goes further than that, and for predicting the probability of second place, uses calculations of the form

Assuming A wins, what are the odds B gets second: Remove A’s chips, and then B has 30/(30+20+10)=50% of the remaining chips, so B is 50% to get second *assuming A wins*.

If you don’t see the analogy yet, the ICM takes as input the stack sizes, which are identical to the probability of finishing first, and uses the first-place percentages to calculate the results of every poll subset. Bill James runs polls and uses the (first-place) percentages to calculate every head to head subset. The ICM assumption that to calculate the result between B/C/D, you just ignore A’s chips, is equivalent to the Bill James assumption that A’s support, if A is not an option, will break evenly among B/C/D based on their poll percentage.

That assumption doesn’t hold in politics, for reasons discussed before, and it doesn’t hold for HoF voting because different people prefer different player types even beyond the roid/no roid dichotomy. As it stands, in the linked poll, Beltre would almost certainly be the leader in 4th-place rankings with ~70% 4th place votes and an average finishing position near or even above 3.0. He’d likely get stomped in every head-to-head matchup, lose the overall rating, etc, but by using only first-place%, he looks like the clear second-preferred candidate, which is obviously very, very wrong.

It could have gotten even worse if Bonds didn’t dominate the roid vote. Let’s say we had a different poll, Beltre 30%, Generic Roidmonster 1 (23.33%), Generic Roidmonster 2 (23.33%), Generic Roidmonster 3 (23.33%) where (if people ranked 1-4) the Roidmonsters were ranked randomly 1-3 or 2-4 depending on whether or not the voter was a never-roider or not. In this case, each Roidmonster would have an average finishing position of 2.3 (Beltre 3.1) and would win the head-to-head with Beltre 70-30… yet Beltre wins the poll only counting first-place votes.

It’s clear that the ICM/James assumptions are violated, and violated to where they’re nowhere close to reality, in polls like this. In the same poll without Bonds, the Bonds votes would go overwhelmingly to Clemens and A-Rod, even though ICM/James assume a majority would go to Beltre. Aggregating sets of votes is going to keep a lot of the same problems because the vote share of any two people in a poll is (well, can be) strongly dependent upon who else is in the poll. The ICM/James model are built on the assumption of independence there, but it’s clearly not close to true in HoF voting or in politics.