Consistency criterion

A voting system is consistent if, whenever the electorate is divided (arbitrarily) into several parts and elections in those parts garner the same result, then an election of the entire electorate also garners that result. Smith calls this property separability and Woodall calls it convexity.

It has been proven a ranked voting system is "consistent if and only if it is a scoring function", i.e. a positional voting system. Borda count is an example of this.

The failure of the consistency criterion can be seen as an example of Simpson's paradox.

As shown below under Kemeny-Young, passing or failing the consistency criterion can depend on whether the election selects a single winner or a full ranking of the candidates (sometimes referred to as ranking consistency); in fact, the specific examples below rely on finding single winner inconsistency by choosing two different rankings with the same overall winner, which means they do not apply to ranking consistency.

Examples

Copeland

This example shows that Copeland's method violates the consistency criterion. Assume five candidates A, B, C, D and E with 27 voters with the following preferences:

PreferencesVoters
A > D > B > E > C3
A > D > E > C > B2
B > A > C > D > E3
C > D > B > E > A3
E > C > B > A > D3
A > D > C > E > B3
A > D > E > B > C1
B > D > C > E > A3
C > A > B > D > E3
E > B > C > A > D3

Now, the set of all voters is divided into two groups at the bold line. The voters over the line are the first group of voters; the others are the second group of voters.

First group of voters

In the following the Copeland winner for the first group of voters is determined.

PreferencesVoters
A > D > B > E > C3
A > D > E > C > B2
B > A > C > D > E3
C > D > B > E > A3
E > C > B > A > D3

The results would be tabulated as follows:

 X A B C D E Y A [X] 9 [Y] 5 [X] 6 [Y] 8 [X] 3 [Y] 11 [X] 6 [Y] 8 B [X] 5 [Y] 9 [X] 8 [Y] 6 [X] 8 [Y] 6 [X] 5 [Y] 9 C [X] 8 [Y] 6 [X] 6 [Y] 8 [X] 5 [Y] 9 [X] 8 [Y] 6 D [X] 11 [Y] 3 [X] 6 [Y] 8 [X] 9 [Y] 5 [X] 3 [Y] 11 E [X] 8 [Y] 6 [X] 9 [Y] 5 [X] 6 [Y] 8 [X] 11 [Y] 3 Pairwise election results (won-tied-lost): 3-0-1 2-0-2 2-0-2 2-0-2 1-0-3
• [X] indicates voters who preferred the candidate listed in the column caption to the candidate listed in the row caption
• [Y] indicates voters who preferred the candidate listed in the row caption to the candidate listed in the column caption

Result: With the votes of the first group of voters, A can defeat three of the four opponents, whereas no other candidate wins against more than two opponents. Thus, A is elected Copeland winner by the first group of voters.

Second group of voters

Now, the Copeland winner for the second group of voters is determined.

PreferencesVoters
A > D > C > E > B3
A > D > E > B > C1
B > D > C > E > A3
C > A > B > D > E3
E > B > C > A > D3

The results would be tabulated as follows:

 X A B C D E Y A [X] 6 [Y] 7 [X] 9 [Y] 4 [X] 3 [Y] 10 [X] 6 [Y] 7 B [X] 7 [Y] 6 [X] 6 [Y] 7 [X] 4 [Y] 9 [X] 7 [Y] 6 C [X] 4 [Y] 9 [X] 7 [Y] 6 [X] 7 [Y] 6 [X] 4 [Y] 9 D [X] 10 [Y] 3 [X] 9 [Y] 4 [X] 6 [Y] 7 [X] 3 [Y] 10 E [X] 7 [Y] 6 [X] 6 [Y] 7 [X] 9 [Y] 4 [X] 10 [Y] 3 Pairwise election results (won-tied-lost): 3-0-1 2-0-2 2-0-2 2-0-2 1-0-3

Result: Taking only the votes of the second group in account, again, A can defeat three of the four opponents, whereas no other candidate wins against more than two opponents. Thus, A is elected Copeland winner by the second group of voters.

All voters

Finally, the Copeland winner of the complete set of voters is determined.

PreferencesVoters
A > D > B > E > C3
A > D > C > E > B3
A > D > E > B > C1
A > D > E > C > B2
B > A > C > D > E3
B > D > C > E > A3
C > A > B > D > E3
C > D > B > E > A3
E > B > C > A > D3
E > C > B > A > D3

The results would be tabulated as follows:

 X A B C D E Y A [X] 15 [Y] 12 [X] 15 [Y] 12 [X] 6 [Y] 21 [X] 12 [Y] 15 B [X] 12 [Y] 15 [X] 14 [Y] 13 [X] 12 [Y] 15 [X] 12 [Y] 15 C [X] 12 [Y] 15 [X] 13 [Y] 14 [X] 12 [Y] 15 [X] 12 [Y] 15 D [X] 21 [Y] 6 [X] 15 [Y] 12 [X] 15 [Y] 12 [X] 6 [Y] 21 E [X] 15 [Y] 12 [X] 15 [Y] 12 [X] 15 [Y] 12 [X] 21 [Y] 6 Pairwise election results (won-tied-lost): 2-0-2 3-0-1 4-0-0 1-0-3 0-0-4

Result: C is the Condorcet winner, thus Copeland chooses C as winner.

Conclusion

A is the Copeland winner within the first group of voters and also within the second group of voters. However, both groups combined elect C as the Copeland winner. Thus, Copeland fails the consistency criterion.

Instant-runoff voting

This example shows that Instant-runoff voting violates the consistency criterion. Assume three candidates A, B and C and 23 voters with the following preferences:

PreferencesVoters
A > B > C4
B > A > C2
C > B > A4
A > B > C4
B > A > C6
C > A > B3

Now, the set of all voters is divided into two groups at the bold line. The voters over the line are the first group of voters; the others are the second group of voters.

First group of voters

In the following the instant-runoff winner for the first group of voters is determined.

PreferencesVoters
A > B > C4
B > A > C2
C > B > A4

B has only 2 votes and is eliminated first. Its votes are transferred to A. Now, A has 6 votes and wins against C with 4 votes.

1st2nd
A46
B2
C44

Result: A wins against C, after B has been eliminated.

Second group of voters

Now, the instant-runoff winner for the second group of voters is determined.

PreferencesVoters
A > B > C4
B > A > C6
C > A > B3

C has the fewest votes, a count of 3, and is eliminated. A benefits from that, gathering all the votes from C. Now, with 7 votes A wins against B with 6 votes.

1st2nd
A47
B66
C3

Result: A wins against B, after C has been eliminated.

All voters

Finally, the instant runoff winner of the complete set of voters is determined.

PreferencesVoters
A > B > C8
B > A > C8
C > A > B3
C > B > A4

C has the fewest first preferences and so is eliminated first, its votes are split: 4 are transferred to B and 3 to A. Thus, B wins with 12 votes against 11 votes of A.

1st2nd
A811
B812
C7

Result: B wins against A, after C is eliminated.

Conclusion

A is the instant-runoff winner within the first group of voters and also within the second group of voters. However, both groups combined elect B as the instant-runoff winner. Thus, instant-runoff voting fails the consistency criterion.

Kemeny-Young method

This example shows that the Kemeny–Young method violates the consistency criterion. Assume three candidates A, B and C and 38 voters with the following preferences:

Group PreferencesVoters
1st A > B > C7
B > C > A6
C > A > B3
2nd A > C > B8
B > A > C7
C > B > A7

Now, the set of all voters is divided into two groups at the bold line. The voters over the line are the first group of voters; the others are the second group of voters.

First group of voters

In the following the Kemeny-Young winner for the first group of voters is determined.

PreferencesVoters
A > B > C7
B > C > A6
C > A > B3

The Kemeny–Young method arranges the pairwise comparison counts in the following tally table:

Pairs of choices Voters who prefer
XYX over YNeitherY over X
AB1006
AC709
BC1303

The ranking scores of all possible rankings are:

Preferences1 vs 21 vs 32 vs 3Total
A > B > C1071330
A > C > B710320
B > A > C613726
B > C > A136928
C > A > B931022
C > B > A39618

Result: The ranking A > B > C has the highest ranking score. Thus, A wins ahead of B and C.

Second group of voters

Now, the Kemeny-Young winner for the second group of voters is determined.

PreferencesVoters
A > C > B8
B > A > C7
C > B > A7

The Kemeny–Young method arranges the pairwise comparison counts in the following tally table:

Pairs of choices Voters who prefer
XYX over YNeitherY over X
AB8014
AC1507
BC7015

The ranking scores of all possible rankings are:

Preferences1 vs 21 vs 32 vs 3Total
A > B > C815730
A > C > B1581538
B > A > C1471536
B > C > A714728
C > A > B715830
C > B > A1571436

Result: The ranking A > C > B has the highest ranking score. Hence, A wins ahead of C and B.

All voters

Finally, the Kemeny-Young winner of the complete set of voters is determined.

PreferencesVoters
A > B > C7
A > C > B8
B > A > C7
B > C > A6
C > A > B3
C > B > A7

The Kemeny–Young method arranges the pairwise comparison counts in the following tally table:

Pairs of choices Voters who prefer
XYX over YNeitherY over X
AB18020
AC22016
BC20018

The ranking scores of all possible rankings are:

Preferences1 vs 21 vs 32 vs 3Total
A > B > C18222060
A > C > B22181858
B > A > C20202262
B > C > A20201656
C > A > B16181852
C > B > A18162054

Result: The ranking B > A > C has the highest ranking score. So, B wins ahead of A and C.

Conclusion

A is the Kemeny-Young winner within the first group of voters and also within the second group of voters. However, both groups combined elect B as the Kemeny-Young winner. Thus, the Kemeny–Young method fails the consistency criterion.

Ranking consistency

The Kemeny-Young method satisfies ranking consistency; that is, if the electorate is divided arbitrarily into two parts and separate elections in each part result in the same ranking being selected, an election of the entire electorate also selects that ranking.

Informal proof

The Kemeny-Young score of a ranking ${\displaystyle {\mathcal {R}}}$ is computed by summing up the number of pairwise comparisons on each ballot that match the ranking ${\displaystyle {\mathcal {R}}}$. Thus, the Kemeny-Young score ${\displaystyle s_{V}({\mathcal {R}})}$ for an electorate ${\displaystyle V}$ can be computed by separating the electorate into disjoint subsets ${\displaystyle V=V_{1}\cup V_{2}}$ (with ${\displaystyle V_{1}\cap V_{2}=\emptyset }$), computing the Kemeny-Young scores for these subsets and adding it up:

${\displaystyle {\text{(I)}}\quad s_{V}({\mathcal {R}})=s_{V_{1}}({\mathcal {R}})+s_{V_{2}}({\mathcal {R}})}$.

Now, consider an election with electorate ${\displaystyle V}$. The premise of the consistency criterion is to divide the electorate arbitrarily into two parts ${\displaystyle V=V_{1}\cup V_{2}}$, and in each part the same ranking ${\displaystyle {\mathcal {R}}}$ is selected. This means, that the Kemeny-Young score for the ranking ${\displaystyle {\mathcal {R}}}$ in each electorate is bigger than for every other ranking ${\displaystyle {\mathcal {R}}'}$:

{\displaystyle {\begin{aligned}{\text{(II)}}\quad \forall {\mathcal {R}}':{}&s_{V_{1}}({\mathcal {R}})>s_{V_{1}}({\mathcal {R}}')\\{\text{(III)}}\quad \forall {\mathcal {R}}':{}&s_{V_{2}}({\mathcal {R}})>s_{V_{2}}({\mathcal {R}}')\end{aligned}}}

Now, it has to be shown, that the Kemeny-Young score of the ranking ${\displaystyle {\mathcal {R}}}$ in the entire electorate is bigger than the Kemeny-Young score of every other ranking ${\displaystyle {\mathcal {R}}'}$:

${\displaystyle s_{V}({\mathcal {R}})\ {\stackrel {(I)}{=}}\ s_{V_{1}}({\mathcal {R}})+s_{V_{2}}({\mathcal {R}})\ {\stackrel {(II)}{>}}\ s_{V_{1}}({\mathcal {R}}')+s_{V_{2}}({\mathcal {R}})\ {\stackrel {(III)}{>}}\ s_{V_{1}}({\mathcal {R}}')+s_{V_{2}}({\mathcal {R}}')\ {\stackrel {(I)}{=}}\ s_{V}({\mathcal {R}}')\quad q.e.d.}$

Thus, the Kemeny-Young method is consistent with respect to complete rankings.

Majority Judgment

This example shows that majority judgment violates the consistency criterion. Assume two candidates A and B and 10 voters with the following ratings:

Candidate Voters
AB
ExcellentFair3
PoorFair2
FairPoor3
PoorFair2

Now, the set of all voters is divided into two groups at the bold line. The voters over the line are the first group of voters; the others are the second group of voters.

First group of voters

In the following the majority judgment winner for the first group of voters is determined.

Candidates Voters
AB
ExcellentFair3
PoorFair2

The sorted ratings would be as follows:

Candidate
 ↓ Median point
A
B

Excellent   Good   Fair   Poor

Result: With the votes of the first group of voters, A has the median rating of "Excellent" and B has the median rating of "Fair". Thus, A is elected majority judgment winner by the first group of voters.

Second group of voters

Now, the majority judgment winner for the second group of voters is determined.

Candidates Voters
AB
FairPoor3
PoorFair2

The sorted ratings would be as follows:

Candidate
 ↓ Median point
A
B

Excellent   Good   Fair   Poor

Result: Taking only the votes of the second group in account, A has the median rating of "Fair" and B the median rating of "Poor". Thus, A is elected majority judgment winner by the second group of voters.

All voters

Finally, the majority judgment winner of the complete set of voters is determined.

Candidates Voters
AB
ExcellentFair3
FairPoor3
PoorFair4

The sorted ratings would be as follows:

Candidate
 ↓ Median point
A
B

Excellent   Good   Fair   Poor

The median ratings for A and B are both "Fair". Since there is a tie, "Fair" ratings are removed from both, until their medians become different. After removing 20% "Fair" ratings from the votes of each, the sorted ratings are now:

Candidate
 ↓ Median point
A
B

Result: Now, the median rating of A is "Poor" and the median rating of B is "Fair". Thus, B is elected majority judgment winner.

Conclusion

A is the majority judgment winner within the first group of voters and also within the second group of voters. However, both groups combined elect B as the Majority Judgment winner. Thus, Majority Judgment fails the consistency criterion.

Minimax

This example shows that the minimax method violates the consistency criterion. Assume four candidates A, B, C and D with 43 voters with the following preferences:

PreferencesVoters
A > B > C > D1
A > D > B > C6
B > C > D > A5
C > D > B > A6
A > B > D > C8
A > D > C > B2
C > B > D > A9
D > C > B > A6

Since all preferences are strict rankings (no equals are present), all three minimax methods (winning votes, margins and pairwise opposite) elect the same winners.

Now, the set of all voters is divided into two groups at the bold line. The voters over the line are the first group of voters; the others are the second group of voters.

First group of voters

In the following the minimax winner for the first group of voters is determined.

PreferencesVoters
A > B > C > D1
A > D > B > C6
B > C > D > A5
C > D > B > A6

The results would be tabulated as follows:

 X A B C D Y A [X] 11 [Y] 7 [X] 11 [Y] 7 [X] 11 [Y] 7 B [X] 7 [Y] 11 [X] 6 [Y] 12 [X] 12 [Y] 6 C [X] 7 [Y] 11 [X] 12 [Y] 6 [X] 6 [Y] 12 D [X] 7 [Y] 11 [X] 6 [Y] 12 [X] 12 [Y] 6 Pairwise election results (won-tied-lost) 0-0-3 2-0-1 2-0-1 2-0-1 Worst pairwise Defeat (winning votes) 11 12 12 12 Defeat (margins) 4 6 6 6 Opposition 11 12 12 12
• [X] indicates voters who preferred the candidate listed in the column caption to the candidate listed in the row caption
• [Y] indicates voters who preferred the candidate listed in the row caption to the candidate listed in the column caption

Result: The candidates B, C and D form a cycle with clear defeats. A benefits from that since it loses relatively closely against all three and therefore A's biggest defeat is the closest of all candidates. Thus, A is elected minimax winner by the first group of voters.

Second group of voters

Now, the minimax winner for the second group of voters is determined.

PreferencesVoters
A > B > D > C8
A > D > C > B2
C > B > D > A9
D > C > B > A6

The results would be tabulated as follows:

 X A B C D Y A [X] 15 [Y] 10 [X] 15 [Y] 10 [X] 15 [Y] 10 B [X] 10 [Y] 15 [X] 17 [Y] 8 [X] 8 [Y] 17 C [X] 10 [Y] 15 [X] 8 [Y] 17 [X] 16 [Y] 9 D [X] 10 [Y] 15 [X] 17 [Y] 8 [X] 9 [Y] 16 Pairwise election results (won-tied-lost) 0-0-3 2-0-1 2-0-1 2-0-1 Worst pairwise Defeat (winning votes) 15 17 16 17 Defeat (margins) 5 9 7 9 Opposition 15 17 16 17

Result: Taking only the votes of the second group in account, again, B, C and D form a cycle with clear defeats and A benefits from that because of its relatively close losses against all three and therefore A's biggest defeat is the closest of all candidates. Thus, A is elected minimax winner by the second group of voters.

All voters

Finally, the minimax winner of the complete set of voters is determined.

PreferencesVoters
A > B > C > D1
A > B > D > C8
A > D > B > C6
A > D > C > B2
B > C > D > A5
C > B > D > A9
C > D > B > A6
D > C > B > A6

The results would be tabulated as follows:

 X A B C D Y A [X] 26 [Y] 17 [X] 26 [Y] 17 [X] 26 [Y] 17 B [X] 17 [Y] 26 [X] 23 [Y] 20 [X] 20 [Y] 23 C [X] 17 [Y] 26 [X] 20 [Y] 23 [X] 22 [Y] 21 D [X] 17 [Y] 26 [X] 23 [Y] 20 [X] 21 [Y] 22 Pairwise election results (won-tied-lost) 0-0-3 2-0-1 2-0-1 2-0-1 Worst pairwise Defeat (winning votes) 26 23 22 23 Defeat (margins) 9 3 1 3 Opposition 26 23 22 23

Result: Again, B, C and D form a cycle. But now, their mutual defeats are very close. Therefore, the defeats A suffers from all three are relatively clear. With a small advantage over B and D, C is elected minimax winner.

Conclusion

A is the minimax winner within the first group of voters and also within the second group of voters. However, both groups combined elect C as the Minimax winner. Thus, Minimax fails the consistency criterion.

Ranked pairs

This example shows that the Ranked pairs method violates the consistency criterion. Assume three candidates A, B and C with 39 voters with the following preferences:

PreferencesVoters
A > B > C7
B > C > A6
C > A > B3
A > C > B9
B > A > C8
C > B > A6

Now, the set of all voters is divided into two groups at the bold line. The voters over the line are the first group of voters; the others are the second group of voters.

First group of voters

In the following the Ranked pairs winner for the first group of voters is determined.

PreferencesVoters
A > B > C7
B > C > A6
C > A > B3

The results would be tabulated as follows:

 X A B C Y A [X] 6 [Y] 10 [X] 9 [Y] 7 B [X] 10 [Y] 6 [X] 3 [Y] 13 C [X] 7 [Y] 9 [X] 13 [Y] 3 Pairwise election results (won-tied-lost): 1-0-1 1-0-1 1-0-1
• [X] indicates voters who preferred the candidate listed in the column caption to the candidate listed in the row caption
• [Y] indicates voters who preferred the candidate listed in the row caption to the candidate listed in the column caption

The sorted list of victories would be:

PairWinner
B (13) vs C (3)B 13
A (10) vs B (6)A 10
A (7) vs C (9)C 9

Result: B > C and A > B are locked in first (and C > A can't be locked in after that), so the full ranking is A > B > C. Thus, A is elected Ranked pairs winner by the first group of voters.

Second group of voters

Now, the Ranked pairs winner for the second group of voters is determined.

PreferencesVoters
A > C > B9
B > A > C8
C > B > A6

The results would be tabulated as follows:

 X A B C Y A [X] 14 [Y] 9 [X] 6 [Y] 17 B [X] 9 [Y] 14 [X] 15 [Y] 8 C [X] 17 [Y] 6 [X] 8 [Y] 15 Pairwise election results (won-tied-lost): 1-0-1 1-0-1 1-0-1

The sorted list of victories would be:

PairWinner
A (17) vs C (6)A 17
B (8) vs C (15)C 15
A (9) vs B (14)B 14

Result: Taking only the votes of the second group in account, A > C and C > B are locked in first (and B > A can't be locked in after that), so the full ranking is A > C > B. Thus, A is elected Ranked pairs winner by the second group of voters.

All voters

Finally, the Ranked pairs winner of the complete set of voters is determined.

PreferencesVoters
A > B > C7
A > C > B9
B > A > C8
B > C > A6
C > A > B3
C > B > A6

The results would be tabulated as follows:

 X A B C Y A [X] 20 [Y] 19 [X] 15 [Y] 24 B [X] 19 [Y] 20 [X] 18 [Y] 21 C [X] 24 [Y] 15 [X] 21 [Y] 18 Pairwise election results (won-tied-lost): 1-0-1 2-0-0 0-0-2

The sorted list of victories would be:

PairWinner
A (25) vs C (15)A 24
B (21) vs C (18)B 21
A (19) vs B (20)B 20

Result: Now, all three pairs (A > C, B > C and B > A) can be locked in without a cycle. The full ranking is B > A > C. Thus, Ranked pairs chooses B as winner, which is the Condorcet winner, due to the lack of a cycle.

Conclusion

A is the Ranked pairs winner within the first group of voters and also within the second group of voters. However, both groups combined elect B as the Ranked pairs winner. Thus, the Ranked pairs method fails the consistency criterion.

Schulze method

This example shows that the Schulze method violates the consistency criterion. Again, assume three candidates A, B and C with 39 voters with the following preferences:

PreferencesVoters
A > B > C7
B > C > A6
C > A > B3
A > C > B9
B > A > C8
C > B > A6

Now, the set of all voters is divided into two groups at the bold line. The voters over the line are the first group of voters; the others are the second group of voters.

First group of voters

In the following the Schulze winner for the first group of voters is determined.

PreferencesVoters
A > B > C7
B > C > A6
C > A > B3

The pairwise preferences would be tabulated as follows:

Matrix of pairwise preferences
d[X, Y] Y
ABC
X A 107
B 613
C 93

Now, the strongest paths have to be identified, e.g. the path A > B > C is stronger than the direct path A > C (which is nullified, since it is a loss for A).

Strengths of the strongest paths
d[X, Y] Y
ABC
X A 1010
B 913
C 99

Result: A > B, A > C and B > C prevail, so the full ranking is A > B > C. Thus, A is elected Schulze winner by the first group of voters.

Second group of voters

Now, the Schulze winner for the second group of voters is determined.

PreferencesVoters
A > C > B9
B > A > C8
C > B > A6

The pairwise preferences would be tabulated as follows:

Matrix of pairwise preferences
d[X, Y] Y
ABC
X A 917
B 148
C 615

Now, the strongest paths have to be identified, e.g. the path A > C > B is stronger than the direct path A > B.

Strengths of the strongest paths
d[X, Y] Y
ABC
X A 1517
B 1414
C 1415

Result: A > B, A > C and C > B prevail, so the full ranking is A > C > B. Thus, A is elected Schulze winner by the second group of voters.

All voters

Finally, the Schulze winner of the complete set of voters is determined.

PreferencesVoters
A > B > C7
A > C > B9
B > A > C8
B > C > A6
C > A > B3
C > B > A6

The pairwise preferences would be tabulated as follows:

Matrix of pairwise preferences
d[X, Y] Y
ABC
X A 1924
B 2021
C 1518

Now, the strongest paths have to be identified:

Strengths of the strongest paths
d[X, Y] Y
ABC
X A 024
B 2021
C 00

Result: A > C, B > A and B > C prevail, so the full ranking is B > A > C. Thus, Schulze chooses B as winner. In fact, B is also Condorcet winner.

Conclusion

A is the Schulze winner within the first group of voters and also within the second group of voters. However, both groups combined elect B as the Schulze winner. Thus, the Schulze method fails the consistency criterion.

References

1. ^ John H Smith, "Aggregation of preferences with variable electorate", Econometrica, Vol. 41 (1973), pp. 10271041.
2. ^ D. R. Woodall, "Properties of preferential election rules", Voting matters, Issue 3 (December 1994), pp. 815.
3. ^ H. P. Young, "Social Choice Scoring Functions", SIAM Journal on Applied Mathematics Vol. 28, No. 4 (1975), pp. 824838.