Policy Review Potential Updates For CAP Polling

Status
Not open for further replies.

Bughouse

Like ships in the night, you're passing me by
is a Site Content Manageris a Forum Moderator Alumnusis a CAP Contributor Alumnusis a Tiering Contributor Alumnusis a Contributor Alumnus
That's highly redundant. If you're going to value condorcet criteria, then value condorcet criteria.

By definition the condorcet winner would beat the PBV winner in a head to head matchup (assuming voter preferences don't change between the polls). An additional poll is redundant and basically says that you don't trust your own voting method.
 

jas61292

used substitute
is a Community Contributoris a Top CAP Contributoris a Forum Moderator Alumnusis a Battle Simulator Moderator Alumnus
By definition the condorcet winner would beat the PBV winner in a head to head matchup (assuming voter preferences don't change between the polls). An additional poll is redundant and basically says that you don't trust your own voting method.
To be fair, this is literally something that has always been the case. While our current method might value something other than the condorcet winner, it is still a viable polling method. Yet, IRV and by extention PBV are methods designed to pick a winner in a single vote, so no runoff vote is needed, and in spite of this, we have always refused to allow it to do the job it was intended to do. We take its results, see a winner, and then run another poll anyways. And since our latest polling changes, that poll is literally the exact same poll again. To me, this has always screamed of not trusting our own polls.

So yeah, I agree with you that redundancy is unneeded, and seems to show insecurity. However, it is already an ingrained part of our system, and so I hardly think that this could be any worse (even if I do think the goal should be to make things better, rather than simply avoiding being worse).
 

QxC4eva

is an Artistis a Forum Moderatoris a Community Contributor
Moderator
By definition the condorcet winner would beat the PBV winner in a head to head matchup
Ah true, I totally missed that woops! Nice catch Bughouse :)

So back to the PBV vs HTH debate we can't avoid now... :\ Here's my reasoning for HTH:

HTH picks the Condorcet winner if there's one. PBV picks the Condorcet winner only if there's enough direct support for it. Direct support gives the winner a "mandate", which doesn't apply to CAP since we're voting for traits and submissions, not candidates. That gives us the freedom to choose either the broad consensus favorite (HTH) or the majority group's favorite (PBV). Broad consensus may satisfy more voters that's why I prefer HTH. But for candidate polls like TL or TLT that require a mandate, PBV may still be better.

If this reasoning sounds okay we can probably go ahead with it.....I don't totally disagree with Heal that the rankings should be treated more uniformly, but that reason alone tends to favor Borda Count over HTH.


And yeah I do find it funny how people are suddenly worried about redundancy when CAP polls already have it, yet they're against proposal D which fixes that! E was an oversight on my part as I'm still against all redundancy :3
 

Bughouse

Like ships in the night, you're passing me by
is a Site Content Manageris a Forum Moderator Alumnusis a CAP Contributor Alumnusis a Tiering Contributor Alumnusis a Contributor Alumnus
So here's an example of why ranked pairs is better... not that this extreme situation is common at all.

http://www.smogon.com/forums/threads/cap-updates-ul-poll.3596158/page-2#post-7241690

First, notice that the outcomes are almost identical and we're really arguing over pretty small differences here at this point (for single-round polls at least). Notice to the only difference between the two outcomes:

in one, Jas prevails over sparktrain (imagine we were selecting top 4 and jas and sparktrain hadn't already won the GL poll, which makes this totally moot anyway...,) and in the other sparktrain prevails over jas.

Well, which way to resolve this is better? The one where jas is assumed to be better than sparktrain based on an 8, 6, 6, 6 vote result? Or the one where we actually look at those other 12 ballots for those other two people with 6 votes (in all pairwise fashions) and see who's actually preferred among that group. As we can see in ranked pairs, doing so brings sparktrain's original 6 ahead of jas's original 8.

sparktrain in a head to head vs jas won 14-10, despite losing 8-6 when it was masked by those other 12 ballots.

I think it's pretty clear that the second is better because the first is basically a "loss" caused entirely by the quirks of the voting system. Had sparktrain lost 8-7-6-6 at that point in the poll, we'd have eliminated the other 12 ballots and sparktrain would then win 15-10 in the last round of that PBV iteration. Or if he'd lost 8-6-6-5 at that point in the PBV, we'd have eliminated the 5... then sparktrain would have eventually won as well. Literally just because he happened to lose in a 3 way tie rather than still losing either of these other ways, he loses out to jas in PBV. That's not a good voting system.
 

BP

Beers and Steers
is a Contributor to Smogon
To be honest I have not been following this thread as closely as I should have in the past months. With this being said please lend an ear for what I have to say:

Ranked pairs works well in my opinion and even though I lost the poll Which I am incredibly salty about and made me question my role in the CAP community as a whole I actually really liked this how this form of polling is conducted and how accurate the results are displayed. It is not just efficient but it made the most sense to me as an individual in terms of picking which participant obtained what place. Long story short I enjoyed this method and I hope it is used more in the future.
 
Admittedly, ranked pairs work quite well, as Broken Phobias had already stated above. The results are more presice than the previous poll formats. Now that we're in Gen 7, I hope this poll format isn't just a one-time thing, unlike the many things we've tried over the course of Gen 6. Congrats to those who received a position as well.

i am a condorcet loser
 

QxC4eva

is an Artistis a Forum Moderatoris a Community Contributor
Moderator
Bughouse you probably know about Arrow's impossibility theorem, here's what he said:
Kenneth Arrow said:
Most systems are not going to work badly all of the time. All I proved is that all can work badly at times.
In other words, for any given system (be it PBV or Ranked Pairs or whatever) it's always possible find examples of it giving unfair results. So showing an example where PBV is unfair doesn't actually prove it's a bad system, or that RP is a better one.

I think most of us here (me included) support RP over PBV by now, but I'm not getting where the notion that PBV is a bad system came from. It's not. :P The reason it sometimes differs from RP is cause they're based off different principles.

PBV elects the most supported candidate, while RP elects the most preferred candidate. There's a slight difference between the two - say I listed 8 entries, it means I prefer my 7th entry over my 8th but it doesn't mean I support my 7th to overall win. In RP, both my 1st and 7th will get one point each for beating my 8th, despite the former having a much greater winning margin. PBV though, will end up ignoring both unless they're the frontrunners. In short, PBV favors all-time high place votes while RP favors getting higher place votes relative to other entries.

I hope we can accept that neither PBV or RP is mathematically perfect and just focus on which one's principles suit CAP better.
 

BP

Beers and Steers
is a Contributor to Smogon
Bughouse I hope we can accept that neither PBV or RP is mathematically perfect and just focus on which one's principles suit CAP better.

QxC4eva is right. We should be focusing on which principle would better suit our voters and the CAP as whole. I have already shared my two cents on the which Polling idea I think I like the most. I think we may want to take a Poll on which form is most highly regarded. Of course the only people voting would be PRC members and regarding the format I think it should be a "You only get 1 vote" sort of thing like Smogon thread polls or the Polls in PS! chatrooms. My one concern with this is how informed are PRC members going to be? This is just my idea on how to come to a conclusion for a polling format. Regarding how informed PRC is on this topic you may want to list examples on what each poll is. I know that some PRC members aren't really as active as they should be.

Hopefully you found my input helpful in some way shape or form. In my opinion this thread has basically reached its climax in terms of what we hoped to accomplish. I personally do not see this thread going anywhere else unless people want to continue to debate on which form of polling is better.
 

HeaLnDeaL

Let's Keep Fighting
is an Artistis a Forum Moderator Alumnusis a CAP Contributor Alumnus
I think most of us here (me included) support RP over PBV by now, but I'm not getting where the notion that PBV is a bad system came from.
Maybe I missed something but as far as I know Bughouse never said PBV is bad. He listed a new, relevant example of why he preferred RP to PBV. He did not call RP mathematically perfect, he just said he preferred it ._.

I mean yeah, he concluded by saying PBV was not good. But not good doesn't = bad and my reading on the line was based more of PBV was not good in that particular situation, not that it was god awful sucky all the time. All of this aside, we'd need to organically find situations where RP sucks it up before we can adequately critique it for sucking up... If we're finding flaws in one system, I believe they definitely have room to be discussed here.

I also just want to say very clearly that showing PBV and Ranked Pairs results in the last polls was entirely experimental and we not definitive at all. I feel very very safe to say that this thread so far has concluded that Condorcet methods are what we will use for polls with 3 candidates. The UL and GL polls clearly had more than three candidates. RP results were posted more of in the nature of transparency as we are figuring out what to do from here.
 
Last edited:

QxC4eva

is an Artistis a Forum Moderatoris a Community Contributor
Moderator
Sorry my bad choice of words! You can replace "bad" with "average" (or whatever you think "not good" means) and my post still reads the same: We can't suggest PBV is worse with an example of it being unfair, as you can always find examples of any system being unfair. Bughouse showed a PBV math quirk and concluded RP is better. But RP has mathemetical quirks too, so how is it better? That's what I'm saying.

Choosing a voting method should always be principles first, or else it'll be an apples vs oranges debate. It's only when the principles are the same (like HTH vs RP) do we then turn to the maths and numbers.


Anyway here's a example of RP being "unfair" ...

Imagine this TL poll slate: jas61292, sparktrain, QxC4eva. I'm the controversial candidate! Now imagine there's a big rivalry between jas and spark, and people who like one oppose the other. So they vote tactically, and here's the results:

50 people voted: jas61292 > QxC4eva > sparktrain
49 people voted: sparktrain > QxC4eva > jas61292
2 people voted: QxC4eva

(kinda exaggeration but it's for clarity ^^;)

So what happened was everyone knew there's no way I'd win that poll, so they listed me 2nd in hopes of "burying" their rival to give their 1st option a higher chance to win. That's a pretty standard strategy, and RP succumbs to it because......... surprise! I'm the Condorcet winner of this poll. If we use PBV jas would win (I still come in second tho cause PBV is not proportionally represensative....but that's a different topic :P)
Code:
Untitled Poll  :: Ranked Pairs output

Ballots: 101
Preferences: 299

[Pair#1] qxc4eva (52:49) sparktrain
[Pair#2] jas61292 (50:49) sparktrain
[Pair#3] qxc4eva (51:50) jas61292

The Condorcet winner is qxc4eva.
The Condorcet loser is sparktrain.

QXC4EVA > [jas61292][sparktrain]
[qxc4eva] > JAS61292 > [sparktrain]
[qxc4eva][jas61292] > SPARKTRAIN

-------------
RUNDOWN
-------------
1. qxc4eva
2. jas61292
3. sparktrain
Code:
Untitled Poll  :: PBV output

Ballots: 101
Preferences: 299

[Round 1.0] jas61292=50*  sparktrain=49*  qxc4eva=2
[Round 1.1] jas61292=50*  sparktrain=49*
[Round 1.2] jas61292=99  --> #1
[Round 2.0] qxc4eva=52  sparktrain=49
[Round 2.1] qxc4eva=101  --> #2
[Round 3.0] sparktrain=99  --> #3

-------------
RUNDOWN
-------------
1. jas61292
2. qxc4eva
3. sparktrain


Here's why I came second: PBV is majoritarian based so there's a potential for landslide victories like this one.
The biggest minority (jas supporters) overrun the entire poll, locking out the views of another big minority (sparktrain supporters)
Code:
Untitled Poll  :: STV output

Ballots: 101
Preferences: 299
Quota: 33.667 (Hagenbach-Bischoff)

[Round 1] jas61292=50*  sparktrain=49*  qxc4eva=2  (SURPLUS=0;  EXHAUSTED=0)  --> jas61292 qualifies
[Round 2] sparktrain=49*  qxc4eva=2  (SURPLUS=16.333;  EXHAUSTED=0)  --> sparktrain qualifies
[Round 3] qxc4eva=2  (SURPLUS=31.667;  EXHAUSTED=0)
[Round 4] qxc4eva=33.667  (SURPLUS=0;  EXHAUSTED=0)  --> qxc4eva eliminated

-----------------
RUNDOWN
-----------------
1. jas61292  [+]
2. sparktrain [+]
3. qxc4eva
In short, PBV's winner is the best winner in isolation, and RP's winner is the best winner in the presence of others. As PBV will refuse to elect any Condorcet winner lacking core support, one can say RP has a higher risk of electing a "weak" candidate. A weak candidate is someone who relies on beating others in the lower ranks, rather than outright having high-ranked votes themselves.

Of course my example doesn't imply PBV is better than RP, as once again it comes down to principle:

Does CAP want to value the winner who is most supported, or the winner who is most preferred?

- For polls where the TLT writes the slate, all entries will be good so there's no concern there'll be a "weak" winner. For those polls (concept, typing, stats etc) I think RP is better.
- For polls where the TLT can't control the slate, it's possible to have a Condorcet winner who is weak. So polls like art, sprites, pokedex I think PBV does better.
- In general if we can only choose one, I prefer RP as it satisfies more voters (most of the time)
 
Three candidates doesn't sound like enough to run Condorcet, and that hypothetical results is hilarious. However, art and other tasty parts of the process would prefer PBV over RP, since those aspects are out of the TLT's control.
 

Bughouse

Like ships in the night, you're passing me by
is a Site Content Manageris a Forum Moderator Alumnusis a CAP Contributor Alumnusis a Tiering Contributor Alumnusis a Contributor Alumnus
you can nitpick edge cases that will make any voting system look silly. the difference is the one I mentioned is far more likely (and in fact already happened!).
 

HeaLnDeaL

Let's Keep Fighting
is an Artistis a Forum Moderator Alumnusis a CAP Contributor Alumnus
Three candidates doesn't sound like enough to run Condorcet, and that hypothetical results is hilarious.
I agree that the hypothetical result is hilarious because it's just not realistic, but 3 candidates is the ideal number to run Condorcet polling methods as discussed earlier in this thread.

And I just want to say that QxC's example was completely and totally avoided by Proposal A in this very thread. So it's not that it hasn't been considered and that QxC is bring up something brand new; it was the very first thing brought up and was not favored due to the preference to total Condorcet.

Before I get into the debate on whether or not QxC's hypothetical scenario is a "fair result" or not, I have some disagreements with the examples of which polls QxC believes offer too great of a chance for a "weak" candidate to win (and I feel like some potentially biased/misleading out throughout parts of this thread; part of this is just we're trying to describe different components of voting that might seem synonymous but actually aren't, which necessitates an increased vocabulary... but realistically, the unbiased way of saying QxC's "weak" is saying instead "not hated" or "not hated/not liked" depending on the context).

- For polls where the TLT can't control the slate, it's possible to have a Condorcet winner who is weak. So polls like art, sprites, pokedex I think PBV does better.
This assertion completely ignores the nature of the art poll series. The very first art poll is MBV and this thread in no way has ever gone out to change that. By having art's first poll be MBV, it functions almost like having a TLT pick a slate for next polls. Art polls 2 and onward where PBV vs Condorcet methods would even be debated thus have a controlled slate. All of the "weak" ("not liked" in this case) will thus be eliminated in the first MBV poll.

Sprites is also a very unique poll due to a number of conditions. First, there's usually a very small number of entries. Furthermore... don't bite my head of for my choice of words here since I realize they aren't perfect, but the sprite poll is one of the most "objective" polls in all of flavor CAP... I'm not saying it's completely objective and there are certainly subjective elements required, but the basis of the creation of sprites is to make something that looks like something else we've already decided on. When judging sprites, voters are not judging the work in the same way they judge the concept art... a big part of (though definitely not all of) the sprites poll involves people judging how well it matches the mon's concept art and how well it matches BW sprite style. In pretty much any other poll, the reason's for voting for X over Y is much more varied than in the sprite's poll... Sprites have very known criteria, probably moreso than any other poll, and this definitely impacts the way the vote goes down. It's completely unrealistic for a "weak" sprite to win because voters are so much more informed based on the more "objective" nature of the poll. In my time at CAP, I've never seen an obviously "lesser" sprite come close to winning and QxC's example of vote above, while already completely unrealistic, is just flat out impossible in a sprites poll. People are far less "polarized" in sprites and there's generally a lack of strategic voting. The community at large really wants to avoid having a bad sprite represent a CAP on the server.

In terms of Pokedex entries, I really have no opinion as I don't pay too much attention there.

Now, turning back to QxC's example... it is really that unfair? In the situation described, QxC as a candidate was completely and totally unhated whereas jas and sparktrain were both polarizing and ended up being hated slightly more than they were not hated... Do we praise them for being more liked than QxC or do we praise QxC for not being hated at all? Quite frankly I think the example provided does produce a fair result because jas/spark were incredibly hated. That said... it's a fantasy of an example and should we really be giving that much credit to it?
 

QxC4eva

is an Artistis a Forum Moderatoris a Community Contributor
Moderator
This assertion completely ignores the nature of the art poll series. The very first art poll is MBV and this thread in no way has ever gone out to change that. By having art's first poll be MBV, it functions almost like having a TLT pick a slate for next polls. Art polls 2 and onward where PBV vs Condorcet methods would even be debated thus have a controlled slate. All of the "weak" ("not liked" in this case) will thus be eliminated in the first MBV poll.
I think you misunderstood my point! It doesn't matter if the slate is altered or not, the point is whether it's altered by the TLT. I get that MBV may be somewhat similar but there's one thing it can't do that TLT can -- guarantee a slate without bad options. That aside, I think your sprite reasoning is very good as it's based off principles yay! No maths or numbers involved :) What we disagree now I think is opinion based.

Also I hope my example was useful despite you guys saying it's unrealistic :D It shows a special case where RP, PBV and STV all give different results, while also exposing a vulnerability in Condorcet systems. Even if you don't like it I hope you can still see it sheds light on where the 3 systems fundamentally differ. And sorry that I exaggerated it, but it's really just to make things clear. Being made up doesn't mean it'll never happen in regular cases though:

35 voters: heal > spark > deck > jas
32 voters: spark > deck > jas > heal
17 voters: jas > spark > deck > heal
16 voters: deck > jas > spark > heal

^ why I didn't use this example, cuz it's harder to see what's going on!

some notes on manipulation.....I think the argument in HeaL's last paragraph is very good, but it assumes everyone voted honestly. What if some didn't? In my example, everyone voted tactically and we ended up with a winner no one is happy with. This is just information (I'm not arguing for anything :P) but Condorcet methods are vulnerable to that "burying" strategy I mentioned. Vulnerable doesn't mean voters can vote unfairly and skew some results, it means they can sometimes vote unfairly and the whole system just pukes up. Electing me as the winner may seem logical but if you factor in the gaming you can hopefully see how RP "messed up". Now it's true that you can also game a PBV poll (and there's even more ways to do that compared to RP) but those ways require knowledge on how the system works which I think is a decent level of safety. AFAIK "burying" is the only thing RP really ever succumbs to so maybe that's the good news :) Other ways I know require knowledge of the system.
 

QxC4eva

is an Artistis a Forum Moderatoris a Community Contributor
Moderator
Since we still use PBV I'd like to propose a "fix" to it, specifically for Proposal A. It'll solve the OP's second problem and the 8-6-6-6 scenario Bughouse mentioned:
Code:
Array ( [integermova] => 0 [brokenphobias] => 1 [boxofkangaroos] => 2 [deckknight] => 4 [darkslay] => 4 [sparktrain] => 4 [krazycake] => 5 [jas] => 7 )
Array ( [brokenphobias] => 1 [boxofkangaroos] => 2 [sparktrain] => 4 [darkslay] => 4 [deckknight] => 4 [krazycake] => 5 [jas] => 7 )
Array ( [boxofkangaroos] => 2 [deckknight] => 4 [sparktrain] => 4 [krazycake] => 5 [darkslay] => 5 [jas] => 7 )
Array ( [deckknight] => 4 [darkslay] => 5 [krazycake] => 5 [sparktrain] => 5 [jas] => 7 )
Array ( [sparktrain] => 6 [krazycake] => 6 [darkslay] => 6 [jas] => 8 )
^ Sparktrain, KrazyCake and DarkSlay have the fewest votes so there's definitely basis to eliminate them. Whether this is "unfair" or not is debatable but it does violate two principles PBV should not violate:
  • The winner must have a majority (>50%) of votes. Jas won with only 30.8% (8/(8+6+6+6))
  • It must be immune to vote splitting. Votes that split between Sparktrain/DarkSlay/KrazyCake may cause them to lose unfairly against Jas.
The solution? Eliminate either sparktrain, Darkslay or KrazyCake at random. Randomness works on probability so lets do a tree diagram!

There's 4 possible outcomes (A, B, C and D) each with their likelihood calculated. The winner is whoever has the highest chance to win:


It's a 50/50 between Jas and Spark so to tiebreak we look at their chance of getting 2nd place, in which Jas is the winner. Turns out to be the same result, except jas now wins by majority and without any vote splitting! Problem solved :D

Code:
[Round 4.0] jas61292=7 > KrazyCake=5 > DarkSlay=4 ~ DeckKnight=4 ~ sparktrain=4 > boxofkangaroos=2 > BrokenPhobias=1 > IntegerMova=0
[Round 4.1] jas61292=7 > KrazyCake=5 > DarkSlay=4 ~ DeckKnight=4 ~ sparktrain=4 > boxofkangaroos=2 > BrokenPhobias=1
[Round 4.2] jas61292=7 > DarkSlay=5 ~ KrazyCake=5 > DeckKnight=4 ~ sparktrain=4 > boxofkangaroos=2
[Round 4.3] jas61292=7 > DarkSlay=5 ~ KrazyCake=5 ~ sparktrain=5 > DeckKnight=4
[Round 4.4] jas61292=8 > DarkSlay=6 ~ KrazyCake=6 ~ sparktrain=6
   |__________ [33.3%] eliminate sparktrain
   |  |__________ jas61292=12 > DarkSlay=8 > KrazyCake=6
   |  |__________ jas61292=13 > DarkSlay=10
   |  |__________ jas61292=18  --> +33.333% win chance
   |__________ [33.3%] eliminate KrazyCake
   |  |__________ sparktrain=11 > jas61292=8 > DarkSlay=7
   |  |__________ sparktrain=14 > jas61292=10
   |  |__________ sparktrain=23  --> +33.333% win chance
   |__________ [33.3%] eliminate DarkSlay
      |__________ jas61292=10 > KrazyCake=8 ~ sparktrain=8
         |__________ [50%] eliminate sparktrain
         |  |__________ jas61292=16 > KrazyCake=10
         |  |__________ jas61292=18  --> +16.667% win chance
         |__________ [50%] eliminate KrazyCake
            |__________ sparktrain=14 > jas61292=10
            |__________ sparktrain=23  --> +16.667% win chance
[Round 4.5] P(first): jas61292=50% ~ sparktrain=50% > DarkSlay=0% ~ KrazyCake=0%
            P(top_2): jas61292=100% > sparktrain=50% | DarkSlay=33.3% > KrazyCake=16.7%
[Round 4.6] jas61292  --> #4
As such, I propose we use probability tree tiebreaking for all PBV polls. Tiebreaking rules are nothing new - in fact we already use one elsewhere. Like PBV, Condorcet Method can also violate its own principles so to fix it we once again use a tiebreaking rule. That tiebreaking rule is called "Ranked Pairs".
 

Bughouse

Like ships in the night, you're passing me by
is a Site Content Manageris a Forum Moderator Alumnusis a CAP Contributor Alumnusis a Tiering Contributor Alumnusis a Contributor Alumnus
but how is that better than ranked pairs? We know that sparktrain beats jas in a condorcet system, yet in this system, jas wins half the time.

This sort of approach makes sense to me if we were modeling projected winners based on a sample of voting, but just doesn't make any sense when we are analyzing the whole population of all ballots. Like, what are we probabilistically trying to sort out... we have perfect information.
 

jas61292

used substitute
is a Community Contributoris a Top CAP Contributoris a Forum Moderator Alumnusis a Battle Simulator Moderator Alumnus
While I don't have a lot to say on this, I did just want to chime in to say that a "fix" should absolutely be added if we continue to use PBV in any situation. And I say "fix" in quotations marks since it is really not a fix to PBV, but rather to our implementation thereof. PBV as a method is supposed to have some tie-breaker, and only one thing should ever be eliminated at once (unless the tie is at 0 votes). The fact that our script takes a situation such as a 10-10-11 vote, and declares the last one the winner is simply not how PBV is supposed to work. QxC's method certainly would be one way to do things, but really how it should be done is not some strange probabilistic way of looking past the tie and finding an alternative winner. Especially as there can be ties early in many option polls, not just at the final few. Instead it should be corrected by having a way to declare one of the tied options the loser, and eliminate it.

How to do that though, I really do not know. But it is how it is supposed to work.
 

QxC4eva

is an Artistis a Forum Moderatoris a Community Contributor
Moderator
The most neutral way to break a tie is doing it by random, or in the real world, "by lot". The only problem is it's luck based, that's why we use a probability model to take out that factor.

It's like comparing Surf and Hydro Pump... who will do more damage? Again we use probability:
90bp * 100% = 90
110bp * 80% = 88
So mathematically Surf does more. The concept may be strange to those unfamiliar with it but that's basically what it is. Jas is right the goal is to eliminate one loser -- which we do randomly then remove the luck with a probability model, hence my proposal.

Also yup, there's other ways to tiebreak PBV:
  • Backwards tiebreaking - Eliminate who did the worst by looking at the earlier rounds. This method eliminates Sparktrain.
  • Condorcet loser - Eliminate the Condorcet loser within the losers. This method will eliminate KrazyCake.
Code:
[Round 4.0] jas61292=7 > KrazyCake=5 > DarkSlay=4 ~ DeckKnight=4 ~ sparktrain=4 > boxofkangaroos=2 > BrokenPhobias=1 > IntegerMova=0
[Round 4.1] jas61292=7 > KrazyCake=5 > DarkSlay=4 ~ DeckKnight=4 ~ sparktrain=4 > boxofkangaroos=2 > BrokenPhobias=1
[Round 4.2] jas61292=7 > DarkSlay=5 ~ KrazyCake=5 > DeckKnight=4 ~ sparktrain=4 > boxofkangaroos=2
[Round 4.3] jas61292=7 > DarkSlay=5 ~ KrazyCake=5 ~ sparktrain=5 > DeckKnight=4
[Round 4.4] jas61292=8 > DarkSlay=6 ~ KrazyCake=6 ~ sparktrain=6
[Round 4.5] (CONDORCET): sparktrain > DarkSlay > KrazyCake
[Round 4.6] sparktrain=11 > jas61292=8 > DarkSlay=7
[Round 4.7] sparktrain=14 > jas61292=10
[Round 4.8] sparktrain=23  --> #4
Neither of these are as neutral as what I proposed, but it's still better than no tiebreaking so I'm not opposed to them. :)

but how is that better than ranked pairs? We know that sparktrain beats jas in a condorcet system, yet in this system, jas wins half the time.

This sort of approach makes sense to me if we were modeling projected winners based on a sample of voting, but just doesn't make any sense when we are analyzing the whole population of all ballots. Like, what are we probabilistically trying to sort out... we have perfect information.
It's neither better or worse than Ranked Pairs. It's only better than barebones PBV (the system we're using now).
I certainly do support RP+RP but right now our system is PBV+RP, so what I proposed is just a better PBV+RP... unless you can point out why it's not.
 
Last edited:

Deck Knight

Blast Off At The Speed Of Light! That's Right!
is a Forum Moderator Alumnusis a Top CAP Contributor Alumnusis a Top Smogon Media Contributor Alumnus
As I understand it, the only "randomness" in QxC's fix is that it chooses each iterative elimination randomly until it goes through each iteration. The only thing "random" about the elimination is that it makes a selection that would not be chosen strictly by vote tallies to do a deep dive into each preference, then aggregates all probabilities together (which is what QxC said, I'm just being redundant)

What we're talking about is the tiebreak method and this one seems sound and efficient
 

Bughouse

Like ships in the night, you're passing me by
is a Site Content Manageris a Forum Moderator Alumnusis a CAP Contributor Alumnusis a Tiering Contributor Alumnusis a Contributor Alumnus
I totally missed the part where it goes down each part of the tree... that's not too terrible. It's unnecessarily involved, since jas is right that eliminating the condorcet loser is the "best" and "standard" option, but this seems ok and has already been coded ha.
 

QxC4eva

is an Artistis a Forum Moderatoris a Community Contributor
Moderator
That's right, Deck Knight! Well explained. Also here's a convo I had with Dogfish44 (who is a voting systems guru ^_^) on how to handle ties in IRV/PBV/STV.
Code:
[10:41 PM] DF44: ... And the fun of solving an IRV 3-2-2 vote tie :P
[10:43 PM] Tomomon: Hah!
[10:43 PM] Tomomon: That was something I wanted to ask about
[10:43 PM] Tomomon: What's the best way to solve a 3-2-2?
[10:44 PM] Tomomon: I know Ireland uses "backwards tie-breaking" or whatever it's called
[10:44 PM] DF44: run every situation (See what happens when you eliminate each). Normally reality ends up being such that the 3 wins.
[10:44 PM] DF44: Or, at least, that's what I do
[10:44 PM] DF44: there are so many systems. I know CAP sometimes eliminates both (Which is eww but understandable)
...
[6:16 PM] Tomomon: How would you have handled this btw?
[6:16 PM] Tomomon: (the 8-6-6-6)
[6:26 PM] DF44: For single-person STV, I'd have probably done as you would've.
[6:28 PM] DF44: See how elimination patterns go. Potentially I'd rerun with any candiate who can't win removed? I very rarely end up doing single-position things with STV (Yes-No isn't that tricky, and if I've got multiple candidates, it tends to correlate with multiple open vacancies?).
His method is similar to mine - examine all possible outcomes then eliminate those with no chance to win. From the tree I posted, those candidates are DarkSlay and KrazyCake so we eliminate both, and Sparktrain ends up winning. I'd say all is good with this method except one thing:

PBV violates Independence of Irrelevant Alternatives (meaning candidates who can't win can still affect who wins the poll). In our case, whether Jas or Spark wins rests entirely on how early KrazyCake gets eliminated -- If we eliminate Cake early, Sparktrain will win. Eliminate Cake later, then Jas wins. DF44's method eliminates Cake right away therefore gives Sparktrain an advantage he wouldn't have under normal PBV conditions. For a tiebreaking rule this is questionable, as it means we're deviating from what our main system (PBV) was meant to do.

(I should mention that an ideal tiebreaking rule is completely neutral and free of favoritism)

DF44's method is still WAY better than any of the arbitrary rules out there that's why I'm throwing it in. As for my method, I'm not aware of problems other than it's NP-hard so for big polls it may be computationally demanding. In general, each method has its own pros and cons but any of them will beat what we using right now. ;P
 

QxC4eva

is an Artistis a Forum Moderatoris a Community Contributor
Moderator
OK it looks like we agree there's a problem with our PBV script but aren't interested fixing it so here's another solution: stop using the PBV script! All the polls using it right now are better off switching over to ranked pairs. Why? Well. RP can work on any slate size (not just 3 candidates) and does not need a tiebreaker like PBV does, and is also a condorcet method. It can pretty much replace PBV entirely and all the op's concerns will be fixed. I propose either a) All PBV polls are to be replaced with Ranked Pairs, or b) All PBV polls are to use a tiebreaking method.

It's worth to note that, while this proposal is a better version of C there's still some inaccuracy with how the multi winner polls work. Incidentally, our single winner polls are fair cause they treat all ballots equally (every ballot gets counted once) but if we run RP on multi-winner polls, some ballots may get counted a few times more than others and we end up with disproportionate results. PBV has this same problem as well. The ultimate solution is to switch to CPO-STV - a condorcet method that counts all ballots exactly once no matter how many winners there are. I must admit, though, I've struggled to explain STV without confusing people so expect me to bring this up again in the near future. Ranked Pairs is okay for now, and certainly beats the system we currently have.
 

Quanyails

On sabbatical!
is a Top Artist Alumnusis a Community Leader Alumnusis a Community Contributor Alumnus
Hey, all! Let close this topic. This thread has been hanging around in a limbo for too long. Thanks to QxC4eva for helping out with this post, and thanks to Birkal for approving it! :)

-----

PBV shenanigans from CAP 24

The issue with our polling methods came up again during CAP 24 Art Poll 2 this round. StephXPM's design was polarizing--it had a lot of ballots that ranked it high and a lot of ballots that ranked it low. In particular, we had this result from the poll:

Code:
[Round 3.1] StephXPM=37 > Falchion=33 ~ Reiga=33
This means that Falchion and Reiga were tied. Under our current system, StephXPM would have taken third. However, let's assume one last voter comes in with a ballot of StephXPM > Falchion > Reiga. Then Round 3 of the results would continue as:

Code:
[Round 3.2] Falchion=52 > StephXPM=45
Which means that even though this voter preferred StephXPM > Falchion, what his vote actually did was launch Falchion ahead of StephXPM. WHAT.

If we use a tiebreaking method or Ranked Pairs, we can see that StephXPM should have ranked in fifth place, and it's only with a plurality that he is considered to be in third place.

The good thing is that we currently use Ranked Pairs on all final polls, but we should make that consistent on all polls, since that affects stuff like the top n results that go to the next round and the rankings of non-winning art.

-----

To avoid scenarios like the above, we will use Ranked Pairs for all ranked polls. The advantages of using Ranked Pairs compared to the current implementation of Preferential Block Voting are:

Efficient:
  • Has rules for breaking internal ties.
  • Does not require additional polls to verify ranking, if we want it.

Robust:
  • Harder to track results manually.
  • More resilient to strategic voting (see above example).
  • Results are more "stable", meaning one voter's ballot can't drastically shuffle the order of final results.

-----

On tiebreaks

It's very hard to produce complete ties with Ranked Pairs. However, it can happen with smaller polls. We will adopt the tiebreaking mechanism discussed in this thread, adapted to match the new polling methods.

-----

In summary
  • We will use Approval Voting (MBV) as we currently do for large, unranked polls.
  • In all other cases, we will use Ranked Pairs.
  • In the event of a tie, we tiebreak using, in order:
    • Borda count
    • The ranking from previous polls, starting with the most recent.
    • Topic Leader's/Section Leader's choice.
If we want to make further adjustments to CAP voting (and I know there are suggestions for further improvements), let's start a new thread!
 
Status
Not open for further replies.

Users Who Are Viewing This Thread (Users: 1, Guests: 0)

Top