Policy Review Polling Options

Status
Not open for further replies.
jas and I had a brief discussion about the above, then Doug shared some of his opinions on irc today. Here's the log of the conversation, edited for clarity and conciseness. I've bolded what I consider to be the most critical points and what I think ought to be discussed.

<paintseagull> I think if we went with single vote IRV we'd have to try pretty hard to educate our voters better about what not ranking everything means

I'd rather do that, and hope for the best, accepting that we may not be perfectly gauging the community consensus, than delete votes that don't have the required # of votes

<jas61292> Yeah, I am not a huge fan of forcing people to rank everything, or even a certain number. I'd rather just STRONGLY recommend ranking them all, and make clear how it works.
Click to expand...

<paintseagull> Doug are you going to post in our polling discussion?

<DougJustDoug> I have been meaning to, but I just haven't carved out the time to make a proper post. Because there is NO WAY i can make a short post on a topic like that. I have way too many opinions and historical context to do anything other than a wall of text!

<paintseagull> i thought you might!

<DougJustDoug> It hate it when I KNOW i'm gonna be long winded about something

Normally I make long posts when I intend to just post a paragraph or two

But when I KNOW it will be long, I tend to never start it in the first place!

<paintseagull> i guess i just have 2 questions: 1) are we being overly naive in not taking enough of the history into account, and 2) are you ok with changing things up

<jas61292> But yeah, I've been hoping to get a post from someone who was around and part of the decisions to make things the way they are now.

<DougJustDoug> I'm almost always OK with changing things up. I'd rather change and fail than let resistance to change become a habit in CAP.

But the history angle is another story

Because we have a LOT of history in trying to make polling as effective as it can be

And I think the crux of difficuly with polling is balancing the "correctness" of the poll with the "community building" aspects of the poll.

The two tend to conflict quite a bit

<paintseagull> yeah, community building is not really being spoken of at the moment

whatever method we choose should be accessible

community building sounds a lot better than "spectacle"

<DougJustDoug> A close poll is great advertising for the CAP project.

Hell, it's a lot of fun!

But it really fucks with getting quality polling results.

It encourages bandwagoning, it encourages cheating, and it amps up dispute drama.

<jas61292> That is one of the reasons I like the idea of just using IRV. Unlike single choice votes, its a lot harder for the average person to just look on and see how close it is. Its possible, sure, but not so obvious. At the same time though, it can still be shown and have that attention drawing factor for the average person.

<DougJustDoug> So, it's a tough nut to crack. We don't want to make polls a boring, hidden process -- even if it yields the best result for the pokemon we are creating. That would be the proverbial tree falling in the forest with no one there to hear it.

But we also don't want the spectacle and easy access to make the process an unfair mess. That discourages quality participants from engaging in the competition. Which, in turn, ultimately hurts overall project perception and activity levels.

<paintseagull> in that case the key to me seems to be compromise

in which case I still think 1 PBV -> 1 runoff is a good option

I think that is the thing about single round IRV that bothers me that i couldn't quite put my finger on

multiple polls feel like a security blanket because it feels like a very sturdy community exercise with lots of steps

single round IRV seems so cold.. unloving.. harsh and unforgiving

i don't think any sense of spectacle should be more important than getting an accurate or rational poll result but i do think it's important to consider that polling is how we recruit new cap participants a lot of the time

<DougJustDoug> I think we lose a lot of community benefit if the polling process hinges exclusively on a single round of voting that requires a complex computer program to decipher who wins. People can't get immersed in a race where they are completely guessing as to who is winning or losing when reviewing the poll thread.

A head-to-head race, or a race with a few participants and single voting is very easy for people to understand and follow.

<jas61292> Well, I personally see the spectacle being more of something deriving from the time period and the contestants than the number of polls. One tension filled poll, to me, is much more exciting than two boring ones.

<DougJustDoug> I agree it is completely subjective as to what is "exciting" or not. Multiple rounds of voting, particularly in cases where everyone knows damn well which option is going to win from the first round or two, is not exciting at all.

<DougJustDoug> I hated the old click polls more than anyone. It was a nightmare in terms of encouraging good voting behavior. But there is no doubt that it was accessible as hell and everyone loved checking in on the poll thread to see the bar graph indicating the state of the race. CAP poll threads got views out the ass!

Even boring steps of the process

People would click into the poll threads just to see the current results

That is great advertising for CAP. Bottom line, this is an internet community. And on the internet, page views are king.

<paintseagull> PBV is nice in that the top 2 should result in an exciting final poll

<jas61292> Click polls are awful, in my opinion. Not just for here, but anywhere, just due to how forums set them up. Most people vote before they even see what the OP says.
Click to expand...

So, I guess the question is one of balance. In my interpretation there are three conflicting factors:
  1. Accuracy - the poll should provide the most accurate representation of the community's opinion. (IRV, PBV are good for this. I read briefly about more arguably more accurate methods with much more obscure counting algorithms)
  2. Efficiency - the poll should take up the least amount of time and not require much moderation (IRV is best for this)
  3. Community engagement - the poll should be fun to participate in and help to encourage new participants (SBV is best for this)

Single round IRV answers 1 and 2 pretty well, and there are worse methods for 3 in terms of complexity and difficulty to understand. Accuracy comes into question when a lot of voters rank a small number of options. Engagement comes into question when voters are forced to rank a certain number of options.

Birkal's proposed 2-rounded PBV + runoff is accurate (arguably redundant, but a failsafe at least), a bit less efficient, but the second vote should be more engaging and helps preserve some of our current tradition of having multiple polls. It's possible that having too many rounds becomes boring and reduces engagement.

Our old method, roughly: (MBV optional) -> IRV -> 3 candidate SBV -> 2 candidate SBV
as we've discussed, is flawed but probably was just as accurate as the above two, if only for the final winner. It was also more inefficient. I feel as though engagement was good sometimes when the last two polls were close, but it could also be too drawn out.

Sorry to throw even more things into the mix, but I hope this is a better framework for us to use while discussing the pros and cons of these and potentially other voting methods.
 
I don't think "spectacle"/voter involvement should be our biggest concern when it comes to competitive stages. Poll-watching those is only for the hardcores, and quan's probably already got a script for irv :p. While flavor (specifically, art) is a different story, I feel safe excluding community engagement from the competitive steps as a huge concern. Let's be honest—if you don't care enough about competitive steps to rank multiple options, you'll be happy as long as the CAP has a cool design, typing, and sprite. The only concern I have with condensing the polling process for competitive steps is that people have lives. At what point are a good amount of those who DO care about competitive steps unable to vote because the poll is too short in length? Can we all find a window in 24h to dedicate to voting on movepool and stats in particular? I can, but owning an iPhone significantly helps in that regard; some people only have an hour or so where they can even access a computer. If we feel that the large majority of involved CAPers will not be significantly limited by only a 24 hour voting window, then I see no reason to not simply make competitive stages a single 24h IRV vote. Jas has already addressed the argument about incomplete votes repeatedly.

The two downsides I see to this are that people couldn't all get their votes in that quick or that IRV, being imperfect, could fail to select the "true winner" especially as slate size grows. If either is significant, then I would support a system like Birkal's over one like Nyk's. Sure with Nyk's someone who only has time for one vote will get just as much of a say as those of us who spend all day on IRC, but most busy people, rather than having a solid chunk of 1h/48 have five minutes here and there. In that case, birkal's system is preferable. They can vote quickly in poll 1 for their one or two or three favorites, and if that doesn't work out, then in poll 2 they can decide among a slate of only two or three. Also it solves the problem of irv inaccuracies at least somewhat; if the "true winner" is not the irv winner then it's likely to be winner of pbv round 2 or 3.

P.S. If we're addressing time sinks in the CAP project then why not hit the big one? That is to say the line of tlt post -> tl post -> mod post at the end of a discussion thread and the similar but less egregious one at the start i see this is going to be its own thread
 
Last edited:
Ok in my opinion this is pretty simple. IRV is very good at doing its job with 3 (or frankly even 4 or 5) options. Where it gets sketchy is when you have many options. Yes, there are potential issues with people not ranking everything. That's the minor issue really though. The issue is using IRV with so many options because it only measures approval, rather than disapproval. And with so many options (AND people not ranking everything), you can get winners that are in fact not even close to the majority's preferred option.

This is where PBV comes in. As a final decision maker, yes it's designed to pick multiple winners and is therefore not suited to being used for the final poll. However, it's quite good at determining what people do like as a whole in the initial stages of voting. Imagine the following situation. There are 4 options in a poll (out of 8) that are actually getting supported to a decent degree. There are 30 votes in this poll. Options A, B, and C have 10, 7, and 6 first place votes. Option D has 3. The other 4 first place votes are split up between the remaining 4 options in the poll. In this situation, it seems quite natural on the surface that A, B, and C would advance to the next round. But what if option D was actually the second place vote on 20+ ballots? Single preference opinions can be quite misleading as to overall support. In this situation, it's actually feasible that option D might even win the first round of PBV, whereas with IRV it would have no shot of making the 3-option second round. Meanwhile, options A, B, and C might actually be wildly unpopular on the ballots of people who didn't vote them first.

In short, PBV is a better measurement of what is generally liked (or better put, what isn't disliked.) Then, IRV is the best method for determining which of a (preferably small-ish) group of options is the most popular. This is why the plan that was developed by Birkal, DLC, paint, and bmb makes sense. It has my full support. Like pretty much every voting system, it's imperfect. But I think it's highly practical for CAP's purposes and more likely to be accurate than our current system.
 
srk1214 said:
Ok in my opinion this is pretty simple. IRV is very good at doing its job with 3 (or frankly even 4 or 5) options. Where it gets sketchy is when you have many options. Yes, there are potential issues with people not ranking everything. That's the minor issue really though. The issue is using IRV with so many options because it only measures approval, rather than disapproval. And with so many options (AND people not ranking everything), you can get winners that are in fact not even close to the majority's preferred option.

This is where PBV comes in. As a final decision maker, yes it's designed to pick multiple winners and is therefore not suited to being used for the final poll. However, it's quite good at determining what people do like as a whole in the initial stages of voting. Imagine the following situation. There are 4 options in a poll (out of 8) that are actually getting supported to a decent degree. There are 30 votes in this poll. Options A, B, and C have 10, 7, and 6 first place votes. Option D has 3. The other 4 first place votes are split up between the remaining 4 options in the poll. In this situation, it seems quite natural on the surface that A, B, and C would advance to the next round. But what if option D was actually the second place vote on 20+ ballots? Single preference opinions can be quite misleading as to overall support. In this situation, it's actually feasible that option D might even win the first round of PBV, whereas with IRV it would have no shot of making the 3-option second round. Meanwhile, options A, B, and C might actually be wildly unpopular on the ballots of people who didn't vote them first.

In short, PBV is a better measurement of what is generally liked (or better put, what isn't disliked.) Then, IRV is the best method for determining which of a (preferably small-ish) group of options is the most popular. This is why the plan that was developed by Birkal, DLC, paint, and bmb makes sense. It has my full support. Like pretty much every voting system, it's imperfect. But I think it's highly practical for CAP's purposes and more likely to be accurate than our current system.
Click to expand...

I'm kinda confused by what you are saying here. You start off stating the problems with IRV, and saying it has to do with the number of options, but I can't really see where the number of options comes into play. The way it picks a winner is identical to IRV, and for a single poll that would decide it, they are the exact same thing. On the other hand, if we are having two polls, then yes, PBV is better for showing community concensus, but that has little if anything to do with the number of options. It has to do with ranking lower placing options based on everyone's opinion, not just the opinion on those who didn't prefer the first place option.

So yeah, I'm just not getting what you mean as far as the number of option, since, as far as I can tell, PBV and IRV are equally effected by the number of choices.
 
Yeah, PBV and IRV give the same 1st place winner, it's 2nd and 3rd that we need PBV for. The issue with both IRV and PBV arises when a significant number of people don't rank enough choices such that the 1st, 2nd or 3rd place options are not on their ballot at all. In this case, those people have essentially not voted. That is why a runoff after PBV would be beneficial and not necessarily redundant if we care about people who don't want to rank every option.
 
Basically, in the scenario I described above, D fails to get through on IRV, but will get through with PBV. Now let's suppose this were a poll with only A, B, C, and D as options and assume that those other 4 unpopular options that only received one first place vote each all had D as their second choice, which means if they hadn't been included, D would gain 4 first place votes. Now, the voting results would be A: 10, B: 7, C: 6, and D: 7. In this scenario with fewer options, there's no difference between the PBV or IRV results for D; it will go through, since I had indicated it has lots of second place support and is likely to win a lot of C's votes and then B's too and therefore win the overall poll in either case. But even these results are not guaranteed to be identical, though they are very likely to be.* In general, these results become more similar across voting systems as options decrease. With 4, most of the time PBV and IRV will do the same thing. However, the chance of a dissonance occurring increases as options increase, and since the initial poll has 6-8 options, there is a higher chance of having differences between who would progress to round 2 under PBV or IRV.

Alhough IRV is a simpler system and would be nice to use, the reasonably high number of options to choose from in the first poll means that IRV is not at all guaranteed to approximate the more appropriate results PBV would generate.



*As paint started to explain above, it's actually still not perfectly congruent, since there may actually still be a difference in terms of whether the PBV would indicate 2 moving on, as opposed to the 3 indicated by IRV. I can write out this scenario fully, if it needs explaining.
 
srk? I'm confused? What's your point? We already decided not to use IRV to rank candidates to move onto further ballots (runoffs) because it's not designed for that. Everyone agrees it's not designed for that. The only thing we will possibly use IRV for from now on is a final poll.
 
I'm really not sure what we're going on about if "everyone" agrees to that already. Clearly not everyone agrees, or this thread would be closed and it would already be the adopted policy. If there isn't a debate about that, what is there still debate on?

If the debate is strictly between the proposed PBV->IRV system vs a single round IRV as Nyktos has proposed, I'm happy to make some arguments for that. But honestly I hadn't even taken the possibility of single round IRV seriously because it's just an awful idea. It provides incredibly minimal benefits in efficiency (assuming this is for only competitive steps, less than a week total throughout the whole length of the process - one day each for concept, typing, ability 1, stats, ability 2, and movepool) while offering the potential of grossly less accurate results.

If it's about other things, please enlighten me. This thread is not at all clear in direction.
 
srk1214 Please re-read this post http://www.smogon.com/forums/threads/polling-options.3496436/#post-5164370

Basically there are 2 parts:
Part A: No more MBV for competitive, and no more IRV for the purposes of creating runoffs. We haven't discussed this for many posts, nobody disagreed with me earlier when I suggested we were in agreement. That we should not use IRV for runoffs is what I think you are arguing for in your above posts.

Part B: Should we change our voting process overall? (Beyond the minor tweaks above)
Nyktos and jas are arguing for ONE IRV poll for any competitive poll. There is debate over how long such a poll would be. The first place winner of this poll would be the final winner. We can run the PBV script if we are curious about the ranking of the candidates but this would not change the final outcome. We could call this SINGLE ROUND PBV/IRV, it is the same thing.
Myself and DLC I think are leaning towards a TWO ROUNDED system in which we use PBV first to get a ranking, then move the final 2/3 on to a final poll (SBV for 2, IRV for 3). This is Birkal's proposal in the thread, but I think Birkal was ok with either option.

We are discussing Part B (hopefully in the context of my post at the beginning of this page where Doug made his concerns about this stuff apparent).

Sorry to rehash this AGAIN. I hope it's clear now.
 
If people want to talk about "accuracy", we should probably try and agree on what that actually means. If accuracy is defined as getting the same results as our current system, then obviously any change has the "potential of grossly less accurate results". (Though again, single-round IRV produces the same results three times out of four anyway...) If the definition is that it would produce the same result as a series of N - 1 SBV polls (for slate size N), then single-round IRV where voters are required to rank all options is more accurate than what we have now; and IRV without the rank-all-options requirement is not inherently less so, though adding human psychology to the mix suggests that it may end up that way as voters continue to rank only one or two candidates. (Which, in the world where we use SBV for everything, is equivalent to no longer voting once the preferred candidates are eliminated – but voters may not think of it that way.) The suggestion to require ranking some particular number of options greater than 1 but smaller than the maximum slate size was my proposal to mitigate that, and doesn't seem to have drawn any strong objections; it would be nice to actually see a case made against that option from one of the people who apparently dislike it. If there's some other measure of accuracy I haven't considered, someone should post about it.

I don't want to psychoanalyze people here but I find it hard not to get the sense that "accuracy" arguments are based more in feelings than facts. It's clear that even with no requirements to rank more than one option, single-round IRV produces the same results as what we have a decent fraction of the time, and there is no doubt in my mind that requiring users to rank four or five options would make that fraction larger, even if requiring all options to be ranked is deemed unacceptable.
 
I think this comes down to what we should expect of our voters. Single ballot IRV theoretically does the best job of balancing accuracy with expediency, but it's the most cumbersome voting method. In a perfect world, every voter would rank every option and IRV would provide us with a single. definitive winner. However, it's not reasonable to expect every voter to comprise a 100% complete ballot for each competitive step, especially for movepools. For stat spreads and especially movepools, most submissions are accompanied by paragraphs upon paragraphs of reasoning (and movepools themselves take a long time to read). It's certainly not a bad thing that submitters explain themselves in great detail, but it makes it extremely difficult to understand and precisely rank every option (not to mention the recent trend in CAP to larger slate sizes). Can we expect people to determine in precise order their ten favorite movepools or their eight favorite stat spreads? I know I have trouble deciding between some of the later entries, especially when I only feel strongly about two or three. Furthermore, if someone's already looked at six entries out of a slate of ten, they could get worn out looking at entry #7, fitting it in exactly between two other options, and repeating for entries 8, 9, and 10. It's not reasonable to expect our voters to work so hard just to comprise a proper CAP vote, and IRV really doesn't work to determine a single winner when people are only ranking 2-4 options. I don't expect to be able to precisely rank every option in a stat spread or movepool poll, and I don't expect our voters to take an hour out of their day to rank nine or ten movepools.

Forcing voters to rank X options will discourage some users from voting their conscience. If a user believes that one entry out of eight is head and shoulders above the rest, then they should not be forced to add on options they don't feel deserve to move on. Furthermore, if the number of options mandated is too high (say, 5 for the purpose of this hypothetical), some voters might meticulously rank their top three (possibly the only three they feel strongly about), not feel strongly about the rest, then "mail it in" and pick two others without really thinking. When I vote in large competitive polls, I often only rank the few options I believe are head and shoulders above the rest. I believe a lot of others vote the same way.

Practically, PBV followed by one IRV ballot with the top three entries is the best way to go forward, in my opinion. PBV works well as a preliminary method, it lets people pick their top few options to gauge community consensus, then focuses the spotlight on the top three options. Ranking three options is not a cumbersome exercise, and PBV does a good enough job that the favorite option should be able to come in at least third in a PBV poll. If we end up with a two round PBV-IRV system, then I believe we should strongly encourage, but not require, voters to rank all three options. It's still not fair to force voters to rank any number of options if they only feel strongly about one.

In summary, a single-ballot IRV is the theoretical best type of vote, but it's pretty impractical when CAP submission posts and slate sizes are often large and that it would be difficult to rank every option or even a large number of options. PBV->IRV might be marginally less accurate but it's the best balance of community building, simplicity, and accuracy. As an aside, forcing voters to rank X number of options should not be on the table.
 
I know this is arguably the laziest thing I could do, but DougJustDoug and I had a conversation today on wrapping up this thread. We believe we've come to an overall community consensus. You can read the log of our IRC conversation in the hide tags below. I highlighted the key takeaways, but feel free to read the entire thing if you want a cohesive idea on the policy we intend to create as the result of the feedback given in this thread.

I will have a formal declaration written up within the next few days based on the information contained within the below chat log and the thoughts presented in this thread. If you have any final thoughts or comments, please make them now. Otherwise, we will be moving forward with this model.


[21:58] <Birkal> Hey Doug, you there?
[21:58] <@~DougJustDoug> Yes, I am.
[21:58] <&@Birkal> Groovy, should we review that polling thread now?
[21:58] <&@Birkal> or do you gotta run soon?
[21:58] <@~DougJustDoug> Sorry for not getting back on last night. But yeah, let's discuss it.
[21:58] <&@Birkal> cool, let me read it over again quick

[22:07] <&@Birkal> I dunno, Doug
[22:07] <&@Birkal> I think the only proposal out there that is generally agreed upon is mine
[22:07] <&@Birkal> single IRV could potentially work, yes, but I feel that it's a drastic change from what we currently have
[22:08] <&@Birkal> if there's a decision to be made here, it's probably between the following three options:
[22:08] <&@Birkal> 1) PBV -> One round of IRV
[22:08] <&@Birkal> 2) One round of IRV
[22:08] <&@Birkal> 3) change nothing
[22:08] <&@Birkal> I feel like there is enough support to go with 1) right off the bat
[22:08] <&@Birkal> but I s'pose we could put it to a vote

[22:12] <@~DougJustDoug> Doing 2) seems too jarring, IMO. And as the first CAP of this gen, I guess I'd like to be more conservative. Doing 1) seems like a good compromise to increase efficiency and make a reasoned voting standard for competitive polls.
[22:13] <@DHR> I think the first is better from reading what other people have said and trying it out during the prevo project
[22:14] <@~DougJustDoug> PBV with a reasonable slate, always trimmed to the top 3 and then IRV with the top 3 to determine the winner makes a lot of sense.
[22:14] <@~DougJustDoug> Even if the voting calculation is opaque to most voters, I think everyone can get the gist of it

[22:15] <&@Birkal> are we going to force ranking all three options with top 3?
[22:15] <@~DougJustDoug> First round uses ranking to get a top 3. Then another, easier to understand, rank and vote will set the finsihing order
[22:16] <@~DougJustDoug> I don't think we need to force everyone to rank all three. It would be more of a mod headache than anything
[22:16] <@~DougJustDoug> But we should encourage everyone to rank all three.
[22:16] <&@Birkal> sure, I think that's reasonable
[22:16] <&@Birkal> so we're setting hardlined top 3 at the start of every competitive poll, yah?
[22:17] <&@Birkal> what if there is a tie between options 3 and 4?
[22:18] <&@Birkal> we could always do double elimination there
[22:18] <&@Birkal> but then that asks for what happens if 2, 3, and 4 tie
[22:19] <@DHR> Then 1 wins by default
[22:19] <&@Birkal> I'd probably just say any tied amount from 3 to lower ones move on

[22:19] <@~DougJustDoug> I would say include all four in the final IRV. Theoretically we could get in a bind if there are lots of ties, which causes a final IRV of 6 or 7 options. But I think that is unlikely. And if so, we can call an audible and deal with it.
[22:22] <@~DougJustDoug> I'm sure there will be some very close polls where a 4th option misses the final poll by one vote, and people will complain that in the past that 4th option would have been advanced. But there will always be controversy, and we're trying to make our polling more EFFICIENT -- meaning a combination of speed and accuracy.

[22:23] <@~DougJustDoug> I think having a set two polls for every competitive slate will be a big time improvement, and I don't think we lose much (if anything) in terms of accuracy.
[22:23] <@~DougJustDoug> And it won't be perceived as a massive shift from what we do today.
[22:24] <@~DougJustDoug> If we think we are getting comfortable with the PBV->IRV later on, then we may consider moving down to just one IRV poll in the future.
[22:24] <&@Birkal> and even though we say just competitive polls right now
[22:24] <&@Birkal> non-competitive ones will likely follow suit in the near future

[22:24] <&@jas61292> I just think it seems silly. We make it two votes, but all its actually doing is counting it the same way twice.
[22:25] <&@Birkal> but not necessarily bound to the smaller slate that competitive polls are required to have
[22:25] <&@Birkal> it is silly from a completely objective point, yes jas
[22:25] <&@Birkal> but you have to consider that this is CAP, and that single IRV puts a lot of importance on the voter's singular vote
[22:26] <&@Birkal> whereas it's a bit more acceptable to "miss a day" when there is two rounds
[22:26] <&@Birkal> it also makes voters feel more powerful, and makes it a bit more of a spectator sport
[22:30] <@~DougJustDoug> I think there is a lot of subjective value in the cutdown, by focusing the community. It makes a clear statement of "the final three" which heightens awareness of the 'race" and history has shown that voters behave differently when the choices are narrowed.
[22:30] <@~DougJustDoug> Not saying they behave better or worse, but it does change things.

[22:38] <@~DougJustDoug> Also, don't discount the level of interest by submitters in finishing position. Meaning, people care whether they come in 5th versus 6th place. The former can be bragged about a "Top 5 finish" and the latter cannot. And one huge issue with ranked ballots in the first place, is that lower finishing positions are very susceptible to gamed ballots.
[22:38] <@~DougJustDoug> This is typically an issue with the large polls like art, but other ballots apply too
[22:40] <@~DougJustDoug> IRV really does very little to indicate who finished in any position other than first place.
[22:40] <@~DougJustDoug> PBV does a better job of that.
[22:40] <@~DougJustDoug> MBV is the best for discouraging gaming, but it's not great for determining winners at all

[22:41] <+bugmaniacbob> how would the PBV+IRV model work with art polls, specifically regarding how many pieces go into the eventual gallery?
[22:42] <@~DougJustDoug> Art will still be MBV ->PBV->Head to head
[22:42] <+bugmaniacbob> ah ok
[22:43] <@DetroitLolcat> All flavor polls will be, right?
[22:44] <@~DougJustDoug> And I want to have a more formal way of determining gallery placement. I've been chatting with the art mods about the gallery for Cawmodore
[22:44] <@~DougJustDoug> For all flavor polls, we're keeping the same voting we've always had.
 
I liked the proposal of always cutting to two better than the one with two ranked polls, but I don't feel like trying to reignite debate so unless other people feel the same I suppose I don't really have an objection at this point.

I mean I still think the whole thing is ridiculous but I guess it's an incremental improvement.
 
<&@Birkal> what if there is a tie between options 3 and 4?
Click to expand...
Cut to 2 instead. I still think having it flexible between 3 and 2 on the final ballot is best. Sometimes it is obviously pointless to include a 3rd place candidate on the final poll. We should not be forced to do this.

<@~DougJustDoug> Also, don't discount the level of interest by submitters in finishing position.
Click to expand...
I'm not for a single round system, but this isn't a valid point against it. Single round PBV can be used instead of single round IRV- it is the exact same thing and we get ranks. Problem solved.

<@~DougJustDoug> For all flavor polls, we're keeping the same voting we've always had.
Click to expand...
Not exactly what I had in mind. Yes we'll still do MBV first. However:
a) We're using PBV as our ranked system across the board from now on. No more misunderstandings amongst mods on this. This will make a difference.
b) There's no reason not to standardize the poll after PBV in the same way as we're doing for competitive polls. This saves us the tediousness of having two SBV polls and I do not think two SBV polls is more exciting than one 2/3 candidate PBV. It is just longer and really can be boring in the end depending on the race. If we explicitly have two SBV polls for the top 3 in flavour and one PBV for the top 3 in competitive, I think that looks really bad. There's no way to justify it to people without just looking like we're desperate for pageviews and postcounts in our forum.

Along with this change should be a push to encourage people to rank as many options as they can. If we change the voting culture in CAP we can keep the option open for changing to a single round system in the future.
 
It looks like we've come to a conclusion here. We'll give the following method a go for CAP 1. If it doesn't work for us or needs tweaking, we will address it at a later date. Voting procedure in the Create-A-Pokemon Project will be changed as follows:

All Competitive Polls:

1) Preferential Block Voting (Round One)
- Consists of voting on anywhere between 6-8 options.​
- Voters may rank as many or as few options as they like.
- Thread always lasts for 24 hours.
2) Preferential Bold Voting (Round Two)
- Consists of voting on the top three options of last round.
- Any option that ties with the third ranked option also moves on to Round Two.​
- Ranking all three options is encouraged, but optional.
- Thread always lasts 24 hours.
- The final winner is always decided at this stage.
- Ties are resolved as follows:
- The number of votes received in the previous poll
- The Section Leader's choice
​
3) Other Notes
- Non-competitive polls will be run at the discretion of the CAP Moderation team.
- Will likely consist of MBV, followed by the above process.​
- Preferential Bold Voting (PBV) is now the standard over IRV in the CAP Project.
​

Let me know if I missed anything here. Otherwise, we're all done! Thank you all for helping out with this discussion. This should help speed up the CAP Process while still keeping the consistency of votes. Good work.
 
Status
Not open for further replies.
Back
Top