In this thread, there are a few issues related to polling that I would like us to address. This includes the types of polls we use for competitive votes, the sizes of slates for competitive polls, and when we should be using preferential block voting (PBV), the vote counting system introduced to count the Topic Leadership Team selection vote, rater than instant runoff voting (IRV). While the first two issues are a lot more linked together than the third, I believe that they all fall under the same general concept of polling issues and are similar enough to address together in one thread.
To give a little background, the reason I wanted to make this thread during this PR cycle was because of something I noticed this past CAP project that I could not remember ever seeing in the past: the use of multiple bold voting (MBV) for the first round of competitive polls. This happened both for the concept poll and the movepool poll. Having looked through the archives, I have found that this was not the only time this has been done, having also been used for the Necturna movepool poll, but, in general, this is something that has not been typically done, so it surprised me that it came up, not once, but twice this project.
Now, what exactly is the issue with this that I think needs discussion? Well, I believe that, in a large slated competitive poll, having unranked votes is a detriment to the project as it overly encourages strategic voting and makes it impossible for people to express more than their vaguest preferences. As I said earlier, the reason I wanted this thread was because of what happened this past project, but to be more specific, it was because of the realizations I had when trying to vote in the first movepool poll. As a lot of you may remember, I was very much against giving Cawmodore all of Acrobatics, Drain Punch, Roost, and other moves. However, I was not equally against all of them. Acrobatics was my number one sticking point, so, if this were a ranked vote, I would have put those movepools without Acrobatics first, followed by those with, ranked based on other moves included. However, this was not a ranked vote. Everything I would vote for would be given equal weight, meaning I would have to decide whether it was more important to vote trying to get rid of those I disliked most, or to stick with my number one most important thing and only vote for the movepools without Acrobatics. I essentially was forced to choose a breaking point and vote based on what I wanted to see (or not see) going forward, and not based on my actual preferences.
To put that in more general terms, an MBV on a competitive poll essentially forces people into strategic voting, something which I believe we do not want in any stage of the competitive process. So, with that said, the first thing here I want to propose is the following:
Proposal - No future concept, typing, stats, primary ability, secondary ability or movepool poll will ever use Multiple Bold Voting
Maybe I'm wrong about this, but I'd like to believe that, for the most part at least, the above proposal will be easy for people to agree on. However, simply saying that we want to do the above is not really a solution. MBV is typically used when we have large slates (around 7 or more) in order to narrow it down, so if we are going to eliminate the use of MBV in competitive polls, we need to do something about situations that involve larger slates.
One easy solution would simply be to use a different type of voting, either IRV or PBV (or something else if anyone has a better idea) in such a situation. Now, I don't know a ton about the statistics of polling, but while, according to the CAP site, MBV "is typically better than IRV for determining the rank of entries between first and last place for polls with very large submission numbers," I believe that for the purposes of competitive polls, no slate that we will likely have will be big enough for any such advantage gained from MBV to be worth the negatives that come with it, nor large enough for the results of an IRV or PBV poll to be significantly inaccurate.
With that said, while I believe that using IRV or PBV for these polls would be better than MBV, I think that simply saying we should do that is ignoring an important issue: why suddenly this project did it come up more than once when throughout the rest of the generation it has only come up one time total? I think the answer can be found in a general trend towards larger slates this past project. Check out the slate size for the competitive steps over the past five projects (all the projects since we made typing into a single poll):
While there are a few exceptions, I think it is pretty plain to see that in general, the trend over the past couple years has been towards larger slate sizes. This is especially evident over the last two projects. CAP 5 set multiple high marks, only to have CAP 6 set just as many the next time, and be at least tied for the high on every single stage but one (typing, where the only project with a larger slate was Mollux: the one with the concept all about typing).
So, why exactly have we seen such increases? Personally, I believe this to be because of the TLT system we implemented prior to CAP 5. Within our current system, each section leader lead discussion and sets slates, while the TL can add or remove from the slate base on their interpretations of the concept. The key element here that I believe leads to large slates is the fact that we have essentially told the section leaders to just look for what the community wants, and leave the analysis for how they fit with the concept to the TL. Now I'm not going to sat that this is a bad thing, but I believe that, in having done this, we have lead our section leaders to start slating a much larger number of things than they otherwise would. Since their job is just to see what the community thinks is a good idea, many times I have gotten the impression that TLT members have decided to just slate anything that got a single intelligent supporter.
Now that sounds all nice in theory, but I really believe that, in doing that, TLT members are ignoring an important part of their role. The job of a section leader when it comes to slating is to make sure that all options that have received significant intelligent support get slated, and while I comment all our TLT members over the past two projects for making sure the "intelligent support" part is satisfied for all things slated, I look back and see that there has been more than one case where things were slated which have had only one or two people providing intelligent support, which seems to show a lack of attention to the "significant support" part.
Note: I don't believe anything regarding this "significant intelligent support," or really anything regarding how a TLT member should be making a slate, has actually been codified in any way. My use of the phrase is based on interpretations of the intent of the TLT system from the thread in which it was created, as well as countless discussions on the topic that I have had with numerous users on IRC over the past year.
Of course, it is always possible that I am wrong about this. Maybe the larger slates are just a coincidence and have nothing to do with the TLT system. Either way though, I personally believe that these incredibly large slates are a problem, and that fixing slate size is a much better solution to the problem of using MBV in competitive polls than simply using another voting method, regardless of how effective it is.
I don't want to throw out any specific proposals on this issue yet without at least some discussion, but I'd like people to at least think about the possibility of setting some sort of limits on slate sizes. I find it hard to believe that, in any given poll, there is ever really more than 6-7 options that really have enough support to be considered viable slate contenders in comparison to the other things being considered.
So, what I would hope to discuss here are:
A) if using IRV or PBV for large slates is acceptable
B) whether or not decreasing slate size is desirable
C) positives and negatives surrounding implementing limits on slate sizes
D) any alternative ways people may come up with of reducing the slate sizes without hard limits
E) any alternative ways people may come up with of fixing the problem of MBV in competitive polls that don't involve reducing slate size or using IRV/PBV
-------------------------------------
In addition to all of the above, there is one more thing related to polling that I would like to discuss. When we implemented the TLT system back before CAP5, we had to come up with a new polling method for the selection of the TLT members. IRV was deemed undesirable for this job as second place in an IRV simply says who would have the most support among people who did not support the first place winner, not who actually has the second most support. We wanted the TLT to consist of the 4 people with the most overall community support, so taking second, third and fourth place in an IRV would not give us what we are looking for. So, instead of IRV we decided to introduce PBV. PBV is a form of voting similar to IRV, but specifically designed to pick multiple winners. Rather than having second place be the person with the most votes by people who did not prefer the eventual winner, PBV finds the IRV winner and then, to find second place, removes them from contention and runs an IRV again, finding the most prefered option of the entirety of the community among all the options other that the first place winner. This repeats until you get the desired number of winners.
While this system is perfect for TLT elections, it would also seems to fit in well in almost any poll where we previously would only use IRV. In fact we already have begun using it elsewhere, including in Art Poll 2 for both CAP 5 and 6, Name Poll 2 for CAP 5, and Movepool Poll 2 for CAP 6. As you can see though, there is not really any consistency as to why or when to use PBV rather than IRV, and that is what I'd like us to discuss. So, let me spark such a discussion with this proposal:
Proposal - Preferential Block Voting will replace Instant Runoff Voting in all standard CAP polls
I personally believe that PBV, as a method designed to pick more than one winner, is more appropriate for our purposes than IRV in pretty much every instance that we currently use it. When we have more than 3 options, we never set out to choose only one winner, as IRV was designed to. Now, in the end we do eventually want to pick only one for any poll other than TLT selection, so in theory the winner of the IRV, which would be the same as the winner of the PBV, should always win the final bold vote. But, we know that is not always the case. The difference with narrowing it down with PBV is that the options voted on in the final bold votes will be the three overall most prefered by the community, rather than the communities number one vs the number one of the smaller portion of the community that did not prefer the first place, vs the number one of the even smaller portion that did not prefer either of the above. This could lead to more blowout final votes, as the competition might be everyone's second favorite, but no one's first, but I believe it would overall cause votes to be more accurate to community preference. As a side benefit PBV could help reduce drama by not always postponing showdowns of controversial options until the last poll.
I don't particularly see a ton that is favorable about sticking to IRV, but if someone who knows more about these methods or the statistics thereof and would want to bring up some points, please do.
-------------------------------------
As a final note, if anyone has any other issues regarding polls, polling methods or slates, feel free to bring them up in this thread. If anything seems large enough to merit its own thread, we can deal with that as it comes up, but for now I think we should try and keep any poll related topics in here.
To give a little background, the reason I wanted to make this thread during this PR cycle was because of something I noticed this past CAP project that I could not remember ever seeing in the past: the use of multiple bold voting (MBV) for the first round of competitive polls. This happened both for the concept poll and the movepool poll. Having looked through the archives, I have found that this was not the only time this has been done, having also been used for the Necturna movepool poll, but, in general, this is something that has not been typically done, so it surprised me that it came up, not once, but twice this project.
Now, what exactly is the issue with this that I think needs discussion? Well, I believe that, in a large slated competitive poll, having unranked votes is a detriment to the project as it overly encourages strategic voting and makes it impossible for people to express more than their vaguest preferences. As I said earlier, the reason I wanted this thread was because of what happened this past project, but to be more specific, it was because of the realizations I had when trying to vote in the first movepool poll. As a lot of you may remember, I was very much against giving Cawmodore all of Acrobatics, Drain Punch, Roost, and other moves. However, I was not equally against all of them. Acrobatics was my number one sticking point, so, if this were a ranked vote, I would have put those movepools without Acrobatics first, followed by those with, ranked based on other moves included. However, this was not a ranked vote. Everything I would vote for would be given equal weight, meaning I would have to decide whether it was more important to vote trying to get rid of those I disliked most, or to stick with my number one most important thing and only vote for the movepools without Acrobatics. I essentially was forced to choose a breaking point and vote based on what I wanted to see (or not see) going forward, and not based on my actual preferences.
To put that in more general terms, an MBV on a competitive poll essentially forces people into strategic voting, something which I believe we do not want in any stage of the competitive process. So, with that said, the first thing here I want to propose is the following:
Proposal - No future concept, typing, stats, primary ability, secondary ability or movepool poll will ever use Multiple Bold Voting
Maybe I'm wrong about this, but I'd like to believe that, for the most part at least, the above proposal will be easy for people to agree on. However, simply saying that we want to do the above is not really a solution. MBV is typically used when we have large slates (around 7 or more) in order to narrow it down, so if we are going to eliminate the use of MBV in competitive polls, we need to do something about situations that involve larger slates.
One easy solution would simply be to use a different type of voting, either IRV or PBV (or something else if anyone has a better idea) in such a situation. Now, I don't know a ton about the statistics of polling, but while, according to the CAP site, MBV "is typically better than IRV for determining the rank of entries between first and last place for polls with very large submission numbers," I believe that for the purposes of competitive polls, no slate that we will likely have will be big enough for any such advantage gained from MBV to be worth the negatives that come with it, nor large enough for the results of an IRV or PBV poll to be significantly inaccurate.
With that said, while I believe that using IRV or PBV for these polls would be better than MBV, I think that simply saying we should do that is ignoring an important issue: why suddenly this project did it come up more than once when throughout the rest of the generation it has only come up one time total? I think the answer can be found in a general trend towards larger slates this past project. Check out the slate size for the competitive steps over the past five projects (all the projects since we made typing into a single poll):
Code:
Poll | CAP 2 | CAP 3 | CAP 4 | CAP 5 | CAP 6 |
-----------------------------------------------------------
Concept 5 7 7 7 9
Typing 4 6 4 5 5
Ability 1 4 3 5 4 6
Ability 2 - 2 3 5 5
Stats 5 7 7 8 8
Movepool 8 7 7 9 10While there are a few exceptions, I think it is pretty plain to see that in general, the trend over the past couple years has been towards larger slate sizes. This is especially evident over the last two projects. CAP 5 set multiple high marks, only to have CAP 6 set just as many the next time, and be at least tied for the high on every single stage but one (typing, where the only project with a larger slate was Mollux: the one with the concept all about typing).
So, why exactly have we seen such increases? Personally, I believe this to be because of the TLT system we implemented prior to CAP 5. Within our current system, each section leader lead discussion and sets slates, while the TL can add or remove from the slate base on their interpretations of the concept. The key element here that I believe leads to large slates is the fact that we have essentially told the section leaders to just look for what the community wants, and leave the analysis for how they fit with the concept to the TL. Now I'm not going to sat that this is a bad thing, but I believe that, in having done this, we have lead our section leaders to start slating a much larger number of things than they otherwise would. Since their job is just to see what the community thinks is a good idea, many times I have gotten the impression that TLT members have decided to just slate anything that got a single intelligent supporter.
Now that sounds all nice in theory, but I really believe that, in doing that, TLT members are ignoring an important part of their role. The job of a section leader when it comes to slating is to make sure that all options that have received significant intelligent support get slated, and while I comment all our TLT members over the past two projects for making sure the "intelligent support" part is satisfied for all things slated, I look back and see that there has been more than one case where things were slated which have had only one or two people providing intelligent support, which seems to show a lack of attention to the "significant support" part.
Note: I don't believe anything regarding this "significant intelligent support," or really anything regarding how a TLT member should be making a slate, has actually been codified in any way. My use of the phrase is based on interpretations of the intent of the TLT system from the thread in which it was created, as well as countless discussions on the topic that I have had with numerous users on IRC over the past year.
Of course, it is always possible that I am wrong about this. Maybe the larger slates are just a coincidence and have nothing to do with the TLT system. Either way though, I personally believe that these incredibly large slates are a problem, and that fixing slate size is a much better solution to the problem of using MBV in competitive polls than simply using another voting method, regardless of how effective it is.
I don't want to throw out any specific proposals on this issue yet without at least some discussion, but I'd like people to at least think about the possibility of setting some sort of limits on slate sizes. I find it hard to believe that, in any given poll, there is ever really more than 6-7 options that really have enough support to be considered viable slate contenders in comparison to the other things being considered.
So, what I would hope to discuss here are:
A) if using IRV or PBV for large slates is acceptable
B) whether or not decreasing slate size is desirable
C) positives and negatives surrounding implementing limits on slate sizes
D) any alternative ways people may come up with of reducing the slate sizes without hard limits
E) any alternative ways people may come up with of fixing the problem of MBV in competitive polls that don't involve reducing slate size or using IRV/PBV
-------------------------------------
In addition to all of the above, there is one more thing related to polling that I would like to discuss. When we implemented the TLT system back before CAP5, we had to come up with a new polling method for the selection of the TLT members. IRV was deemed undesirable for this job as second place in an IRV simply says who would have the most support among people who did not support the first place winner, not who actually has the second most support. We wanted the TLT to consist of the 4 people with the most overall community support, so taking second, third and fourth place in an IRV would not give us what we are looking for. So, instead of IRV we decided to introduce PBV. PBV is a form of voting similar to IRV, but specifically designed to pick multiple winners. Rather than having second place be the person with the most votes by people who did not prefer the eventual winner, PBV finds the IRV winner and then, to find second place, removes them from contention and runs an IRV again, finding the most prefered option of the entirety of the community among all the options other that the first place winner. This repeats until you get the desired number of winners.
While this system is perfect for TLT elections, it would also seems to fit in well in almost any poll where we previously would only use IRV. In fact we already have begun using it elsewhere, including in Art Poll 2 for both CAP 5 and 6, Name Poll 2 for CAP 5, and Movepool Poll 2 for CAP 6. As you can see though, there is not really any consistency as to why or when to use PBV rather than IRV, and that is what I'd like us to discuss. So, let me spark such a discussion with this proposal:
Proposal - Preferential Block Voting will replace Instant Runoff Voting in all standard CAP polls
I personally believe that PBV, as a method designed to pick more than one winner, is more appropriate for our purposes than IRV in pretty much every instance that we currently use it. When we have more than 3 options, we never set out to choose only one winner, as IRV was designed to. Now, in the end we do eventually want to pick only one for any poll other than TLT selection, so in theory the winner of the IRV, which would be the same as the winner of the PBV, should always win the final bold vote. But, we know that is not always the case. The difference with narrowing it down with PBV is that the options voted on in the final bold votes will be the three overall most prefered by the community, rather than the communities number one vs the number one of the smaller portion of the community that did not prefer the first place, vs the number one of the even smaller portion that did not prefer either of the above. This could lead to more blowout final votes, as the competition might be everyone's second favorite, but no one's first, but I believe it would overall cause votes to be more accurate to community preference. As a side benefit PBV could help reduce drama by not always postponing showdowns of controversial options until the last poll.
I don't particularly see a ton that is favorable about sticking to IRV, but if someone who knows more about these methods or the statistics thereof and would want to bring up some points, please do.
-------------------------------------
As a final note, if anyone has any other issues regarding polls, polling methods or slates, feel free to bring them up in this thread. If anything seems large enough to merit its own thread, we can deal with that as it comes up, but for now I think we should try and keep any poll related topics in here.
Last edited:
