Policy Review Policy Review - Polling Process

[Deck Knight]

Approved.

Policy Review: Polling Process

If you are not an experienced member of the CAP community, it is strongly recommended that you do not post in this thread.

This thread is intended to contain intelligent discussion and commentary by experienced members of the CAP project regarding CAP policy, process, and rules. As such, the content of this thread will be moderated more strictly than other threads on the forum. The posting rules for Policy Review threads are contained here.

This post struck me by user foibles:

I was saying that darkie was way better then deck knight, not his movepool.

foibles later employed the “I was kidding” defense or some such. The fact such things as username even influence a competitive project is ridiculous, no matter now n00b or pro the person is.

Which leads me to a serious proposal: Blind polling.

Blind polling would be conducted for any poll that requires submission from users. It would work by having people submit their submissions to the TL and then updating them via PM of any changes they would like.

Once the submissions are in, they get named based on the order they were received e.g. Name 1, Movepool 1, etc. They will be posted in the second post of a topic and updated just like the Allowed/Controversial/Unallowed moves are in the movepool topics.

Ideally, there would be a deadline for submissions, and then updates a few days later.

PMs would be formatted in such a way:

CAP X Part y: Movepool Submission

Updates would be:

CAP X Part y: Movepool Submission (Update)

After the poll is complete, the submitter of the winning option would be announced.

Pros:
- People vote for the best option rather than on their personal opinion of the submitter.
- Allows newer, lesser known users an equal playing field with more experienced users.
- Retains “bragging rights” while diminishing bandwagoning based on popularity.

Cons:
- Requires more time and availability of the TL
- Is more difficult to moderate on polls like Art, where feedback is appreciated and the designer could accidentally out themselves.
- Might cause idiotic rumors to surface like "this is clearly the work of Cartoons!, go Cartoons!"

I’m open as to which polls we should apply this to. Complete Movepool and Base Stats Submissions are the ones I feel require this change the most, since they are the most important competitively, and tend to be attached to a username directly. Art is so fluid and dynamic that I’m not sure it would be practical, but an opinion would be helpful all the same.

 

[Fuzznip]

The Art poll would be a complete mess if we had to run it like this to be honest. Like you said, no one can get feedback and they can think it's done by someone they know, as you said Cartoons!.

I'm fine if we do that with the Movepool and Base Stat submissions, but we should have a Movepool/Base Stat discussion thread before we do the blind polling so people know what they must submit. We could probably do this with the Sprite poll aswell.

I don't see any major problems, it's just more stressful for the TL. Getting like, 20 pms a day. xD

 

[ParadOxymoron]

This is a really great idea! Seriously. I just don't think it's really necessary for, though — and, as you said, would really get in the way of — the art polls. EoE beating out DougJustDoug proved that much. The sprite polls, too. I think it would be easy enough to pick out which one is Wyverii's, for example, and besides, noone's ever voted in a sprite poll based on anything but the quality of the sprite.

It's only in things like stat spreads and movepools, when they're often quite similar in many aspects, causing people to be able to go either way, when reputation of post count and how many times they have won the poll in the past can come into play.

 

[Colloquy]

This is a good idea. As others have said, it prevents people from voting on reputation and stuff. People were even saying to vote for certain user's work.

I think it should be in effect in art polls and sprite polls. Does it matter who created the stuff you're voting for? Just vote for it if you like the art! The TL can reveal the artists at the end, or when they get picked off. You can get feedback, too, just not over PM or something.

So, I'm all for this, but we might want to test this in a CAP round before making it the official way to do it.

I don't see any major problems, it's just more stressful for the TL. Getting like, 20 pms a day. xD

20 PMs a day isn't that much. Mafia hosts have to get even more, and they have more work to do in their games than just putting the stuff in a thread.

Also, DK, you shouldn't mind what foibles says, that was just a dick move that wasn't funny at all.

 

[Kaxtar]

First and foremost, I am not going to act like a true, expirenced member of the CAP project, however I do know all my stuff.

Deck Knight, you are a godsend for bringing up this topic. I fully support it. Many people always vote for the most expirenced members, especially in the Base Stat and Movepool polls. Do you thinnk it is a fluke that Darkie, Beej, Deck Knight, aragornbird and the like almost always win polls? Many people vote for the person who they know more, and agree with. I'm not flaming all those people, but I'd wonder who would really win an important poll if it was done blind. And furter more, how can all these noobs with great minds and ideas, ever become a driving force in this process if their only way was to get more attention than more respected community members? Only in art and spriting polls should it not be done blind, because all art is a WIP, and the artists need feedback from the community on how to improve their designs.

Again, I fully support blind polling in all polls except art and spriting.

 

[Wyverii]

Blind polling is interesting but I personally think it should be based on the TL's discretion as for which polls to apply it to if any. A TL should know when it is necessary and like choosing whether to use a bold or click vote poll they should be able to make this as easy as it should be for the entrants and voters.

As for flavour based polls i'd discourage blind polling. It's a waste of effort on what is already a popularity contest though newcomers can certainly do well in them regardless.

I put out my support to this idea.

 

[zyrefredric]

this is my first CAP, and for the short time i've spent here, i have to say that there is a great possibility for me (as a noob) to just vote for those who i think is the most prominent member (darkie, DJD, Deck Knight), esp on those polls that i have not much inkling about, such as BST, biases, movepool, etc. honestly in these polls i just voted for the one who seemed to be the most popular without even knowing what i am voting for (i have a lot to learn)

same with Kaxtar ^^, "I fully support blind polling in all polls except art and spriting".. I voted for Cyzir's art and Wyveri's sprite NOT because they're popular, but because i simply believe their works were purely awesome ^_^
when it comes to art and sprite, the person who created it doesn't really matter

 

[Cartoons!]

Regarding the comments on artist feedback, don't people who make BSTs and movepools generally like to receive feedback as well? It seems anything that would restrict the sharing of ideas would do more harm than good in the long run.
A compromise might work where, say, the movepool submission thread operates as usual, but the poll itself is blind. People who vote but who haven't been paying attention to the process wouldn't be able to vote by name unless they scoured the submissions thread which a) would make them less ignorant as they'd have to keep track of the details or b) they'd be too lazy to do anyway (alternatively, the submissions thread could just be temporarily hidden).

It's great that you brought this up though, as voting by popularity is an issue that doesn't seem often enough discussed.

 

[hydrolphin]

At first glance, I thought this would be an excelent idea. However, I found the one issue. It prevents discussion about what we are voting on. The only place where I see this as being easily done is with the movepool, but for stats, art, and sprites, we need a discussion thread. With these, we could just take the name off at the poll, so someone actually would have to look and think.

The movepool is what I find to be most different than the rest, as once one is posted, there are generally no dramatic changes anyway. My proposal is if this notion gets enough support in this thread, we do a test run on CAP8 on just the movepool, to see how things work. If it works, then we put it into effect. If it fails misrably, then we won't have COMPLETELY screwed up CAP8.
Also a recomendation: If we do do this, than why not make it the ATL's job. The ATL has little to do anyway, and this will give them a job to do. It also will make the TL's job less hard.

 

[Colloquy]

At first glance, I thought this would be an excelent idea. However, I found the one issue. It prevents discussion about what we are voting on. The only place where I see this as being easily done is with the movepool, but for stats, art, and sprites, we need a discussion thread. With these, we could just take the name off at the poll, so someone actually would have to look and think.

I fail to see how it prevents discussion. You might say that you can't discuss it because you can't say, "Oh, I prefer Addie's design over Katherine's", but what if the TL just gave them numbers, so you could say that you liked #1 over #4 or something. In a discussion thread, does it really make a difference if you know who created something, this way?

I may have totally misunderstood your argument here, though.

 

[hydrolphin]

What I meant, is developmental discussion, or how something can be improved. Someone can't find out if people actually like their first Stat spread, so they don't have a chance to edit it. This generally isn't a problem with the Movepool though, because normally edits to the movepool are because it wasn't legal, not because people don't like it.

All I'm really saying is we shouldn't jump into this to quickly. We can try it with the movepool first, and if it works, try it elsewhere.

 

[DougJustDoug]

I agree with Deck Knight that "name recognition" may swing some votes in some polls. I don't think art polls or sprite polls is really a problem, but perhaps is a bigger factor in stuff like movepool and stat spread.

HOWEVER, you need to remember that having a submitters name next to a submission on a CAP poll, is a form of "reward" for the submitters. It's a little slice of e-fame on the CAP project -- not just for the winner, but for everyone that makes the slate and advances in the polls. This recognition should not be dismissed. For longtime project members, perhaps you don't care if your name is in big bold letters in the poll. But for new members, that's a sign of accomplishment and credibility to be the submitter of a "top option" -- even if you don't win.

Subtle motivating factors like this are a big part of the fuel that keeps the CAP machine running. As the head of the CAP project, I spend a lot of time looking at things like this -- since contrary to popular opinion, projects like Create-A-Pokemon do not magically stay afloat on their own. It's very easy to inadvertently cut off the project's life blood, in the interest of "fairness" or "efficiency".

Yes, I agree that blind polling might be the fairest way to conduct a poll. But removing the recognition for all the submitters, will cause more harm to project motivation, for very little net gain by having a slightly fairer polling outcome for certain particpants.

This is not unlike the suggestion a while back to have completely hidden polls -- in an attempt to prevent bandwagoning. But, like I said then, the ability to monitor poll results is a huge part of the momentum and excitement of a CAP project. To lose that excitement is too big a price to pay for a small benefit in polling fairness.

This is pretty much the same thing. Yes, DK is probably right. But, I still oppose the proposal, because of the hidden harm this could cause -- which may not be immediately obvious to many of you.

 

[jagged_angel]

I fully support this proposal despite Doug's concerns. I would hope that revealing the name of the winning BST/Movepool submitter after their option has won would be the chance for the creator to have their slice of e-fame.
If people congratulate the winner of said polls once they've been announced, I would hope it would still create enough warm fuzzies for people to feel rewarded.

I see this proposal also as a way to Encourage participation, not reduce it. I know I personally would be more likely to submit things if I thought the fame of X-Act or darkie was not going to put me at a disadvantage right from the start.
I wonder how many others feel the same way? Perhaps if blind polling was introduced, more people would participate as they all have a fair chance, and people would put more effort into their submissions as it would really be down to that one post to speak for them.

In conclusion, I welcome this proposal, as long as the winner of each blind poll is announced and subsquently feted when they win it.

 

[Goodbar]

I think this is a great idea, except for the art polls. I also don't think voting based on user is a problem in these art polls, because they're aesthetically pleasing to follow and because of that easier to formulate opinions on. Like, I don't have to look far to know that darkie is a smart experienced user and that could be reason enough for me to vote on his stat spread submission, but I don't think the sheer fact of knowing who is a good artist is going to stop people from viewing all the art pieces and making a decision.
I also don't see how discussion is stunted, or how people wouldn't be able to send out pms to have their submissions changed. You'd just be referring to a number rather than a name. Something else that came to mind is that this might be a task that could be taken up by the ATL, so the TL doesn't have that much extra work and the ATL has at least that much involvement.
Also Doug I realize your concerns but I think those can be fixed by making the submitters name visible after the polling. They're not usually included in the evolving pokemon summary but I wouldn't be opposed to that being changed to allow for the blind voting. Win/win?

 

[Deck Knight]

I agree with Deck Knight that "name recognition" may swing some votes in some polls. I don't think art polls or sprite polls is really a problem, but perhaps is a bigger factor in stuff like movepool and stat spread.

HOWEVER, you need to remember that having a submitters name next to a submission on a CAP poll, is a form of "reward" for the submitters. It's a little slice of e-fame on the CAP project -- not just for the winner, but for everyone that makes the slate and advances in the polls. This recognition should not be dismissed. For longtime project members, perhaps you don't care if your name is in big bold letters in the poll. But for new members, that's a sign of accomplishment and credibility to be the submitter of a "top option" -- even if you don't win.

Subtle motivating factors like this are a big part of the fuel that keeps the CAP machine running. As the head of the CAP project, I spend a lot of time looking at things like this -- since contrary to popular opinion, projects like Create-A-Pokemon do not magically stay afloat on their own. It's very easy to inadvertently cut off the project's life blood, in the interest of "fairness" or "efficiency".

Yes, I agree that blind polling might be the fairest way to conduct a poll. But removing the recognition for all the submitters, will cause more harm to project motivation, for very little net gain by having a slightly fairer polling outcome for certain particpants.

This is not unlike the suggestion a while back to have completely hidden polls -- in an attempt to prevent bandwagoning. But, like I said then, the ability to monitor poll results is a huge part of the momentum and excitement of a CAP project. To lose that excitement is too big a price to pay for a small benefit in polling fairness.

This is pretty much the same thing. Yes, DK is probably right. But, I still oppose the proposal, because of the hidden harm this could cause -- which may not be immediately obvious to many of you.

My thought process on the recognition of the winner at the end does not preclude having a sort of summary of ALL submitters. I suppose retro-editing the polls could work, but seems impractical, most people I imagine don't look back at old polls.

People know their own submissions, so if for example I was Movepool 2 and GT was Movepool 5, I would know which Movepool was mine, and know I made it into the Top 4, Top 2, etc. Anyone discussing any of the Movepools would grant me insight into what I should do to change mine.

If we were to release the names of all submitters after the final poll is conducted, we would be able to address your concerns. I know it isn't as immediate as seeing your username attached to something as the polls progress, but I understand that "bragging rights" for lack of a better term, are important.

It's fairly clear this process would be ineffectual for Art. It's mostly Final Movepool and Base Stats Submissions where this is most applicable. Type, Allowed Moves, and Name don't have a direct username attached to them, and I wouldn't even bother trying this with Concept Submissions. At the time Concept is decided, the stakes aren't so terribly high that people can't work their way around them.

 

[Darkflagrance]

I'm fine if we do that with the Movepool and Base Stat submissions, but we should have a Movepool/Base Stat discussion thread before we do the blind polling so people know what they must submit. We could probably do this with the Sprite poll aswell.

To add to this suggestion, if we implemented blind polling the best policy would be to ask potential submitters to send prototype movepools to the TL/ATL and have them edited into the original post or posted somewhere in the discussion thread before moving on to polling, so that the community can compare them and discuss them.

 

[darkie]

Overall, I don't like this proposal mostly because it's unnecessary. Sure, my movepool won the poll, but maybe it was because, well, more people liked it! Likewise, Elegy of Emptiness's artwork won because more people liked it.

The idea itself is not bad, but I believe it's a little too idealistic.

Do you thinnk it is a fluke that Darkie, Beej, Deck Knight, aragornbird and the like almost always win polls?

That has been the first CAP poll I've ever won besides the TL votes, so I really don't know why you're saying this, lol.


edit: What happens if, during the course of a poll, a submitter releases which submission was theirs?

 

[billymills]

I think I'd support this for concept and base stats only.

When there are so many concepts and base stat spreads submitted, people often get voted for secondary reasons (not wanting to read the whole OP for instance). I see no problem with implementing this addition for those two polls.

I'm not exactly sure how we should attempt movepool. Tennis' idea should be tried at least once for movepool, and only 1 change needs to happen. If tennis' idea works as it should, then the person who submitted will not even matter. Otherwise, we could test this on movepool the time after.

For art: art doesn't matter. I don't care about how people vote for art, because people may just dislike it. Art is person by person, and most of the time, they aren't going to vote for something they hate. (This is unlike concept and base stats because 'I voted ______ because he seems smart' is a common reason, and should be removed).

 

[jagged_angel]

Overall, I don't like this proposal mostly because it's unnecessary. Sure, my movepool won the poll, but maybe it was because, well, more people liked it! Likewise, Elegy of Emptiness's artwork won because more people liked it.

I'm not sure you can compare Movepool and Art in this respect, since art requires real talent, massive amounts of work, is far more easily judged regardless of the creator, and is most importantly a Flavour aspect of the project.

You're right in saying that Maybe your movepool was just more favourable but given the similarities between the two movepools and the fact that some people admit they are swayed by the submitter, it's unlikely. There's simply no way to tell how much of an influence username/post count/fame has unless this proposal is implemented.

 

[Flounder]

I too, support this idea but in a limited fashion.

Most, if not all, user-based submissions can be influenced just by the "oh, I like this guy better" instinct. I'm not going to mention names, however as I'm sure some of you are aware, during the movepool and a few other parts of CAP 8, many well-known members of the community acted completely out of character just because they didn't like the submission(s) of a particular user.

Art, sprite, and name are hardly crucial or competitive, so it's not like influence based on submitter is a huge issue. In the case of movepool and stat spread, it can make a notable difference. The "discussion" areas of something such as movepool should not be eliminated in the event that this is voted in, obviously.

All in all, I'm in favor of this.

 

[Kaxtar]

Do you think it is a fluke that Darkie, Beej, Deck Knight, aragornbird and the like almost always win polls?

That has been the first CAP poll I've ever won besides the TL votes, so I really don't know why you're saying this, lol.

edit: What happens if, during the course of a poll, a submitter releases which submission was theirs?

Well Darkie, I was just only using your name for refrence in that you recently won a poll, and you are well known. But the and the like part is what is important.

For example, X-Act won the movepool polls for both CAP 5 and 6. Now he did a few cool things with them, especially that brillant move with the Egg Move thing for Argo. And I proabaly would have voted for him anyway.
But for those who just look at the names of the submitters, and not even bother looking at the actual thing their voting for, do you think those people would vote for the noobish member whose submission might be better than a respected member's or that respected member own submission?

Situation 1

Or if you want me to put it this way, if it was a Base Stat Poll, for example, and you could not look at the actual submission, and the choice was down to Kaxtar and X-Act (lol), who would you vote for, X-Act-the math god himself, or Kaxtar-who is hell is he anyway? Now, you'll all say this is bull, such curcumstances are next to impossible, when pigs fly, etc. blah blah blah... But what if this situation was in reverse?

Situation 2

What if, it was down to submission 3 and submission 5. Kaxtar's was submission 3 and X-Act's was submission 5. Let's say that Kaxtar's is actualy better than that of X-Act's (heavens forbid), you would end up voting for Kaxtar, right? All this fame and genius stuff is out of the race, and it only comes down to who has the better package, the better Stats.

I'd much rather have this be the case the Situation 1, even more so then what we have now. This way, everyone gets a fair chance at winning, and overall quality should increase. Nobody can really complain, as long as the winner is announced at the end of the poll. Those noobs can feel proud that they won, and those that are highly respected can say that is was not my fame that won this poll for me, but that many people like my submission. Win-Win.

Now there are a few disatvantages of polling blind I'd like to adress, along with possible solutions.

TL availibility is one, as DK already said. But that can be worked around with many soultions. One such possible option is for the TL to assign TL Helplers (for lack of a better name) to split up PMs and post seperatly, so one/two TL/ATL will not be overloaded with work. Another is to extend the time allowed for submissions.

For the second problem that DK mentioned, just skip the blind polling on Art and Sprite polls.

For that third problem, the idotic rumors, those people could be infracted, posts deleted, and have the poll go on.

And to you Darkie, your question, hopefully people will not say what their own submisson is. But if so, the post could be deleted, poster infracted, and the poll could continue. If the idoit who said that was telling the truth, it would be the TL's responsibility to have that person's submission removed.

And I'd just like to add, thanks for reading this long and (hopefully not) boring post. If anyone has questions or comments, ask or PM away. And I mean NO disprect to those who's name I either directly or inderictly mentioned. Although I was not lying about how X-Act is our god and all xD.

 

[darkie]

Situation 1

Or if you want me to put it this way, if it was a Base Stat Poll, for example, and you could not look at the actual submission, and the choice was down to Kaxtar and X-Act (lol), who would you vote for, X-Act-the math god himself, or Kaxtar-who is hell is he anyway? Now, you'll all say this is bull, such curcumstances are next to impossible, when pigs fly, etc. blah blah blah... But what if this situation was in reverse?

Situation 2

What if, it was down to submission 3 and submission 5. Kaxtar's was submission 3 and X-Act's was submission 5. Let's say that Kaxtar's is actualy better than that of X-Act's (heavens forbid), you would end up voting for Kaxtar, right? All this fame and genius stuff is out of the race, and it only comes down to who has the better package, the better Stats.

Why can't I look at the submission? Your descriptions are completely unobjective. If someone votes without looking at the submissions now, why would they be compelled to look at them without names? In this case, putting numbers to the submission is no better than the case we have now.

And to you Darkie, your question, hopefully people will not say what their own submisson is. But if so, the post could be deleted, poster infracted, and the poll could continue. If the idoit who said that was telling the truth, it would be the TL's responsibility to have that person's submission removed.

What if that option was winning with a VAST majority of the vote? Is it fair to the community to strike down an option that literally everybody wants just because someone tells which option is theirs?

 

[moi]

The concern I have with this is that the support given to a submission by a user will potentially allow you to identify them as the creator as most users supported their own until it was out of contention.

Also just chipping in saying this definitely isn't needed for the art

 

[Grimoire]

I just wish to point out that the only reason why this could be considered unnecessary for art but needed for stats would be because art is a flavour bit. Being a member of a few art forums and having followed several art contests. . . well let us just say people get really silly about art.


As for Blind-Polling itself. . . well I have had bad experiences with it. So far every regular contest type thing I have followed that switched to blind polling has either bottomed up or drastically decreased in participation(to the point where it was half dead, at least in comparision to how it was before). In addition bandwagon'ing and the ilk still occurred. Arguably this was due to reasons other than the switch and they were less 'serious' than CaP is. . . but I am careful when it comes to rocking the boat with blind voting.

If you wish to end that sort of stuff the only thing I have seen that remotely worked was the 'well thought out' vote requirement scheme. People still tried to bandwagon, but at least now they had to find an 'acceptable' reason to do so(at least you are making them think). The problem is it did the same thing as blind-polling. Greatly killed off participation. Both in submissions and in voting.

I have seen combinations of polling and thought out voting where a poll vote(a vote without a thought out reason attached) counted as far less than a thought out vote. That was somewhat interesting and appeared to work(the same people were winning, but at least now there was plenty of valid reasoning backing them). In addition neither participation nor voting decreased after this was used. In fact it was garning MORE participation than in the past(though arguable it was just more well known by then).



Long story short I would not be against a trial run on blind voting, but I really do not blindy support it being fully implemented, yet. Combined with my unfortunate experiences, it just sounds like a lot of work on the TL's part with little gain, and possibly some losses.

 

[tennisace]

Personally I think this is a horrible idea. Gauging support is one of the BIGGEST parts of moving submissions from discussion to the actual poll. There were parts of CAP 5 where I had to go on judgement alone along with support from users. Another huge part of things like movepool (and really any other submission poll) is that ALL submitters have the ability to respond to feedback, whether it's on IRC, the Server, or on the forums. That wouldn't be allowed here because you're limiting the democratic process in a sense. In congress, representatives don't just blindly submit bills to the speaker, and vote on them. They have a chance to defend their proposition and make changes accordingly. With this, there is no room for that crucial debate portion, except by proxy.

Another reason why I don't support this proposal is because it puts a ton of strain on the TL. TLs already take enough crap from everyone when they don't put so and so's submission in. Some TLs could feasibly rig the poll, and nobody would know the difference if they can't reveal the submitter!

Finally, a lot of you are assuming that the people you recognize most don't make good submissions, when in reality that's usually the reason people are recognized here, for good ideas. This is besides the fact that a "best submission" is completely subjective, and as we all know, everyone in CAP has their own ideas about how a project should turn out.