1. Welcome to Smogon! Check out the Smogon Starters Hangout for everything you need to know about starting out in the community. Don't forget to introduce yourself in the Introduction and Hangout Thread, too!
  2. Welcome to Smogon Forums! Please take a minute to read the rules.

Policy Review Topic Leadership

Discussion in 'CAP Policy Review' started by DougJustDoug, Dec 8, 2012.

  1. I agree with srk and CiteAndPrune, but want to add a couple of other thoughts. First, there has never been a TL poll (to my knowledge) that featured only one candidate that clearly appeared to be qualified. Unless one is ever completely satisfied with the outcome of a CAP, one can always speculate as to what another TL might have done differently. Sure, CAP 4 was an extreme case, but we had no way of predicting that beforehand, nor can we ever truly know what would happen in any project with a different TL, however appealing that hypothetical scenario will always be to someone.

    Second, community members demonstrate their TL qualification (aka what gets them past the initial screening) by being active in the forum, and making insightful and persuasive posts. Even if a new member decides to vote with only limited knowledge of the candidates, for whom are they most likely to vote? I would assume the people that make frequent and persuasive posts, and while "insightful" is less of a certainty, it tends to help with the "persuasive" part. How else would a so-called "noob" have any idea whom to vote for? This point is conjecture, and I don't actually know how well this problem solves itself, but I would imagine that it does to some extent. Remember, also, that BMB was the kind of insightful poster we wanted before he was actually chosen as TL, and that one of the major points of the TLT model is to prevent a TL from running the project into the ground anyway.

    I hear what you're saying, Pwnemon, but I just don't think it's enough to end a community vote. If the TL vote had never been so open, I might look at this differently, but I think we should have stronger reasons than these to take a vote away from the greater community.

    I'm also against simultaneous TL and TLT elections, because again, I don't think the drawbacks to holding them separately that you identified outweigh the benefits of simplifying the process. As I said in my previous post, one of the most important things about CAP leadership is that it be easy to understand, which means that it is best to avoid simultaneous votes and any possible confusion over the results.
  2. DougJustDoug

    DougJustDoug Knows the great enthusiasms
    is a member of the Site Staffis an Artistis a Programmeris a CAP Contributoris an Administratoris a Battle Server Admin Alumnusis a Smogon IRC SOp Alumnusis a Tiering Contributor Alumnus
    CAP Leader

    Joined:
    Jun 26, 2007
    Messages:
    3,098
    Before I post the conclusion to this thread, I wanted to make a point about TLT qualifications.

    In the model going forward, candidates are going to be pre-qualified (probably by CAP mods) that they meet certain basic qualification criteria. I'm not sure how specific those qualification requirements will be, but we will not be accepting candidates that are random noobs signing up in the nomination thread.

    Some of you may remember a user named Linton that used to run for TL every CAP back in the early days. That wasn't the problem, the problem was his nomination post was pretty much the ONLY post he made in the CAP forum every project! The guy was almost completely unknown, and he had very little activity or involvement in the project aside from click-voting in polls -- yet for some reason he felt qualified to run for Topic Leader every CAP. He actually became kinda famous for appearing out of nowhere every CAP and running for TL. And since the PRC had to rank all candidates on their ballot, they had to include this Linton bozo. Predictably, Linton came in dead last every time.

    In this model, we won't be allowing users like Linton to even make the poll. Mods will reject the nomination outright. I don't think we should even expose ourselves to the possibility of troll voting to taint the process (like how troll voting caused BOTW to get shut down).

    But all "unqualified" nominees may not be so clear as the Linton case. Most people that throw their hat in the ring for TL are incredibly well qualified. But with a new lower bar and more positions for the TLT, I think we could run into some trouble if we don't put in some defined qualifications.

    I think we should require all TL and TLT candidates to have won or "placed well" in a certain number of prior CAP polls to be eligible to serve as TL or the TLT.

    The main function of topic leadership will be to lead discussions. But ultimately, these leaders will be making subjective calls about community consensus. If we elect leaders that have a PROVEN track record of being part of the community consensus or even DRIVING community consensus (which must happen if you win a poll) -- then we have a much more reasonable assurance that their subjective opinion is actually representative of the majority viewpoint, EVEN if they decide to go nuts and railroad the process.

    Not to keep harping on BMB, but if you look back on his participation history in CAP -- Bob never won anything. He never even made a final poll, as far as I can see. To the best of my recollection, Bob only made it to one slate in his entire participation history. And if you remember Bob's posting history -- Bob tended to be contrarian much of the time in discussions. Don't get me wrong, Bob was a workhorse and was INCREDIBLY active in CAP. But, in hindsight, it's pretty obvious that BMB's outlook on what is good for creating pokemon, was NOT generally shared by the community at large. Bob's rise in CAP, and the reason he was supposedly "qualified" for TL was mainly due to his activity and diligent work ethic. But as a leader of community opinion, he was almost completely unsuited for the job.

    Like I said, I see this now only in hindsight (which is 20/20). At the time, I was very satisfied that BMB was perfectly qualified for the job. But now, when comparing BMB's track record of driving majority opinion, it's hard to put it in the same league as someone like capefeather (who narrowly lost to BMB for TL), who has outright won a couple of CAP polls, and has placed well in others. I am NOT saying if capefeather was selected, the project would have worked out great. I'm just saying that there IS some objective information available to us that gives us an indication of whether people are "representatives of the CAP majority", and we should start using that information when qualifying candidates.

    The trend of good TL's with histories of winning polls goes way back. Deck Knight has probably won more polls than anyone else in CAP history, and he has finished near the top countless times. Rising Dusk was a consistent winner before becoming TL, and his posting history led public opinion in almost every CAP he participated in. Reachzero won the Concept for Krillowatt (and concept is a VERY difficult, very subjectively-influenced poll to win), and was a consistent decision influencer in many previous polls. Darkie, tennisace, Beej -- all these guys were poll winners and top finishers in CAP before becoming TL. Basically, even if they were "selfish" at times as TL, they tended to be aligned with the community anyway, which probably acted as a natural mitigator to negative consequences.

    I'm not too sure what exact requirements we should place on Topic Leadership candidates. I think winning certain polls should probably "count more" than winning others. I also think a "top finish" is relative to the size of the poll and the number of quality options. Perhaps we don't even need a hard requirement. Simply list the requirement stated above, and ask candidates to give their resume that they think exemplifies the qualification. Then mods can make a subjective call based on the evidence supplied in the nomination.

    Regardless of the exact requirement we put in place, I think all of us as CAP leaders need to start vocalizing the idea that a person's CAP poll track record is an indicator of whether their opinions are representative of the majority. It needs to be a voting criteria. Not the ONLY criteria, but A criteria. Right now, it rarely comes up in discussion of candidate qualifications, and I think it is very relevant.
  3. DarkSlay

    DarkSlay This Wouldn't Have Happened In Romney's Smogon
    is a CAP Contributor

    Joined:
    Aug 12, 2009
    Messages:
    1,414
    I had a nice chat in IRC about how I very much dislike using poll victories as a measurement to decide who is fit to run for TL and TLT. Here's the transcript below (don't worry, it's not too long. Really just my thoughts on the matter.):
    Show Hide
    [16:38:52] <DarkSlay> Hi CAP.
    [16:39:02] <SpecsX> hi
    [16:39:30] <DarkSlay> So, I really don't like using poll victories as a way to choose TL/TLT candidates.
    [16:40:23] <DarkSlay> I just think the application process, in which pretty much a candidate's entire history of contributing to the CAP process as well as involvement in polls and such, should suffice.
    [16:41:41] <SpecsX> so pretty much like the prc
    [16:42:12] <DarkSlay> Except more thorough and more selective. More effort, to be exact.
    [16:42:28] <SpecsX> i see
    [16:42:34] <SpecsX> im reading dougs post now
    [16:42:42] <DarkSlay> Really, though, yeah, the current system does just fine. It suffers from laziness, since many people read the names and not the submission posts.
    [16:43:43] <DarkSlay> Going off of poll victories, on the other hand, takes a step backwards in the direction we want to go for two reasons:
    [16:44:34] <DarkSlay> One, if you thought choosing winners through popularity was an issue before, imagine if poll victories were put into the equation (no offense to poll winners and finalists, including myself. Just an observation.)
    [16:45:50] <DarkSlay> Two, I think using poll victories as a huge weight in the TLT/TL selection process is complex and unfair. Is someone who won the sprite poll weighted the same as someone who won the movepool poll? What about contributing members who have never won a poll?
    [16:47:20] <DarkSlay> If you want a case in point about how unreliable polls are, I offer pretty much the furthest I've ever gone in a poll: CAP11 movepool poll. Anyone remember that issue?
    [16:48:05] <SpecsX> yes
    [16:48:12] <SpecsX> er
    [16:48:15] <DarkSlay> It's not to say that my movepool wasn't good, but that whole drama between Dusk and Jibaku was pretty much the reason why I made it to the finals and had such a close race.
    [16:48:16] <SpecsX> i agree with you
    [16:48:20] <SpecsX> not i remember
    [16:48:30] <DarkSlay> I'd be the first to admit it.
    [16:48:39] <SpecsX> yeah sometimes there are just better submissions than yours
    [16:48:50] <SpecsX> and theres not a whole lot you can do
    [16:49:07] <SpecsX> particularly if you're an artist
    [16:49:55] <DarkSlay> I actually think the current system is great, but as it stands, it's good "on paper" but not "in practice", as I'm under the belief that many people just don't read TL application posts and just go at voting immediately.
    [16:51:08] <SpecsX> bob made a worthy candidate in other ways
    [16:51:16] <SpecsX> he definitely contributed
    [16:52:10] <DarkSlay> tbh Associating Bob's tyrannical escapade being prevented with the TL selection process doesn't really solve our problem. Putting his agenda forwards wasn't a result of him never winning a poll.
    [16:53:24] <SpecsX> well i mena
    [16:53:26] <SpecsX> mean
    [16:53:33] <SpecsX> maybe he wanted to win that badly?
    [16:53:37] <SpecsX> i dont think so
    [16:53:48] <DarkSlay> That's going a bit too psychological, isn't it? lol
    [16:54:03] <DarkSlay> Are we trying to deduce the movements of a serial killer now? :P
    [16:54:04] <SpecsX> yeah
    [16:54:30] <SpecsX> HE MUST BE A SERIAL KILLER NOW!!!!!!!!!
    [16:54:36] <SpecsX> it all makes sense
    [16:54:44] <SpecsX> he loses the respect of CAP
    [16:54:47] <SpecsX> he kills peopel
    [16:55:00] <DarkSlay> Just emphasize the current TL application process, and make voters actually pay attention to TL application submissions.
    [16:55:25] <DarkSlay> As for ideas on how to accomplish this, a 24 hour grace period in which a thread detailing the sumbissions only, before the poll is put up, might be a good idea.
    [16:55:31] <DarkSlay> That's at least a start.
    [16:55:49] <DarkSlay> There will always be bias, but at least this gives voters the chance to be informed.
    [16:56:02] <SpecsX> yes
    [16:56:09] <SpecsX> and pretty much forces them to
    [16:56:11] <DarkSlay> Or at least, the opportunity to be biased. Not "always".
    [16:57:13] <DarkSlay> Since TL/TLT selection will essentially be done after PR stuff and before the project, it won't really add time to the project either.
    [16:57:29] <SpecsX> right

    To sum up:

    • Using poll victories as a means of selection has the potential to be complex, unstable, and (frankly) unfair.
    • The current process, in which applicants post applications for the PRC to vote on, is sound as is on paper. What needs to be changed is the emphasis on voters actually looking at the applications and applicants posting thorough anecdotes about their CAP contributions.
    • In order to help achieve this, two things should happen: one, TL slate selection should be more precise and restrictive depending on the application received (as in quality and even length). Two, a grace period of at least 24 hours should be made to put up a thread with only the slate's application posts, so that voters have the time to read applications.
  4. DougJustDoug

    DougJustDoug Knows the great enthusiasms
    is a member of the Site Staffis an Artistis a Programmeris a CAP Contributoris an Administratoris a Battle Server Admin Alumnusis a Smogon IRC SOp Alumnusis a Tiering Contributor Alumnus
    CAP Leader

    Joined:
    Jun 26, 2007
    Messages:
    3,098
    What follows is a post I made in April 2010 in the private Smogon staff forum, when Deck Knight was nominated to be a CAP Moderator. Some non-CAP members opposed DK becoming a CAP mod because of his controversial history in Congregation, where people felt DK was often offensive and disruptive with his political and social views.

    This is the post I made, and I think it says a lot about achievement at the highlest levels of the CAP project, and what is required to win or place well in CAP contests.

  5. Birkal

    Birkal We have the technology.
    is a member of the Site Staffis an Artistis a Super Moderatoris a Community Contributoris a CAP Contributoris a Smogon Media Contributoris a Battle Server Admin Alumnus
    CAP Head Mod

    Joined:
    Oct 12, 2010
    Messages:
    3,815
    We've talked about the credentials for TL and TLT applications in #cap for about the past two hours. We reached a general consensus among ourselves, but I thought it'd be prudent to throw up the results here in the thread. If you agree with these, then great! If not, speak now or forever hold your peace wait until the end of CAP5 to make a PR thread about it.

    TL Application Requirements:
    (Fulfills either of the following requirements)

    1) Has finished once in the top 2 of any official, named CAP voting stage, specifically concept, stats, and movepool.

    2) Has done any of the following twice:
    a) Finished in the top 4 of any official, named CAP voting stage, specifically concept, stats, and movepool.

    b) Was the driving contributor to a non-named, official CAP voting stage, specifically abilities and typing. This will need to be backed up with evidence and is subject to interpretation by the moderators.​


    TLT Application Requirements:

    1) Has done any of the following once:
    a) Finished in the top 4 of any official, named CAP voting stage, specifically concept, stats, and movepool.

    b) Was the driving contributor to a non-named, official CAP voting stage, specifically abilities and typing. This will need to be backed up with evidence and is subject to interpretation by the moderators.​


    When I say "named" and "non-named", that refers to polls where the user's name is visibly presented and used in the polls. That include concept, stats, and movepool polls. Non-named would be typing and ability polls. Note that for non-named, you need to be a LARGE driving contributor to that discussion. This would likely be evidenced by the applicant being a) the first one to significantly develop the idea and b) the one arguing for it throughout the discussion thread.

    If we go with these credentials, we'd have well over twenty members applicable for TLT for CAP5 based on contributions to Gen5 CAPs alone. I feel that these requirements are easy enough that a lot of people can apply for TLT or TL, and yet it prevents those who have little idea of how to lead our community from obtaining a competitive leadership position.

    Any thoughts? If you got 'em, say them quick, because I think DJD is intent on closing this soon.
  6. paintseagull

    paintseagull pink wingull
    is an Artistis a Forum Moderator
    Moderator

    Joined:
    Jul 22, 2010
    Messages:
    943
    So we are considering putting rules onto TL applications instead of onto the rules for the conduct of TLs/Section Leaders once they get the job? Restricting TL apps further (they are already fairly limited) will likely give us less applications, whereas making the code of conduct for TLs more clear and strict should give us more applications (less fear of screwing up, more guidelines to follow). Seems counter-productive to the goal of empowering the community as much as possible.

    I can see the logic behind the TL application requirements but it sets an awfully bad tone in my opinion.

    Sorry if I seem so contrarian in this thread all the time, it's just that all the obvious options that come to my mind are not being explored at all (at least from what I can read)
  7. srk1214

    srk1214 Epke Motherf**in Zonderland
    is a CAP Contributoris a Tiering Contributor

    Joined:
    May 28, 2010
    Messages:
    2,476
    The following is entirely selfish, though I hope justified:

    Restricting to poll-winners/high placers, as far as my memory serves, keeps me out of eligibility for a TLT position. I'm sure it also keeps other good users out. I'm totally fine with this being the position we take if we are operating under a definition where a user, me or anyone else, reasonably well-established who has yet to place high in a poll is not qualified. But I'm not so sure I'm "unqualified" due to not placing well in polls. I have been slated for major competitive aspects 3 times. But iirc I've gone out in the first round of voting each time. I hardly think I'm the only qualified person, nor do I think that these restrictions are so onerous we will be wanting for applicants. But, as I said, being selfish, frankly I want to be a TLT member. Not necessarily on CAP5. But I do want to be able to at least apply, and I fear these "qualifications" do keep certain qualified people out.

    I do see myself as qualified. If you disagree, feel free to disregard this entirely. And please tell me, too, so I can save myself the embarrassment of applying and being considered unqualified.



    As an additional point, I can't think of too many scenarios in which an unqualified applicant would be slated or even then voted for, restrictions or no.
  8. capefeather

    capefeather no shit
    is a Forum Moderatoris a CAP Contributoris a Battle Server Moderator Alumnus
    Moderator

    Joined:
    Apr 26, 2009
    Messages:
    2,884
    To me, the poll success "requirement" isn't much of anything new. It's something that would, or at least should, be part of a TL's credentials regardless (and most past TLs did have it). People already use posts that they made in past CAP project discussions as evidence to justify their self-nominations. So I don't think that setting it "in stone" would disqualify very many (if any) potential candidates from being nominated. If it turns out in CAP 5 that only a couple of people run for TL, well, I highly doubt that that would be the fault of this requirement, which is practically an unspoken requirement anyway. I know I originally didn't want to run for TL until I won every competitive stage at least once, though I relaxed that a bit to provide an alternate option for the underpopulated CAP 3 and CAP 4 nominee list. This just seems like a natural way for people to get a sense of how much sway they have over the CAP community, which is a big part of being able to lead discussions with it. It can be argued that this could encourage more applications from people who may not necessarily have had an idea as to what it takes to be a TL other than perhaps being part of some "CAP elite".

    Yes, we should certainly fix the TL guidelines themselves. However, as people have been saying, we're not trying to deal solely with "bugmaniacbob", and fixing the TL guidelines doesn't solve every problem that we could be solving, and that's what the model changes and visible application requirements set out to do.
  9. DougJustDoug

    DougJustDoug Knows the great enthusiasms
    is a member of the Site Staffis an Artistis a Programmeris a CAP Contributoris an Administratoris a Battle Server Admin Alumnusis a Smogon IRC SOp Alumnusis a Tiering Contributor Alumnus
    CAP Leader

    Joined:
    Jun 26, 2007
    Messages:
    3,098
    While I think the idea of qualification requirements has merit, I don't think we'll get this ironed out quickly. I knew when I posted it, that I was throwing this into the mix very late in the game.

    I think we have enough on our plate with the new TL + TLT system. The idea of qualification requirements is not a "make or break" aspect of this whole review, and I really want to conclude this PR and move on to the next one.

    So let's take the entire concept of explicit qualification requirements off the table for CAP 5. We can bring it up again after CAP 5, if it makes sense at that time.

    I'll post the conclusion to this PR tomorrow morning.
  10. jas61292

    jas61292 used substitute
    is a Forum Moderatoris a Community Contributoris a CAP Contributoris a Tiering Contributoris a Battle Server Moderator
    Moderator

    Joined:
    Sep 30, 2010
    Messages:
    3,021
    I know Doug just said that they won't happen for CAP5, but I just want to briefly post about it to say that I am adamantly against any hard restrictions. The fact is, while many qualified candidates will have won polls or at least done well, doing so really doesn't mean shit to me. As we say all the time, CAP is about the journey, not the destination, and as such we should be emphasizing quality contributions in discussion, not success in polls. Its true the two often go hand in hand, but that is not necessarily the case.

    While I hate using myself as an example, I have no better one to go to than that. As I'm sure most of you know, this past CAP I ran for TL for the second time and came in a close second place to bmb. I know many people here supported me, and I would like to think that I have the credentials to be qualified for TL. However, at that time, had there been any hard requirements based on poll success, I would not have been allowed to run. Until CAP4, I had never even been slated in a poll, and I still have never won one. But I don't think anyone of the people on this PRC who supported me would suddenly think that I am not qualified because of that. Many times I have been a primary supporter of winning options, so if we are looking for a track record of being part of community consensus, it is definitely there, but poll results do a horrendous job of showing that.

    That being said, I don't think supporting winners really is important either. CAP is about learning, and just because your first choice didn't win does not make you less of a contributor. It is about what you add to the project with your posts, not whether it ended up on the final product. We should not be reacting to this CAP by making adding requirements that are necessary and not even good representations of qualification. Correlation does not equal causation, and we should not be making rules that are clearly based on the idea that they are the same.
  11. DougJustDoug

    DougJustDoug Knows the great enthusiasms
    is a member of the Site Staffis an Artistis a Programmeris a CAP Contributoris an Administratoris a Battle Server Admin Alumnusis a Smogon IRC SOp Alumnusis a Tiering Contributor Alumnus
    CAP Leader

    Joined:
    Jun 26, 2007
    Messages:
    3,098
    (co-written by Birkal and DougJustDoug)

    Conclusion:

    We are going to move ahead with the TLT model. Although there is some speculation on how viable this model is, the general consensus seems to be that we should try it out. The main driver of this new model will be to re-focus topic leadership on driving and identifying the "intelligent community consensus". We still want strong leaders and we want them to post actively in threads to drive the community to make high-quality submissions and decisions that are consistent with the chosen Concept. But we also expect our discussion leaders to always remember that this is a community project, first and foremost. The collective desires of our intelligent community participants is more important than any one person's vision or opinions, topic leader or otherwise.

    We will use this new topic leadership model for CAP5 and reconvene as a PRC once it's over to compare the advantages and disadvantages of the two models. But for now, we are going to move forward with the TLT. Here is what the Topic Leadership positions will look like:

    - Topic Leader
    - Topic Leadership Team
    --- Typing Leader
    --- Abilities Leader
    --- Stats Leader
    --- Movepool Leader

    The Topic Leader retains their name, while the others are called the TLT. Individual members of the TLT, referred to non-specifically will be called a section leader. In this model, the TL is in charge of making the slate for concept polls. Otherwise, their job is to make sure the concept is represented throughout the process. They will also be in charge of the Concept Assessment and Threats Discussion. In terms of public relations, the TL is essentially the leader of the project. They do not make the majority of the slates, but they should be a vocal presence in each stage of the process. Their goal should be "quality control" not "project control". They should strive to ensure that the community always considers the chosen concept AND the concept assessment when making decisions.

    The TLT will be in charge of making the slates in their respective section. They will essentially work like the past TL, but only in one specific section. They should be active discussion members, helping shape and guide the discussion. By dividing up the workload, and since their responsibilities last for a relatively short time, we are assuming we will have active TLT participation in every step of the CAP process. During the sections where they are not leading, they will be allowed to make submissions.

    CAP Process Guide and Topic Leadership Rules
    The CAP process guide will be edited to incorporate these new changes. The Topic Leadership Guide will be rewritten almost completely to reflect this new model and approach to topic leadership. In that guide we will include a specific list of rules that topic leaders and section leaders must follow, including some broad language that explicitly states that all topic leaders actions must support the spirit and best interests of the CAP project, regardless of any other specific rules. It will clearly indicate that CAP mods are responsible to make interpretations as to topic leaders adherence to the rules and that CAP mods have the power to enforce those rules however they see fit.

    Some specific points of process that will be incorporated into our process guide in the approriate places:
    Vetos

    The TLT member will create their slate. The TL then has a few options. They can either add one item to the slate or delete one. The third option is that they can veto the slate entirely; in that case, the CAP moderators will make a new slate that adequately addresses the consensus of the community. Regardless of what the TL does, they must provide reasoning. If their reasoning is not adequate or supported by the previously held discussion, then the moderators will make a new slate.

    This can be referred to as -1/+1/veto in the future.


    Elections

    All votes will be public. Applications will go first, where users can apply for either the position of TL and/or general TLT. These are just like current TL applications; users can say which areas they'd prefer to be a TLT member for. The TL poll will take place first. Once they are selected, the TLT will be voted on. The top four members will be our TLT. The most popular voted member will get first choice on which section they'd like to leader. The second most popular chooses their preferred area, and so on. Hopefully, the TLT could just chat about who they think would fit best where, but that's the official process if it needs to come down to it.


    General Logistics

    The CAP mods will still make the process threads. They will make sure that +1/-1/veto has proper reasoning. If there are any arguments between the TL and the TLT, the moderators will serve as the mediators.​


    Our concluding thought is that we should do CAP5 with the TLT model as thoroughly as possible. If we do it half-heartedly, we won't have a good comparison with the TL model. Let's put forth our full effort with this TLT model and see what unfolds.

Users Viewing Thread (Users: 0, Guests: 0)