1. New to the forums? Check out our Mentorship Program!
    Our mentors will answer your questions and help you become a part of the community!
  2. Welcome to Smogon Forums! Please take a minute to read the rules.

Pokemon Testing: Standard Ladder or Suspect Ladder?

Discussion in 'Pokémon Policy' started by Aeolus, Jan 28, 2009.

  1. Aeolus

    Aeolus Bag
    is a Tutor Alumnusis a Tournament Director Alumnusis a Site Staff Alumnusis a Battle Server Admin Alumnusis a Smogon IRC SOp Alumnusis a Tiering Contributor Alumnusis a Contributor Alumnusis an Administrator Alumnus

    Joined:
    Sep 12, 2005
    Messages:
    3,639
    Ok, soon we will be making a decision about how to handle the next round of suspect testing. With participation at a record low for the most recent test of Latias (only 24 people qualified for voting consideration), the idea that testing could be carried out on the Standard ladder surfaced. Essentially, the decisions hinges on one tradeoff: participation vs. stability of the standard ladder.

    If we conduct the test on the standard ladder, participation will be through the roof and many more users will qualify to vote on the fate of the suspect. However, there is the chance that many users will be turned off by the appearance of a formerly uber pokemon on the ladder and will leave Smogon's server for an alternative server.

    If we conduct the test on the suspect ladder, participation will be low and the decision to (semi-permanently) alter the standard game after the test will rest with very few. The upside to this idea is that our standard ladder remains stable and the status quo of our dominance atop the shoddy registry remains in tact.

    We want your input.
  2. Great Sage

    Great Sage

    Joined:
    Jul 31, 2006
    Messages:
    6,666
    As a compromise, perhaps "reverse" the current positions of the standard and suspect ladders? The suspect ladder will become the default ladder like standard is now, but there will still be an alternative ladder available to play "real" OU.
  3. Imran

    Imran
    is a Team Rater Alumnus

    Joined:
    May 3, 2008
    Messages:
    1,292
    I am all for keeping the suspect ladder, I outlined my reasoning here.

    EDIT: That post is a little wordy, in short I'm worried about participation on the standard ladder, as you outlined in the OP
  4. kd24

    kd24 yo daddy musta been a drug dealer? why, cuz you're dope
    is a Pokemon Researcheris a Site Staff Alumnusis a Forum Moderator Alumnusis a Smogon IRC SOp Alumnusis a Tiering Contributor Alumnusis a Contributor Alumnusis a Smogon Media Contributor Alumnusis a Battle Server Moderator Alumnus

    Joined:
    Dec 21, 2007
    Messages:
    5,289
    i think we should just use the standard ladder for testing. 1.) i really doubt we would lose many people, if any at all. people are attracted to smogons name because many battlers on here are the best and there is still a uu and uber ladder to use. secondly, the people "testing" the suspect will become huge. It will become much easier to determine the metagame and to achieve the proper voting rating. the more people voting, the better chance that the desicion is correct.
  5. Caelum

    Caelum qibz official stalker
    is a Site Staff Alumnusis a Smogon IRC AOp Alumnusis a Forum Moderator Alumnusis a Tiering Contributor Alumnusis a Contributor Alumnusis a Battle Server Moderator Alumnus

    Joined:
    Apr 5, 2008
    Messages:
    1,656
    What in the world ever gave you that idea.

    Before I make a comment, would you be resetting the standard ladder ratings each month? If so, that's not fair to people that do not want to participate.
  6. kd24

    kd24 yo daddy musta been a drug dealer? why, cuz you're dope
    is a Pokemon Researcheris a Site Staff Alumnusis a Forum Moderator Alumnusis a Smogon IRC SOp Alumnusis a Tiering Contributor Alumnusis a Contributor Alumnusis a Smogon Media Contributor Alumnusis a Battle Server Moderator Alumnus

    Joined:
    Dec 21, 2007
    Messages:
    5,289
    '-' i meant that having a pool of 60-80 voters or so should be much more effective than a small pool of 20, because you could easily have 10 of those 20 people making the wrong desicion for a clearly broken pokemon ;-;
  7. Bologo

    Bologo Have fun with birds and bees.
    is a Contributor Alumnus

    Joined:
    May 7, 2006
    Messages:
    2,841
    Using the standard ladder for testing sounds like an okay idea, and like KD said, I really doubt we'd lose that many people. We already commonly go over our user limit on the server, and besides, testing is usually an exciting thing.

    However, if we do this, we have to reset ratings this time. In the Skymin test, there were a ton of people who didn't even participate in the test (ie. they didn't play for several weeks), and they still qualified due to their high rating the month before. This is probably the biggest drawback to testing on the standard ladder, because I'm not sure how many people want their rating to be resetted without wanting to test.

    We could try that older suggestion of registering a new shoddy nick in some thread so that people would have a fresh rating, and they would be people that want to test. That could be our alternate to resetting ratings if people are into that.

    Also, remembering the Wobb test on the Official server, we need to give people a lot of warning that the tests will occur on the standard ladder. However, I'm sure that's a given.
  8. reachzero

    reachzero the pastor of disaster
    is a Super Moderator Alumnusis a CAP Contributor Alumnusis a Tiering Contributor Alumnusis a Battle Server Moderator Alumnus

    Joined:
    Oct 18, 2008
    Messages:
    1,524
    I am in favor of Great Sage's suggested compromise. If the Suspect turns out to be nearly unbeatable (that is to say, Uber on merit), it would quickly become very disruptive and upset the metagame (arguably the way Abomasnow/Walrein is in UU). As a result, the metagame will become much less enjoyable, and their would be no metagame to fall back on for those who want to opt out. However, if Suspect is the "default" ladder, no one will be able to plead ignorance, and I see participation rising significantly.
  9. Jumpman16

    Jumpman16 np: Michael Jackson - "Mon in the Mirror" (DW mix)
    is a Smogon IRC SOPis a Site Staff Alumnusis a Team Rater Alumnusis a Battle Server Admin Alumnusis a Researcher Alumnusis a Tiering Contributor Alumnusis a Contributor Alumnusis an Administrator Alumnus

    Joined:
    Dec 19, 2004
    Messages:
    4,763
    As I have shared privately with Aeo, Doug and Mekk, I feel that "participation" during the actual past Suspect Test may not have been as low as we are stating. When we keep in mind that we are definitely going to go through with the idea of "Suspect EXP" that I originally raised to Aeo and Doug and that Doug has the tools to easily calculate now, we are reminded that we stand to gain many qualified "participants" that just missed out on out the 1655/65 requirements. I am prepared to make people like Bolts and ipl actually vote regardless of whatever numbers they can put up if they don't use the Suspect throughout the whole test, and besides that, the numbers we have clearly indicate that these two were the only ones in the pool of 24 qualifiers who didn't use the Suspect. We stand to gain many, many more, and my point is that the number of people we gain is entirely up to us.

    Doug is going to send me, Aeo and Mekk some numbers on who would have passed on the "unpublished requirement" of 1600/70. I will reiterate that those numbers were totally pulled out of my ass. However, the point of the actual numbers Doug is going to get us can serve as the benchmark for the number of voters we feel will be enough for each Suspect Test round. We have not yet decided how much Latias Experience is "enough", but we also have those numbers (Suspect Used and Suspect Sent) and can decide what's high enough.

    I do feel that totally doing away with the Suspect Ladder is, given our goal to increase "participation", not only unnecessary, but potentially very harmful. We are going to be testing some very wacky things in the coming months after we're done with Lati@s and Manaphy, including Evasion, OHKOs and possibly Species Clause. We don't want to shy away from testing these things, but it will definitely cease to be "standard pokemon" as we even remotely know it, and if the only options every battler has are to "play with Evasion or play unrated (or UU, ubers etc) or play somewhere else", it's going to be really shitty. And I don't think making a secondary ladder that will be just like the old Standard for people to play rated battles on is necessarily a good solution, given that we may very well have solved our problem already by deciding on the correct Suspect EXP requirements.
  10. Kevin Garrett

    Kevin Garrett is a competitor
    is a Tutor Alumnusis a Tournament Director Alumnusis a Site Staff Alumnusis an Artist Alumnusis a Super Moderator Alumnusis a Tiering Contributor Alumnusis a Smogon Media Contributor Alumnusis a Battle Server Moderator Alumnusis the Smogon Tour Season 12 Championis a SPL Winner

    Joined:
    Jan 5, 2008
    Messages:
    3,298
    I support the notion that fewer voters isn't necessarily a bad thing. Since Jump, Aeolus, and Doug can now determine how many voters they want, I don't think the varying activity of the suspect ladder will be a major problem. Since the best way to do it on standard would be to clear everyone's rating, it wouldn't appeal to countless people. And the Smogon Tour also represents that most recent tiers on each of the standard ladders, so it would interfere with that as well.
  11. Aeroblacktyl

    Aeroblacktyl The pizza doesn't scream in the oven! LOL!
    is a Tutor Alumnusis a Tournament Director Alumnusis a Site Staff Alumnusis a Team Rater Alumnusis a Forum Moderator Alumnusis a Tiering Contributor Alumnusis a Contributor Alumnusis a Battle Server Moderator Alumnusis a Past SPL + WCoP Winner

    Joined:
    Feb 19, 2005
    Messages:
    8,335
    random question: when will we actaully get to play 'real ou' without ever having to worry about team/metagame changes etc
  12. X-Act

    X-Act np: Biffy Clyro - Shock Shock
    is a Site Staff Alumnusis a Programmer Alumnusis a Smogon IRC SOp Alumnusis a Researcher Alumnusis a CAP Contributor Alumnusis a Tiering Contributor Alumnusis a Contributor Alumnusis an Administrator Alumnus

    Joined:
    Feb 17, 2006
    Messages:
    4,675
    I don't care where the testing is done, really. But I'll warn you that if the testing is done in the Standard ladder, the 1655/65 requirements should be thrown out of the window.

    Those requirements assume that everyone starts from a 1500/350 rating/deviation. If someone on the Standard ladder is already at a rating of, say, 1950/30, he would need to play just one or two games, lose both of them, and qualify to vote. This is what happened with the Shaymin-S vote, and that is the main reason why it was utterly flawed.

    So if you choose to do it on the Standard ladder, we'll need other criteria to get the people with voting rights. I'll gladly try to help with these criteria if the Standard ladder is chosen, though.
  13. DougJustDoug

    DougJustDoug Knows the great enthusiasms
    is a member of the Site Staffis an Artistis a Programmeris a CAP Contributoris an Administratoris a Battle Server Admin Alumnusis a Smogon IRC SOp Alumnusis a Tiering Contributor Alumnus
    CAP Leader

    Joined:
    Jun 26, 2007
    Messages:
    3,061
    If we test on the Standard ladder, how do we answer the user that says:

    "I came here to play the normal pokemon metagame. And now I'm forced to face all these uber pokemon. I don't want to be part of this test. How do I get out of this test?"​

    I don't think we should force people to play a metagame that we fully acknowledge may be broken. The word "suspect" means "We suspect it may not be broken". But the reverse is also true, we also "suspect it may be broken". If we know that the suspect metagame may be broken, I don't think we should be putting that metagame front-and-center at Smogon.

    It would be like the FDA testing out new food additives by putting them in the foods in every grocery store in the world, and then seeing if the general populace starts puking their guts out. It's one thing if willing test subjects sign up to test experimental food and drugs. Those people know what they are getting into, and they do it willingly (probably for pay). But to subject the general public to something that MAY be harmful -- that's just not right.

    I know we are not talking about people's lives here. This IS just pokemon, so maybe the analogy is a bit strained -- but I think the general point still holds true. We should not make everyone participate in an experiment that may be "harmful".

    I'm not too worried about losing people to other servers. I think the general Smogon populace will probably stay here, even if we subject them to all sorts of metagame abuse. But, it just doesn't seem "fair" to me.

    We need to encourage participation, not force it. I think there are plenty of people interested in experimenting with the metagame. We need to figure out a way to allow us to conduct the experiment, and collect meaningful feedback on the results -- without making everyone in the community become a test subject, willing or not.
  14. Aeolus

    Aeolus Bag
    is a Tutor Alumnusis a Tournament Director Alumnusis a Site Staff Alumnusis a Battle Server Admin Alumnusis a Smogon IRC SOp Alumnusis a Tiering Contributor Alumnusis a Contributor Alumnusis an Administrator Alumnus

    Joined:
    Sep 12, 2005
    Messages:
    3,639
    I've altered my position some since I last spoke with Jump... but I have to say that I now agree that testing must be done on the suspect ladder rather than the standard one. I do like the idea of making the suspect ladder the default just to boost awareness of the test... but we'll talk about that later. We've also come up with a small incentive for people who choose to actively participate... but you'll learn more about that later. Thanks for all the input guys.

Users Viewing Thread (Users: 0, Guests: 0)