Using The Gen 4 UU Testing Process For Gen 5

[Heysup]

Just to be clear, I've never said that Smogon battlers don't aim to win in general, or that it's not a goal of the battlers. And yes, I've read Smogon's Philosophy... I'm just saying that there's no true incentive in an economic sense, which is what your original argument was using. I'm not even sure what you're arguing about right now.

I enjoy my Tiering Contributor badge and my check mark, if you want purely economic reasons why people want to win. I enjoy having my name on the leaderboard. You're acting like you need a monetary reason to play to win, when that's not the case. Economic reasons would suggest that you want rewards, some sort of tangible / intangible (depends how you look at badges and the leaderboard) reward, which Smogon does supply. It rewards its hard workers, and doesn't reward it's not hard workers. Look who has lots of badges (someone like Phil) and who doesn't if you want proof.

You also have to take into account how much people want to win as well. There are cases where I'd be using an alt and not caring whether I won or lost because I'm testing new things or just being totally stupid (Like running Pikachu in Ubers, SD Skarm, and other random sets). While I want to win, a lot of the time, I (and I'm quite sure many others) would be just playing to try new things, and winning is just something nice to have come out of it.

Whether or not you're using a team that one could define as "bad", you're still trying to win. In fact, read this part of the smogon Philosophy (this article has proven useful for this thread :D):

It should be noted, however, that the player who uses a lesser Pokémon and still attains victory consistently is rewarded

While many people enjoy using "lesser" Pokemon, the fact that they still try to win is nearly undeniable. You don't join a battle and try to do anything else but win the battle. You can test things, sure, but you still want your test to do well, no?

I'd once again delve deeper into this issue, but this point will be made moot by the fact that alts will not be a problem on PokeLab.

 

[TAY]

I think we're getting a bit off track here with the "play to win" argument when that is really a minor point. People will play to win whether the test is going on or not, and even if the test does discourage "playing to win" by some amount, the majority of players will still try pretty damn hard to win when it comes down to it.

Anyway, I think this is the main source of the disagreement:

You're not considering the fact that if the metagame can be stable in said 6 months, it will with the UU test. The banlist won't change if the metagame has no broken Pokemon.

Is there some reason that we wouldn't just end the UU process after it is stable two cycles in a row? I would be OK with that. We all know that the vast majority of Dream World abilities will not be justification for a ban, and if and when something particularly powerful comes up (possibly Shadow Tag Shandera, Speed Boost Blaziken) we can run the process again (and stop if nothing is banned; if something is banned we continue until there are no bans twice). I don't like the idea of not having a set ending point, but at least this way we have and end point.

(Another option would be to make it more difficult to ban with each cycle, e.g. increase the required majority by 10% each cycle)

I just don't see the point of spending countless man-hours of effort writing nominations and making threads and arguing when we know that the metagame is balanced anyway.

 

[Heysup]

Anyway, I think this is the main source of the disagreement:

Is there some reason that we wouldn't just end the UU process after it is stable two cycles in a row? I would be OK with that. We all know that the vast majority of Dream World abilities will not be justification for a ban, and if and when something particularly powerful comes up (possibly Shadow Tag Shandera, Speed Boost Blaziken) we can run the process again (and stop if nothing is banned; if something is banned we continue until there are no bans twice). I don't like the idea of not having a set ending point, but at least this way we have and end point.

I'd just like to point out that this has not happened yet in UU and we thus have no evidence of the best solution.

I do actually think you have a reasonable suggestion to add to the process. This is the only change I'm really accepting of because it simply has no occured yet in the UU test (which is because we keep getting Pokemon dropped down rather than the test taking a long time).

The difference between OU and UU is not the mindset of the playerbase. It is the fact that one tier is ever-changing and one is not, this is something that should have been brought up by my opposition at this point; it's the only thing that matters. With the UU tier never really remaining the same permanently, it doesn't make sense to end the test, even after 2 stable metagame nominations. OU on the other hand doesn't get drop ins. This is why it makes sense to alter the test slightly to fix the only difference.

I kind of wish this would have been brought up sooner, rather than people bringing up irrelevant speculation and baseless theories about the UU ladder.

 

[Seven Deadly Sins]

I'd be fine with that alteration, and indeed intended for something like that to be the norm. 2 rounds without suspects seems fine, because it takes into account metagame fluctuations and "discovering stuff". In tandem with that, it'd be a good idea to have a list of "dream world suspects", which are just specific Dream World abilities that we can assume will cause shake-ups, like Speed Boost Blaziken and Drought/Drizzle Ninetales/Politoed, and then if the test has "ended" by that time, we can give them their own specific test where they are automatically nominated as suspects and then voted on.

Still, it seems like this process is likely the best way to handle things, regardless of whether or not we have a starting banlist. The only difference that a starting banlist would affect is how long it takes to get to the "stable metagame", but not how the process itself functions.

 

[obi]

Would you mind posting the exact specifications of the Gen 4 UU testing process? It's gone through some changes, and I think it would be best if your exact proposal were laid out in this thread rather than having people going other places to find it. Are there any changes at all you'd like to make?

 

[Heysup]

The current UU process is:

1. Open ladder and set upper and lower requirements (ie 1800/45 upper and 1600/50 lower) and a deadline (ie 4 weeks, though the first round was 6).

2. After deadline passes, open a bold nomination thread. This allows people who make valid arguments nominate Pokemon.

3. Get a consensus (however you would like, either by %s or by collective agreement) from the nomination thread.

4. Reveal the potential voters (the people who made the upper and lower requirements by the deadline) and set the deadline for paragraphs for lower requirements. Upper requirements send in a couple sentences.

5. Moderators / Judges review paragraphs and decide who's voting on which suspect.

6. Open up threads to vote on each suspect.

7. Pokemon with a super majority voted BL are immediately banned. Pokemon with a 50% + 1 majority are left in for another round. If they are voted 50% + 1 again, they are banned the round after. Pokemon with less than 50% BL votes are returned to the UU tier.

8. Rinse and repeat. TAY's process would assume that we repeat this until there are two rounds of no suspects. The UU metagame commonly gets drop ins, so ending the test doesn't make sense.

I'm not sure if this is exactly what SDS proposed, but this is how UU works now (feel free to point out any mistakes).

 

[Seven Deadly Sins]

I think the one thing that we decided should be changed is the removal of paragraphs, and the institution of a "test end protocol" which would kick in when we had 2 consecutive rounds of No Suspects, but otherwise, that's the one. So, to cover the suggested changes...

1. Open ladder and set requirements and a deadline (preferably 4 weeks).

2. After deadline passes, open a bold nomination thread. This allows people who make valid arguments to nominate Pokemon.

3. Suspects chosen based on nomination submissions.

4. Reveal the voters, and open up threads to vote on each suspect.

5. Pokemon with a super majority voted BL are immediately banned. Pokemon with a 50% + 1 majority are left in for another round. If they are voted 50% + 1 again, they are banned the round after. Pokemon with less than 50% BL votes are returned to the UU tier.

6. Rinse and repeat. After 2 rounds of "No Suspects" the test is officially over.

7. If any Dream World Pokemon determined to be suspect enters the metagame after the end of the test, a special round is opened specifically for that Pokemon. Nominations do not occur for this test: the suspect is considered auto-nominated and the test would automatically progress from testing to voting. The test would last 1-2 rounds depending on whether or not it received a supermajority, and then after a decision is made regarding that Pokemon, the test ends again.

 

[Mario With Lasers]

Lol 3am, bear with me here.

7. If any Dream World Pokemon determined to be suspect enters the metagame after the end of the test, a special round is opened specifically for that Pokemon. Nominations do not occur for this test: the suspect is considered auto-nominated and the test would automatically progress from testing to voting. The test would last 1-2 rounds depending on whether or not it received a supermajority, and then after a decision is made regarding that Pokemon, the test ends again.

I'd say this is the only step I have a problem with. From what I gather, you're suggesting that we may deem a pokémon Suspect even before it's officially released. Isn't this exactly what happened with Shaymin-S? We had a month or two of theorymoning ("oh god Serene Grace Seed Flare/Air Slash"), which only got worse once Platinum came out and we discovered Skymin's base stats and that it learned Earth Power. You may say Skymin really deserved the boot in the end, or that it clearly was Suspect-worthy at least, but the problem is, setting a Suspect ladder for it merely two days after Platinum was released stirred up in every single player the idea that Skymin was "too good", just like Garchomp and the Latis etc etc. The difference was, we had played with Garchomp for two years and we knew the Latis since 2002, so we had some basis to vouch for them as Suspects. We had zero basis for Skymin, we didn't even wait till the hype had worn off to "test" it.

As we don't know yet if every single DW pokémon will be available since day one, 5th Gen OU testing will probably be like 4th Gen UU, where we may have long (or not) periods of "no Suspects" and suddenly, something drops down from the heavens. But can't we just treat them as new additions to the metagame, instead of auto-Suspecting them? What if they release Drizzle Politoed, Shadow Tag Shanderaa and Harvest Exeggutor at the same time? What if these additions actually make something else broken, such as changing the metagame in such a way MH Erufuun becomes elegible for the Support Characteristic?

I believe all DW pokémon, from Light Metal Metagross to Drizzle Politoed, should be implemented and treated as "normal pokémon"; they will be the coolest kids on the block already, there'll be no need for Smogon to shine any spotlights on them. I've never seriously played UU but I used to read the "np:" threads, and I don't recall having ever seen any OU-dropdown being auto-nominated. Hell, as far as I know, Dugtrio has never been nominated. For such a pokémon which was clearly bound to be broken in UU according to theorymon... it didn't live up to its hype, it seems. And Heracross appears to be shaping up to have the same fate. I've read a few posts and comments in the same vein about Shadow Tag Shanderaa too. So, instead of going the Shaymin-S route, why don't we go the Bullet Punch Scizor route with DW pokémon?


EDIT -- Heyusp just said what I wanted to in fewer words, damn

 

[Heysup]

7. If any Dream World Pokemon determined to be suspect enters the metagame after the end of the test, a special round is opened specifically for that Pokemon. Nominations do not occur for this test: the suspect is considered auto-nominated and the test would automatically progress from testing to voting. The test would last 1-2 rounds depending on whether or not it received a supermajority, and then after a decision is made regarding that Pokemon, the test ends again.

I don't see the need to completely ignore the best aspect of the process: nothing is automatically deemed a suspect. In my honest opinion I don't think we should stop testing until all of the dream world abilities are released (unless there's a break between the no suspect meta and DW abilities being released) and tested. If we do get the 2 required no suspect metagames, then I think as soon as Game Freak starts implementing the DW abilities, we should start up rounds of testing as if it were still the same testing process. We should absolutely not automatically name them suspects, that goes against the whole idea of this test in the first place.

 

[Firestorm]

Yeah, I'm with Heysup and Mario with Lasers on this. What would the purpose of bypassing the system in place be for DW Pokemon?

 

[Seven Deadly Sins]

There's a very, very important part there:

If any Dream World Pokemon determined to be suspect enters the metagame after the end of the test, a special round is opened specifically for that Pokemon.

Obviously if, say, Shadow Tag Shanderaa dropped down during the main test, it would just be absorbed by the metagame and handled normally. However, if we're going to actually end the test, we need to be able to handle the most egregious examples of significantly metagame-changing Pokemon such as Drought Ninetales, Drizzle Politoed, Speed Boost Blaziken, Shadow Tag (anything), etc.

It's either that or simply not end the test. We need to be able to account for Dream World's late entries, because they have the potential to cause problems if they show up once the test has ended.

This isn't bypassing anything. This is making sure that ending the test doesn't leave things hanging when potentially ridiculous Dream World stuff enters the metagame. I'm open to other suggestions, but we're looking at a metagame where we can have potentially insane things drop out of the heavens onto the metagame, and we need to be able to deal with them. The way I proposed is just one potential method.

 

[cim]

Why should the test end? Can someone tell me that? Maybe the vote at the end is just a formality to say "no suspects" once again, but I don't see what the harm in it is.

 

[Heysup]

Obviously if, say, Shadow Tag Shanderaa dropped down during the main test, it would just be absorbed by the metagame and handled normally. However, if we're going to actually end the test, we need to be able to handle the most egregious examples of significantly metagame-changing Pokemon such as Drought Ninetales, Drizzle Politoed, Speed Boost Blaziken, Shadow Tag (anything), etc.

We cannot just open the test for "specific" Dream Wold Pokemon. That's purely based on theory and speculation. As I see it we have two legitimate options to choose from without compromising the validity of the test:

1) We don't end the test until all Dream World Pokemon have been released, regardless of whether or not we get two "No Suspect" metagames in a row or not before that. (Yes I know you suggested this as an alternative)

2) We end the test after two "No Suspect" metagames and if there is a sufficient break between this metagame and when Dream World abilities are being released. After this, when Dream World abilities start to get released, we reopen the test until all Dream World abilities have been released and tested. This removes any speculation as to which Dream World Pokemon are actually going to be good and which aren't before we actually test it.

Obviously I've been advocating the first option since the assumption that people will tinker with tiers just because they can is incredibly flawed, but if it makes more people happy then there is no harm doing the second option.

 

[TAY]

I don't see the problem with re-opening the test for specific abilities. There is no need to treat all DW abilities uniformly. We aren't stupid; we (the policy makers) will know when the test needs to take place.

If you would like a more general policy, we could just hold a vote among policy review members for DW abilities 1 month (or more?) after they are released. The test would be re-opened if 75% of PR members think there should be a test. (Note that I am using arbitrary numbers here).

We don't currently know how DW abilities will be released, nor do we know whenther nintendo even plans to release them all (or even more than a hundred or so). So it seems silly to me that you are suggesting we commit to one of two options that assume a very specific pattern of release. This is why I like my suggestion regarding DW much better: it is successful no matter how nintendo releases DW mons.

If nintendo decides to tell us the release schedule (or pattern) of DW pokemon, then I could see us choosing one of those options. With what we know now, however, it just seems foolish.

 

[Heysup]

I don't see the problem with re-opening the test for specific abilities. There is no need to treat all DW abilities uniformly. We aren't stupid; we (the policy makers) will know when the test needs to take place.

I feel we should learn from our mistakes about Theorymonning suspects. Why risk making the same one again? We don't know anything. Especially when a previously NU/UU Pokemon gets a huge buff, we have no idea what we are dealing with. This isn't like the Uber banlist which we can justifiably say we know enough about. This is Pokemon that we have no idea how they will function and their effects on the metagame.

Policy Reviewers aren't stupid, but we cannot know enough about something that doesn't exist yet.

If you would like a more general policy, we could just hold a vote among policy review members for DW abilities 1 month (or more?) after they are released. The test would be re-opened if 75% of PR members think there should be a test. (Note that I am using arbitrary numbers here).

Maybe I'm the only one that thinks Smogon is already unnecessarily exclusive as it is. Not all policy reviewers are experienced / knowledgeable / current battlers either, we should always consider that.

I don't see the harm in giving a fair test to everything. This option simply offers no benefit and comes with the risk that the Policy Reviewers are wrong and painting a bad picture for the good, knowledgeable, and current battlers who don't get a say.

We don't currently know how DW abilities will be released, nor do we know whenther nintendo even plans to release them all (or even more than a hundred or so). So it seems silly to me that you are suggesting we commit to one of two options that assume a very specific pattern of release. This is why I like my suggestion regarding DW much better: it is successful no matter how nintendo releases DW mons.

If nintendo decides to tell us the release schedule (or pattern) of DW pokemon, then I could see us choosing one of those options. With what we know now, however, it just seems foolish.

How does the implementation affect any of the two options?

We know nintendo is implementing them at some point down the road. They can do this two ways: all at once or gradually. Either way, both of the options make perfect sense and don't depend at all on how they are implemented.

Obviously option one is just to not end the test, and it is the superior option in my opinion. This is obviously unaffected by how Dream World abilities being implemented.

Option two works perfectly fine for both ways as well. If they are all implemented at once, we open up a round. If they are implemented gradually (as it may seem I'm assuming), they will still all get a test.

I don't know where you got the idea that I'm assuming a specific way that Dream World abilities are going to be implemented. We obviously can't know until it happens, and both of my suggestions accommodate every possible scenario.

 

[Seven Deadly Sins]

Of course, this is coming back to the idea that "just because something is suspect doesn't mean it's going to be banned". The point is to have a list of "shit to watch out for", and assume that the voters will make the right choice. We're not trying to auto-ban them or anything, but it seems like the cleanest way to achieve a sense that testing is finite, while also accounting for the potential that Dream World might cause problems after the end of the test.

 

[cim]

In my mind, there is no "testing" at all with this process. Nothing is ever a Suspect until immediately proceeding the vote, so you shouldn't be "testing" things if you're voting for this. At least, that's how I see it. I don't see why stopping asking whether or not a Suspect is a Suspect will somehow fix a problem. If there's been two rounds of No Suspects, are people seriously going to suddenly become stupid and go "oh we should ban obvious non-uber!!!!"? I guess I just don't get it.

Also: I'd recommend we not require paragraphs, then voting. The paragraphs basically contain the vote. If we're going to have people write some stuff about Pokemon (which I don't agree with anymore as long as there's a high barrier of entry. It just makes the process take longer and can inadvertently and easily introduce bias), then they're already telling us how they're going to vote so separating the two is just silly.

 

[Heysup]

Of course, this is coming back to the idea that "just because something is suspect doesn't mean it's going to be banned". The point is to have a list of "shit to watch out for", and assume that the voters will make the right choice. We're not trying to auto-ban them or anything, but it seems like the cleanest way to achieve a sense that testing is finite, while also accounting for the potential that Dream World might cause problems after the end of the test.

I don't see the benefit of doing it when there are obvious drawbacks, most notably encouraging people to go out of there way to prepare for it, use it more than they normally would, directly affecting the validity of the test. Additionally, it's just inconsistent. If we are using this process, we should use this process. You said it best yourself, why change a process that's already proven to be great?

In my mind, there is no "testing" at all with this process. Nothing is ever a Suspect until immediately proceeding the vote, so you shouldn't be "testing" things if you're voting for this. At least, that's how I see it. I don't see why stopping asking whether or not a Suspect is a Suspect will somehow fix a problem. If there's been two rounds of No Suspects, are people seriously going to suddenly become stupid and go "oh we should ban obvious non-uber!!!!"? I guess I just don't get it.

I actually think this is the beauty of this method of suspect "Testing", you don't really feel like you're testing, but playing competitively and being able to point out and fix what's imbalanced/abuseable/unfair in the metagame.

A suspect, whether or not you want it to, means something is going to be isolated and tested. You are paying attention to that Pokemon more than the others. Encouraging people to pay extra attention to one Pokemon is stupid. If the Pokemon is a suspect, people will nominate it as such and this will happen naturally. We shouldn't fuck with the process for literally no reason whatsoever.

 

[Firestorm]

Of course, this is coming back to the idea that "just because something is suspect doesn't mean it's going to be banned". The point is to have a list of "shit to watch out for", and assume that the voters will make the right choice. We're not trying to auto-ban them or anything, but it seems like the cleanest way to achieve a sense that testing is finite, while also accounting for the potential that Dream World might cause problems after the end of the test.

I can understand the point is to have a list of "shit to watch out for" but I think what Heysup is saying, and I agree with, is that there shouldn't be such a list. That's just slanting public opinion of the suspect. It will rise as an issue naturally.

 

[Seven Deadly Sins]

I'd really like to point this bit out:

I actually think this is the beauty of this method of suspect "Testing", you don't really feel like you're testing, but playing competitively and being able to point out and fix what's imbalanced/abuseable/unfair in the metagame.

A suspect, whether or not you want it to, means something is going to be isolated and tested. You are paying attention to that Pokemon more than the others. Encouraging people to pay extra attention to one Pokemon is stupid. If the Pokemon is a suspect, people will nominate it as such and this will happen naturally. We shouldn't fuck with the process for literally no reason whatsoever.

Having played UU extensively pre-Cresselia (i never really could get into the cress metagame and eventually dropped out, but have been considering getting back into it), I'd like to bring major attention to the first paragraph here. The "test" really doesn't feel like a test at all. The whole point of it is to create the most effective team and play to win, and then once the end of the round comes around and nominations open up, you have the necessary experience required to evaluate the metagame and determine whether or not in your experience, a Pokemon is not conducive to an optimal competitive metagame. Nothing is thrust upon the voters and said "here, take a look at this, we think it's broken, we want you to confirm/deny". The most competitive players "run the show", so to speak, and who better than the most competitive portion of the playerbase to evaluate what makes the game more or less competitive?

I can understand the point is to have a list of "shit to watch out for" but I think what Heysup is saying, and I agree with, is that there shouldn't be such a list. That's just slanting public opinion of the suspect. It will rise as an issue naturally.

Right, I agree. Looking back, my addition is a bit heavy-handed, and actually goes completely against the idea of the UU testing protocol, and for that I apologize and withdraw my submission. That said, potentially troublesome Dream World abilities will have to be addressed. However, I think that's the kind of thing that we're going to have to take a harder look at once we know *how* the Dream World abilities are being released- Whether or not they may be random or predetermined, how often they're released, and how many are released at a time. Unfortunately, these aren't questions that we can answer right now.

I think more research and information is required before we address the issue of Dream World in the testing protocol, but we've got a long time before that becomes relevant, as Dream World entries only become an issue once the test is "over", which is guaranteed to take at least a couple months. I think we should put the Dream World Abilities issue on the back burner and save this thread for a consensus on the overall outline.

To restate, with my clunky Dream World bit edited out...

1. Open ladder and set requirements and a deadline (preferably 4 weeks).

2. After deadline passes, open a bold nomination thread. This allows people who make valid arguments to nominate Pokemon.

3. Suspects chosen based on nomination submissions.

4. Reveal the voters, and open up threads to vote on each suspect.

5. Pokemon with a super majority voted BL are immediately banned. Pokemon with a 50% + 1 majority are left in for another round. If they are voted 50% + 1 again, they are banned the round after. Pokemon with less than 50% BL votes are returned to the UU tier.

6. Rinse and repeat. After 2 rounds of "No Suspects" the test is officially over.

This is essentially the current UU testing protocol, except with the idea of "Upper and Lower requirements" removed, since paragraphs have always been one of the sticking points of any test, and mainly just serve to slow things down and make them more subjective. We'll come back to Dream World once more information is available, but until the end of the test, they'd just be treated the same way any other new entry to the metagame is; that is to say, no special treatment other than "hooray this is now available for use, go nuts!"