If it were only that easy...maybe it can be, though. There's no simple way to go from 493 pokémon (or, technically, 492, which is the subject of one of the topics I will get to) to perfectly created and balanced tiers that allow for perfectly played competitive metagames. There is, though, an objective and regimented way of going about creating the rules fairly. With our own Smogon server on Shoddy Battle, we have our long-awaited power over key influences on competitive battle, but with this power comes a world of responsibility.
We can pick it up from scratch. It has been said that we need, in no particular order, to examine:
Wobbuffet
Lati@s
Deoxys-S
Mew
Manaphy
Garchomp
Darkrai
Arceus
Event Moves
Legendary IV Clause
Evasion Clause
Species Clause
UU Tiers
That's a lot. Some feel that, to move any pokemon from any tier fairly, its performance in the metagame in which it is being tested must be analyzed in a tournament. Some would champion its performance on the ladder over a fair period of time, say 2-3 months. Blue Kirby has suggested a long-term "tour" featuring any one pokemon that needs to be tested. All are good ideas, but first:
1. We must decide on the order of importance of the issues concerning competitive pokemon.
Divide and conquer. We can decide in here what the most important issues are currently, by ranking them in order, and then taking the ones with the lowest "points" and dealing with them in that order (1=Wobbuffet, 2=Garchomp, 3=Manaphy...you get the idea). There is no question, though, that we should be focusing our efforts on the most important issues first and foremost.
The next step refers to the methodology above.
2. We need to come to a decision on how exactly to conduct an analysis on a given issue.
Sure, it may be the case that a test of Garchomp and Lati@s would all benefit from a "Special Smogon Tour", but this may not be the way UU Tiers or Species Clause should be tested. It may also be the case that it would greatly benefit the community to test more than one suspect at a time. Regardless, we need to pin down what would be the most effective way of tackling the most important issue or issues at any time before we can go forward. And, of course, issues like Arceus and the Legendary IV clause can't be "tested" the same way, but rather discussed in Policy Review/Stark Mountain properly.
After this, it gets a little more difficult, but nothing we can't collectively handle.
3. We must fairly collect many statistical data from the battles conducted concerning the issue in question.
This includes both usage statistics (including pokemon, move and item) and battle logs as far as I'm concerned. Without hard data, it is hard to separate a real analysis from mere theorymon or conjecture. Our server will be capable of gathering usage statistics just like Colin's is, and in the event a tourney or "Special Smogon Tour" is being run, we can make it very plain that battlers are to save logs of all their battles so they can be properly analyzed.
4. We must analyze these data to the best of our ability to determine our collective stance on the issue in question.
This is the hard part, and where "we" really comes into play. It will not do our community any good for one or two people to be analyzing a handful of logs on how broken Wobbuffet is without any direction. To do this fairly we would need to have at least a dozen smart, trusted, battle-tested people analyzing battle logs. There is very much power in numbers in this regard, as I imagine a system where three people analyze the same log and give one of the following three assessments:
[pokemon] reasonably dominated this battle.
[pokemon] did not dominate this battle.
[pokemon's] impact on this battle was inconclusive.
That's just my idea, and you are welcome to propose changes or your own altogether. I do feel that analyzing battle logs is crucial, and that if we have 3-4 people sounding off on the same log we can get a more fair assessment on the pokemon's impact. We can make a sub-forum for these logs, and have people independently assess them so as not to sway the opinions of other log analyzers.
I also feel that we all need to realize how big an undertaking this is going to be, and that there are no two ways about analyzing an issue if we want to do it fairly. I actually just had a faint pang of excitement right now typing this, thinking about how Garchomp could thus fairly be sent off to ubers, because even though my heart's told me forever that I wouldn't want to see it banned, I cannot argue and in fact embrace the results of a fair and thorough analysis no matter the results, and no matter the issue. That takes us to the final step:
5. We need to accept the outcome of our analysis, implement the necessary policy change (if there is one) on the Smogon Shoddy server, and let the community know the results of the analysis of the issue in question.
If a given issue is touched by enough people who adhere to Smogon's Philosophy, and given the effort it deserves, then the outcome is almost secondary.
That's it. Please post in this thread to what extent you agree with this five-step process, and anything you'd change, add or take out. This needs a lot of input and attention if our server, and, on a larger scale, our community is going to be a long-term success.
We can pick it up from scratch. It has been said that we need, in no particular order, to examine:
Wobbuffet
Lati@s
Deoxys-S
Mew
Manaphy
Garchomp
Darkrai
Arceus
Event Moves
Legendary IV Clause
Evasion Clause
Species Clause
UU Tiers
That's a lot. Some feel that, to move any pokemon from any tier fairly, its performance in the metagame in which it is being tested must be analyzed in a tournament. Some would champion its performance on the ladder over a fair period of time, say 2-3 months. Blue Kirby has suggested a long-term "tour" featuring any one pokemon that needs to be tested. All are good ideas, but first:
1. We must decide on the order of importance of the issues concerning competitive pokemon.
Divide and conquer. We can decide in here what the most important issues are currently, by ranking them in order, and then taking the ones with the lowest "points" and dealing with them in that order (1=Wobbuffet, 2=Garchomp, 3=Manaphy...you get the idea). There is no question, though, that we should be focusing our efforts on the most important issues first and foremost.
The next step refers to the methodology above.
2. We need to come to a decision on how exactly to conduct an analysis on a given issue.
Sure, it may be the case that a test of Garchomp and Lati@s would all benefit from a "Special Smogon Tour", but this may not be the way UU Tiers or Species Clause should be tested. It may also be the case that it would greatly benefit the community to test more than one suspect at a time. Regardless, we need to pin down what would be the most effective way of tackling the most important issue or issues at any time before we can go forward. And, of course, issues like Arceus and the Legendary IV clause can't be "tested" the same way, but rather discussed in Policy Review/Stark Mountain properly.
After this, it gets a little more difficult, but nothing we can't collectively handle.
3. We must fairly collect many statistical data from the battles conducted concerning the issue in question.
This includes both usage statistics (including pokemon, move and item) and battle logs as far as I'm concerned. Without hard data, it is hard to separate a real analysis from mere theorymon or conjecture. Our server will be capable of gathering usage statistics just like Colin's is, and in the event a tourney or "Special Smogon Tour" is being run, we can make it very plain that battlers are to save logs of all their battles so they can be properly analyzed.
4. We must analyze these data to the best of our ability to determine our collective stance on the issue in question.
This is the hard part, and where "we" really comes into play. It will not do our community any good for one or two people to be analyzing a handful of logs on how broken Wobbuffet is without any direction. To do this fairly we would need to have at least a dozen smart, trusted, battle-tested people analyzing battle logs. There is very much power in numbers in this regard, as I imagine a system where three people analyze the same log and give one of the following three assessments:
[pokemon] reasonably dominated this battle.
[pokemon] did not dominate this battle.
[pokemon's] impact on this battle was inconclusive.
That's just my idea, and you are welcome to propose changes or your own altogether. I do feel that analyzing battle logs is crucial, and that if we have 3-4 people sounding off on the same log we can get a more fair assessment on the pokemon's impact. We can make a sub-forum for these logs, and have people independently assess them so as not to sway the opinions of other log analyzers.
I also feel that we all need to realize how big an undertaking this is going to be, and that there are no two ways about analyzing an issue if we want to do it fairly. I actually just had a faint pang of excitement right now typing this, thinking about how Garchomp could thus fairly be sent off to ubers, because even though my heart's told me forever that I wouldn't want to see it banned, I cannot argue and in fact embrace the results of a fair and thorough analysis no matter the results, and no matter the issue. That takes us to the final step:
5. We need to accept the outcome of our analysis, implement the necessary policy change (if there is one) on the Smogon Shoddy server, and let the community know the results of the analysis of the issue in question.
If a given issue is touched by enough people who adhere to Smogon's Philosophy, and given the effort it deserves, then the outcome is almost secondary.
That's it. Please post in this thread to what extent you agree with this five-step process, and anything you'd change, add or take out. This needs a lot of input and attention if our server, and, on a larger scale, our community is going to be a long-term success.
