Basing UU and NU off usage stats

[coyotte508]

Currently the tiering criteria for defining UU is pokemon under 3.41% usage in OU being put in UU.

If the same criteria as DP is kept for 5th gen, I'd like to address the problem of Pokemon being around that 3.41% limit.

The problem I see is pokemon easily moving in or out of UU if their usage vary a little in OU. Like Machamp falling in UU for one month or suicune leaving UU for one month too (those are just examples, in 4th gen you could take heracross falling into UU or some other pokemon moving out of UU). With the increased variety of pokemon it wouldn't be surprising if there were more pokemon around that limit now.

I see that as a problem because it makes the UU and NU tiers a bit unstable for online players or NDS players, especially if some pokemon are really close to the 'limit'. Of course the metagame evolves and there are bound to be changes to those tiers, I'm not denying that.

One possible solution to solve that 'flickering' problem could be introducing different boundaries for moving out of UU/entering UU. You could say that, for example, a pokemon will move from UU to OU if its usage in OU rises above 3.50% and move from OU to UU if its usage falls under 3.32 %. That way trends will still be followed if they are consistent, but there wouldn't be pokemon moved for a slight one-month change.

A problem that could arise would be initial tiering mattering on definite tiering. For example, what if a pokemon, at the moment of the initial tiering, has 3.42% usage making it OU but in the following months has usage between 3.35% and 3.40%? This pokemon will stay OU but if UU had been made one month later it would be UU... One could say that the 3.41% limit is arbitrary from the start, and that a very slight deviation for a few pokemon is acceptable. However, that kind of particular case can be taken in account, by saying that another way of an OU pokemon becoming UU, other than breaking the 3.32% limit, is being under 3.41% usage for at least three successive months. That'll "fix" the criteria.

Same could be applied to NU.

That was just an example solution, to prevent an odd and situational tier change, which would for example ask a lot of work for nintendo DS players or contributors that would be for nothing if the tier change is cancelled the next month. Of course, everything could change if a new tiering formula for 5th gen was created.

Anyway I'd like to hear your thoughts, especially from 4th gen UU and NU players, and I think now's the best time to set the policy for 5th gen UU and NU tiers, while they aren't created.

 

[cim]

The solution we've used is to just adjust UU and NU every 3 months. Flickering happens, but it keeps the metagame pretty fresh and I think as a whole most people don't mind it. I don't really think solutions to the problem do enough good to justify the complications and stuff.

 

[Mr.E]

I don't think the minor tier changes every few months is all that big a deal. Changing team members is trivial on a simulator, of course, and the Wi-Fi crowd is already jumping through enough other stupid hoops (capturing, EV training, etc. if not "cheating" them) that 0-2 pokémon shifting between the tiers every few months should be pretty low on their list of annoyances. I don't think it's too much to ask of the competitive crowd to pay attention to the occasional tiering shift any more than the OU crowd would pay minor attention to the Suspect Tests and notice when Garchomp/Salamence got banned.

NU is still as much a competitive tier as BL, though... Nobody gives a shit about Vespiquen and Whiscash. :justin2: Sorry doods.

 

[shrang]

Just on the topic of this, since we have about 100 more Pokemon being introduced into the game, it might be about time that we defined one or two tiers below NU (Keep the names simple). NU is big enough of a clusterfuck already, wait until we add another 100 Pokemon into it.

 

[Chou Toshio]

The thing is that the UU system sorts itself out with time. In 5th Gen, there will be a lot more time (since we only organized UU somewhere around the middle of the generation in Gen 4).

Given time, Pokemon that "flicker" (like roserade) would have a pre-set UU/BL status upon a second or third "fall" to UU.

 

[Slipperjeans]

I agree with the different boundaries suggested by coyotte.

I also agree with what shrang said, about having a tier below NU.
4th gen NU is competitive. Just because Smogon doesn't support it doesn't mean it isn't competitive. NU has the biggest number of neglected Pokémon of all tiers and it deserves to be cut into another tier.

 

[cim]

All the second cut would do is clean up the presentation of "the most popular pokemon" and "pokemon not popular in NU but still legal in it since no one's honestly going to play this metagame". Nice but not vital to start out with...

 

[LonelyNess]

Just something to note - this conversation has been had many times and there was a consensus among people who play the UU tier with great frequency (such as myself) that the fluctuation of the tier is one of the draws of the metagame, rather than being something that people dislike.

Of course if this opinion has changed since the last time it was brought up (Tangerine always wanted to change UU to a set list that didn't fluctuate), then please disregard me.

 

[uragg]

I would agree with LN on that point. I haven't played UU for as long as some others here, but in my experiences from the Yanmega period onward, the suspects and Pokemon moving up and down during each tiering shift were pretty much what made the tier fun and exciting to play. Look at the most recent metagame, the one we had before Heracross dropped. We had two periods of utter stability with no real suspects, and people complained more about boredom and staleness than they appreciated the stablility and balance of the metagame.

As for the possible additional tiers below NU, CIM makes a good point in that we should try to resolve the issue of how we tier UU from BL/OU before we go into that.

 

[Mizuno]

I think the main concern may be the beginning of the metagame, where OU is not truly established and Pokemon fluctuating between tiers may be very high. We didn't have a UU Suspect Testing process in the beginning of Gen 4, but now we do and too much 'flickering' could delay or have an effect on suspect testing (and thus, delay archiving a 'stable' metagame).

I like the system coyotte508 suggests. Another method that could be used (by itself or in combination with coyotte's bethod) is that instead of using the usage statistics for just one month to decide the tiering of a Pokemon, the average usage of that Pokemon in all previous months since the last tier change should be used. For example, Heracross would drop from OU to UU if its average usage the last 3 months was bellow the OU cutoff point. I think this helps against 'flickering' and should allow for Pokemon to move between tiers as the OU metagame stabilizes.

 

[Seven Deadly Sins]

Just something to note - this conversation has been had many times and there was a consensus among people who play the UU tier with great frequency (such as myself) that the fluctuation of the tier is one of the draws of the metagame, rather than being something that people dislike.

Of course if this opinion has changed since the last time it was brought up (Tangerine always wanted to change UU to a set list that didn't fluctuate), then please disregard me.

Seconding this. As a former avid UU player, one of the big draws was that basically, you had a constantly changing metagame where you always had to look out for the possibility of new threats entering the metagame, especially as trends in OU shifted, and I'd be very disappointed if this changed, especially considering the way that Gen 5 OU might be shaping up.

 

[Erazor]

I have to agree with LN and SDS here, a constantly changing metagame injects freshness into the tier, and is more fun to play in. A fluctuating Pokemon will have a set 'status' anyway, after the first time.

 

[Heysup]

I'm sure many of the other UU players can agree that we honestly don't mind the fluctuation. Of course, it can get out of hand (ie Rhyperior, Alakazam, Gallade, Raikou, Froslass, Umbreon at once during one metagame), however if it has somewhat reasonable fluctuation I believe it's actually ideal. That's often the reason people end up playing UU, because it's never stale or boring like OU often gets.

I think we would simply need to readjust the tiering % to compensate for the extra Pokemon and essentially play it by ear.

 

[eric the espeon]

I like the system coyotte508 suggests. Another method that could be used (by itself or in combination with coyotte's bethod) is that instead of using the usage statistics for just one month to decide the tiering of a Pokemon, the average usage of that Pokemon in all previous months since the last tier change should be used. For example, Heracross would drop from OU to UU if its average usage the last 3 months was bellow the OU cutoff point. I think this helps against 'flickering' and should allow for Pokemon to move between tiers as the OU metagame stabilizes.

We currently use the last four months with much more weight given to the most recent ones (I think the fourth month may actually have zero weight currently).

Anyway, while I take the point that change is good for the UU metagame, large/fast changes seem to have been ruled unwanted before (which is why we had a tri-monthly then a quarterly update to the UU list). If we decide to implement an anti-flicking dual cutoff we would be able to update the tiers to reflect the current state of play much more often without destabilizing UU more than the current "no anti flicking/very rare updates" system. I have always been a fan of keeping the UU list up to date, and coyo's proposition (or a variant with slightly less arbitrary numbers) should be able to counter the main problem with frequent updates (actually running the number is pretty easy, even with weights for months).

 

[jrrrrrrr]

Those concerns were addressed in this thread. Basically, we want the tier to be fluctuating because it serves as a better safeguard against things becoming broken. That's why we added the clause to nominate BL pokemon to come down into UU. Also, since UU typically draws fewer players than OU, we liked it because it kept the tier fresh. This isn't a terrible idea, I just think the current system is better suited for what we want.

 

[Wichu]

Just on the topic of this, since we have about 100 more Pokemon being introduced into the game, it might be about time that we defined one or two tiers below NU (Keep the names simple). NU is big enough of a clusterfuck already, wait until we add another 100 Pokemon into it.

I'd place an extra tier in between OU and UU (MU; mid-used), rather than below NU (what would you call it? Negatively-used?). Of course, it's just a semantic difference.

But I don't think we need to complicate matters with an extra tier. I'd rather just expand the tiers we already have; for example, to roughly 75 Pokémon per tier.

Anyway, on the main topic, I like that the UU meta was changing every so often. Although I only started playing towards the end of 4th gen (some time after Milotic became UU, I think), I enjoyed the shakeup that the addition of Heracross caused (whether or not it was broken).

However, I like Mizuno's suggestion of taking the average usage since the last tier update. This should prevent annoying 'flickering', while still allowing the metagame to change.

 

[Ice-eyes]

I disagree with Wichu; if anything, the tiers are too big, and the bottom of them clogged by stuff no-one ever uses, stuff that you're genuinely surprised to see in a battle; I'm talking about things such as Umbreon and Electivire in OU or the likes of Nidoking in UU. I would actually prefer tiers of 40-45 pokemon rather than about 50; it's not as if it will consign more pokemon to the scrapheap, because stuff in the bottom 5 of OU is almost never seen anyway (particularly in the upper echelons of the ladder).

I do agree that a constantly shifting UU provides variety and stops the metagame from becomingstale; I don't think we should try to make all metagames extremely similar, so there can be something for everyone. As it stands, Ubers, OU, UU and Little Cup are all extremely different and I for one would prefer it if it stayed that way.

 

[coyotte508]

Just something to note - this conversation has been had many times and there was a consensus among people who play the UU tier with great frequency (such as myself) that the fluctuation of the tier is one of the draws of the metagame, rather than being something that people dislike.

Of course if this opinion has changed since the last time it was brought up (Tangerine always wanted to change UU to a set list that didn't fluctuate), then please disregard me.

I would agree with LN on that point. I haven't played UU for as long as some others here, but in my experiences from the Yanmega period onward, the suspects and Pokemon moving up and down during each tiering shift were pretty much what made the tier fun and exciting to play. Look at the most recent metagame, the one we had before Heracross dropped. We had two periods of utter stability with no real suspects, and people complained more about boredom and staleness than they appreciated the stablility and balance of the metagame.

As for the possible additional tiers below NU, CIM makes a good point in that we should try to resolve the issue of how we tier UU from BL/OU before we go into that.

Seconding this. As a former avid UU player, one of the big draws was that basically, you had a constantly changing metagame where you always had to look out for the possibility of new threats entering the metagame, especially as trends in OU shifted, and I'd be very disappointed if this changed, especially considering the way that Gen 5 OU might be shaping up.

I have to agree with LN and SDS here, a constantly changing metagame injects freshness into the tier, and is more fun to play in. A fluctuating Pokemon will have a set 'status' anyway, after the first time.

I'm sure many of the other UU players can agree that we honestly don't mind the fluctuation. Of course, it can get out of hand (ie Rhyperior, Alakazam, Gallade, Raikou, Froslass, Umbreon at once during one metagame), however if it has somewhat reasonable fluctuation I believe it's actually ideal. That's often the reason people end up playing UU, because it's never stale or boring like OU often gets.

I think we would simply need to readjust the tiering % to compensate for the extra Pokemon and essentially play it by ear.

Those concerns were addressed in this thread. Basically, we want the tier to be fluctuating because it serves as a better safeguard against things becoming broken. That's why we added the clause to nominate BL pokemon to come down into UU. Also, since UU typically draws fewer players than OU, we liked it because it kept the tier fresh. This isn't a terrible idea, I just think the current system is better suited for what we want.

Let's get things straight. The solution that I proposed doesn't remove freshness. The decision adopted for UU in 4th gen was to adjust tiers every three months, while with the solution in the OP tiers would be updated every month. If anything, the metagame you played is staler as it updated only every three months.

Basically what it does is:

• Prevent flickering
• Pokemon crossing the limit slowly will change tiers in 3 months
• Pokemon having a significant shift in usage will change tiers in 1 month

I never wanted to reduce fluctuation, just flickering.

The only thing undefined is the antiflickering number, which I set arbitrarily to 0.09%. A too low antiflickering number will make tiers change too quickly, a too big antiflickering number will make all pokemon take three months to cross a tier, which is more or less the current system.

I'm not saying this is the only way to go (you could sum usage/limit differences over months and use more mathematical concepts) but it's one possible change to make. The only risk is having the antiflickering number set too low, which could result in slightly more uncalled tier changes than with the current system and make the tier 'less stale'. But from what I read from your posts, this shouldn't be a concern.

Of course, if you don't want the metagame to update more often than every three months, then forget it, but this is not the feeling I got your posts =/

The solution we've used is to just adjust UU and NU every 3 months. Flickering happens, but it keeps the metagame pretty fresh and I think as a whole most people don't mind it. I don't really think solutions to the problem do enough good to justify the complications and stuff.

Fair enough.

 

[Seven Deadly Sins]

If anything, I'd like to see the UU cutoff raised. Those Pokemon that hover around #40 are basically irrelevant in OU, and they spend so much time shuffling around the UU cutoff that it seems odd to keep them out. I've always thought that OU was too big.

 

[coyotte508]

About that, where does the 3.41% number come from?

http://www.smogon.com/articles/tiers

It's said it should be one pokemon out of 20 teams, which would be 5%. Why and where is the 176.14073324 number defined? (couldn't find it through X-Act's posts)

Edit: Found it (thanks Jibaku)

http://www.smogon.com/forums/showthread.php?t=37925

Basically what's discussed is the prediction function. But

The usage statistics do not tell the probability of a Pokemon appearing on a team. If you sweep a team with just Garchomp, only Garchomp is counted toward the usage statistics. If I use all six of my Pokemon, all six are included.

That's probably the reason why 3.41% is used and not 5%.

Because, from the introduction to tiers page:

After the value of P is calculated for every Pokemon, these are summed up again. This summation S should result in a number that is extremely close to 1. Finally, the Pokemon whose P value is greater than or equal to S ÷ 176.14073324 make the OU tier. This number is the usage that Pokemon require in order to be present in at least one out of every 20 Standard metagame teams on average.

It probably accounted for the fact that Shoddy Battle logs sometimes didn't show all the pokemon and statistics had incomplete info. So it took in account the lack of info in statistics to say that what displayed as 3.41% in the statistics was probably 5% usage in reality.

I don't know if the statistics were fixed since then, but that must have stayed.

Anyway, there is no reason to use the 3.41% number anymore. Because if you're using the statistics of a PO server, what's displayed as 5% will actually be what's in 1/20 of the teams.

This will reduce the numbers of pokemon in OU (as an example, the main PO server would have 38 pokemon in OU). Some are still close to the threshold (5.11%, 5.09%, 5.06%, 4.97%) so the antiflickering might still do good. The main point though is that there is no reason at all to use the 3.41% number, and that 5% should be used, unless I'm horribly wrong. (But then please tell me where it comes from)

 

[JabbaTheGriffin]

I'm going to admit i skimmed this topic and i saw in the #dreamworld topic that there was something about raising the bar to 5%.

I definitely agree with any higher cutoff than what we have now and definitely have thought since day one pretty much that the bar was far too low.

I'm pretty sure it's not arbitrary it's pretty much supposed to mean that you should see that mon on one in every 20 teams. I've always been a fan of lowering the n in this scenario to 10 teams. Not sure what percentage that number would turn out to be on average, but anything higher than what we use now I will support.

 

[capefeather]

Well, that in itself is arbitrary, which is the point. There's no magical meaning behind "one in every 20 teams" or "one in every ten teams". I remember a thread where Doug and X-Act talked about implementing a 50th-percentile sort of cutoff. ("Overused" consists of the smallest collection of Pokémon that are collectively on at least half of all teams.) I just thought that this should be brought up again.

 

[coyotte508]

I'm going to admit i skimmed this topic and i saw in the #dreamworld topic that there was something about raising the bar to 5%.

I definitely agree with any higher cutoff than what we have now and definitely have thought since day one pretty much that the bar was far too low.

I'm pretty sure it's not arbitrary it's pretty much supposed to mean that you should see that mon on one in every 20 teams. I've always been a fan of lowering the n in this scenario to 10 teams. Not sure what percentage that number would turn out to be on average, but anything higher than what we use now I will support.

5% is one in every 20 team, one in every ten teams would be 10%. I don't really know why 3.41% was chosen in the first place but it's probably an issue with the way stats were collected (see my post above).

Well, that in itself is arbitrary, which is the point. There's no magical meaning behind "one in every 20 teams" or "one in every ten teams". I remember a thread where Doug and X-Act talked about implementing a 50th-percentile sort of cutoff. ("Overused" consists of the smallest collection of Pokémon that are collectively on at least half of all teams.) I just thought that this should be brought up again.

Funnily enough, the stats of the current PO server give the exact same tiers for that method and the 5% cutoff. (Dream World stats, summing until getting 300%)

That method is interesting and consistent, but I'm not sure it's the best method. It could make several pokemon at 1% or 2% OU, in theory, if no assumptions are made on the tiers. And having a pokemon that's only in one out of 50 teams OU wouldn't be great. Though in practice there may be no problem, it's just the theorical side that disturbs me.

In my opinion, declaring something OverUsed if they're used in at least one out of 20 teams is "fair enough". That's intuitive and not-so-arbitray, if you take the definition of what's OverUsed it means pokemon used a lot, and pokemon used in more than one out of 20 teams sound fair enough.

 

[Seven Deadly Sins]

I agree with the 5% cutoff, it's logical and makes sense to me. Better than "3.41%", I suppose, even if it does have a logical backing.

 

[coyotte508]

If a pokemon needs to be used in one of every 10 teams to be OU, there would be 10 pokemon OU, by the current stats. If it's in one of every 10 battles, there would be even less OU.

3.41% means you see a pokemon in one of every 29 battles, which *would* correspond to one in every 20 teams (which is the goal announced by http://www.smogon.com/articles/tiers). Though by PO's stats 3.41% means one in every 29 teams, and 5% one in every 20 teams.

I can't understand why PO's stats and Shoddy's stats were so similar if PO features whole teams and Shoddy only the pokemon sent out in battles. It would mean that nearly all pokemon were always shown in Shoddy's battles making the two systems similar, that the statistics collection mechanism changed since its creation or that the number of battles is calculated by the sum of all the pokemon seen, divided by 6. You'd need to ask Super or DougJustDoug to have the answer.

But to the point, using current stats available, the number of OU pokemon:

1 out of 5 teams: Dream World - 1 / Wifi - 0
1 out of 10 teams: Dream World - 10 / Wifi - 12
1 out of every 15 teams: Dream World - 24 / Wifi - 26
1 out of every 20 teams: Dream World - 35 / Wifi - 37
1 out of every 25 teams: Dream World - 47 / Wifi - 44

If you use pokemon in every battle instead of pokemon in every team, all the numbers above would decrease.