No, the objection is that no one even can use said Pokémon, and they forget why they didn't like it in the first place. Similar, yes, but seriously you underestimate the power of a good judge like yourself... It is very hard to bullshit an argument on a Pokémon.
Actually, several people have specifically stated that the month-long period is faulty because it doesn't provide useful information on a given Suspect:
Finally, since they can't play with the suspect during this one month process, how are they supposed to know if its broken or not? The very idea of taking the suspects out compromises peoples opinions on the matter. Could you imagine if Jump and Aeolus made a post tomorrow saying "ok, to see whether or not you think Garchomp is broken, we are going to prevent you from using it"?
And the suspects and the metagame are entirely intertwined. You can't examine a suspect without examining the metagame and vice-versa. I can study an electron by observing how it interacts with other forces, objects - this is no different.
Playing a metagame without a pokemon does not tell you anything about the pokemon's performance in that tier. How are we supposed to tell if Froslass is capable of enabling a teammate to sweep easily if we can't even use it to find out?
Finally, those that played in both months have been given an alternate metagame that not only proves nothing about the suspects themselves, but also puts a skew on their vote by allowing them to get a feel for which metagame they "like better".
I agree with Gay Dolphin that the month-long testing period is rather ridiculous; it really does not help us to determine the power of the Suspects, or give us any helpful information regarding the Suspects themselves.
The concern expressed is the same throughout—you don't gain experience with a Suspect, so how can you be expected to have a qualified opinion? It's the same thing as listening to someone who played with and against Crobat three times.
Chris is me said:
The key to the proposal you quoted was what came immediately after that line. I understood this. My and Obi's fears come entirely from data contamination, as the UU votes are very much based on the metagame at hand, contrasting with the OU suspect test.
Explain in detail how the data can be contaminated.
The fact that you get more Suspect Experience by using it than by not using it is the issue. You can get by with just playing against it, but why would you want to if you well and truly believe it sucks, or it doesn't work on the same team as another Suspect? You would be motivated to build these teams because it increases the chances, it would be foolish not to. Then the metagame is affected.
If the Suspect actually sucks, then most people either won't use it very much or win as much as they would otherwise when they do. There are no two ways about this. A player can't maintain that a Suspect sucks if every one else is not only using it, but doing well enough with it to get the Rating/Deviation needed to vote, and more importantly the SEXP to qualify their votes (since, as I stated, winning with a Suspect is very important to gaining SEXP for it). The best example of this is when FiveKRunner claimed that Latios was "dead weight most of the time", which is silly in theory but rendered even sillier when you contrast his SEXP with that of those who also had the Rating/Dev to potentially vote but had much better SEXP with Latios.
Besides, this doesn't even apply to UU like it does to OU where you know the Suspect(s) beforehand. Say for the sake of simplicity we go back to before Crobat and Froslass and Raikou and Abomasnow and Gallade and Staraptor were even nomimated. There aren't any Suspects. So, if after a month, a player both thinks Crobat sucks so he didn't use it that much and also only faced it six or seven times (relatively much lower than most everyone else who played on the UU Ladder for that month, a real number isn't needed), why should we listen to his opinion on Crobat? Why? And why should he even want to sound off on Crobat? He literally and objectively did not experience it very much at all, so why should his opinion on Crobat bear any weight?
(I'd make a "how are people supposed to know this" quip, but honestly that's not even the issue anymore, and if the system was open to begin with then we could make these criticisms in the first place in an attempt to make an objective barometer with no contamination)
(it never has been an issue, doug, chaos and now aeolus all understand the formula, how objective it is, and how much work i put into making it objective)
Isn't that evidence that Crobat is not a great Pokémon? Perhaps "he's using it incorrectly" but then his bold vote chronicling his experiences would come into play where he bashes how bad NP Crobat or whatever gimmick is. But if he loses when he tries to use it and provides a detailed analysis of his experimentation with the sets of the Suspect, and they match up with "the broken set" as proposed in the nomination... Why would that vote be bad?
It's not like if you have low Suspect Experience you can say "this set sucks lol" and get away with it in a vote.
Ideally the Suspect process would only nominate close to broken Pokémon, but strange things have happened.
That vote would be bad if and only if he only used it a few times, which Doug and I can clearly and objectively determine at a glance. And yes, his bold vote would come into play—I actually make it a point to not cross-reference a player's SEXP with his or her submission before reading the submission first, as this would actually stand a chance of biasing me for or against the vote.
And of course you wouldn't be able to get away with "this set sucks lol", this is the entire reason I'm suggesting using SEXP for the UU Suspect Test.
I don't see the point. You're an admin on a Pokémon site, an active participant in the community, that doesn't sound so far fetched. Are you trying to say that you're so experienced that random users who clearly don't know how to use Smogon.com can tell? That's not much of a qualification.
This isn't an attempt at saying these battlers have the wrong idea, but these are hardly qualified judges. I'm sure chaos gets just as many RMTs (maybe less since he doesn't have the badge).
I brought it up as a nod to my supposed "eloquence" or whatever you want to call it. As far as they can see I don't actually play, yet I am still regarded as some kind of expert. How often I play or when I play is a non-issue—maybe I play a ton under alts, maybe I don't—as the point is that one can be eloquent and therefore give "judges" the impression they know what they're talking about when they really don't. The "really" as far as it relates to the Suspect Test process is SEXP.
And I very much doubt chaos gets very many RMT requests at all. As I'm trying to say, the
only reason people have any reason to believe I'm an "expert" on OU is because of my posts and consistent activity in Unchartered Territory, Stark Mountain, and Policy Review, and maybe the knowledge that I wrote a handful of early DP Anaylses. chaos will be the first to tell you he doesn't have much of a handle on DPPT and has not for two years.
When the objective measurement for said data contaminates it, we have a problem. Why would you pose that question when that is the complete objection everyone has been posting about in the entire thread? It's obvious that's "our" answer here, but this post has done nothing to quell our objections on the data contamination front. In OU, this isn't an issue since the entire premise of the test is different (not to mention the Suspect Ladder prevents contamination of the "real" metagame if you feel like comparing) but in UU it just won't work.
Again, you're going to have to clarify what you mean by "contaminated data". And the quotes I included above indicate that I'm addressing one of the objections posed in this thread (the only one that can be addressed with SEXP).
