You have to "experience" a pokemon to know much about it. I quote the word "experience" -- because I am not just talking about "using" it. There are many facets to "experiencing" a Pokemon.
Does putting a suspect on your team connote "experience"? Yes, it does. Because, presumably, you had to think about that pokemon's role in relation to other team members. If you later battle 100 times, and 6-0 every opponent with your lead pokemon and never use the suspect in battle -- I could argue you still got a little bit of experience with the suspect. More experience than people that never even bothered to try and put the pokemon on their team.
Does using a suspect in battle connote "experience"? Yes it does. Because despite all your best laid plans during team building, there's nothing like sending that pokemon out in battle and seeing if it works the way you planned. Whether the suspect works or not, you probably learned something about the pokemon by battling with it.
Does facing a suspect in battle connote "experience"? Yes it does. Because despite all your team building planning, and all your battle experiences with using a suspect -- there's always other people out there with different ideas and different strategies. By facing a suspect wielded by a different battler, you learn new things about that pokemon and how it can be used.
I won't keep going into finer details of experience. Hopefully, you get my point. I mention it here again (we've said it many times before) because Jumpman and I have continually referred to "experience" with relation to SEXP, and yet many people in this thread and other places continue to imply that SEXP = "Using a pokemon". And because these implications are being repeated over and over by people (CIM, SDS, etc -- I'm looking at you) -- members of the community reading these discussions now accept it as fact that "SEXP = Using a pokemon". Well, it is not true.
And don't give me that bullshit, "Well, you won't disclose exactly what is in SEXP, so I don't really know what is in it." That is NOT an excuse to continue to proliferate the lie that "SEXP = Using a pokemon". That reasoning is basically saying, "Well, I don't know the full truth. So I will intentionally make false statements and encourage others to believe it. And until you disclose the full truth, how do we really know that my statements AREN'T true?" If anyone out there thinks that kind of reasoning is good, logical thought -- then I will no longer dignify your arguments with a response.
So, I have given a clear explanation of what we mean when we refer to "experience", regardless of what other lies you have heard to the contrary. Which brings me to the real issue that needs to be clarified:
"Does a player need experience with a pokemon in order to participate in deciding its tiering status?"
The current Smogon tiering leaders believe the answer to that question is a resounding "YES". All of our current processes and procedures are based around the notion that tiering should be decided by people with three qualifications:
1) They are smart
2) They are skilled battlers
3) They have experience with the pokemon being tested
There is no way to 100% accurately measure those three characteristics.
We try to measure #1 by reading a submitted paragraph and analyzing the quality of the arguments and reasoning. It's subjective as hell, I know. And it can be gamed, by getting someone else to write your paragraph, or copying from someone else. But, since we can't rely on people submitting IQ scores or anything like that -- it's pretty much the best thing we have. If you have better ideas about how to ensure our tiering voters are smart, I'm interested to hear it. If you disagree that voters should be smart -- that's fine too. But, be aware that we probably aren't going to change that goal any time soon.
We try to measure #2 with ladder ratings and deviation scores. It's an imperfect measurement, I know. If anyone has bothered to look at the details of Glicko2, and the way we use it in Shoddy Battle -- it's really not a great way to represent battle skill for competitive pokemon. But, it's the best we have right now. When we get X-ACT's GLIXARE system in place, maybe that will improve. But, for now, we are using the the Glicko2 ratings to identify skilled battlers. If you disagree that voters should be skilled battlers -- that's fine too. But, be aware that we probably aren't going to change that goal any time soon.
Which brings us to #3. Prior to gathering SEXP data -- there was no way to measure experience (as defined above). We could only hope that smart people with good ratings would actually go out and experience (as defined above) the suspects being tested. However, we knew there was actually no way to ensure that with our previous process. And based on some rumors and innuendo, we were suspicious that some "qualified voters" actually had little to no "experience" with the pokemon being tested. In our opinion, that is a very bad thing.
Many pokemon players have deep-seated preconceived notions about certain pokemon. Maybe it developed from their ingame play. Maybe it comes from their theorymon impressions they got when they first saw the pokemon's dex entry. Maybe these players have heard lots of rumors and believed them. Whatever the reason, we know that some players will develop strongly held beliefs about a pokemon's competitive relevance -- without actually experiencing that pokemon very much in actual competitive play. We knew this was true prior to developing SEXP -- but we had no way to prove it objectively.
With SEXP, we have an objective measurement. Is that measurement a perfect representation of "experience" with a pokemon? No, far from it. In fact, SEXP is no more accurate than the other measurements we use for determining voter qualifications. We know that paragraphs are a shitty way to measure intelligence -- but it's the best measurement we have right now. Glicko2 ratings are a shitty way to measure pokemon battle skill -- but it's the best measurement we have right now. And SEXP is a shitty way to measure experience -- but it's the best measurement we have right now.
I know many people that object to SEXP, are simply upset that the data is not public. You argue, "Well I don't know if SEXP is good or bad. Only the admins have seen it." Yeah, well... get over it.
We don't disclose the exact reasons for rejecting every voter's paragraph logic either -- and no one has ever complained about that at all. For all you know, Jumpman may be rejecting paragraphs based on whether he likes the username of the battlers. All you have is a vague description of the general criteria Jumpman SAYS they use in evaluating paragraphs. Beyond that, you simply have to trust that paragraph evaluators are doing a decent job.
We have rejected voters for cheating their battles, and I have never disclosed the algorithms we use for detecting cheaters. For all you know, we may be rejecting voters based on bad data. If you ask the cheaters, they will certainly deny any wrongdoing, and claim that our information is erroneous. Ultimately, you have trust that we are doing a decent job with cheat detection.
SEXP is no different than some of these other unpublished aspects to voter qualification. Yet for some reason, you people have started flag-waving about "admin secrets", as if it's some big cover-up. And personally I'm sick of it. I'm sick of people ignoring published statements about what "suspect experience" means. And I'm sick of people beating conspiracy drums about the things that aren't published. Suspect EXP is a useful tool, that's all. It's not a perfect tool, but nothing in the tiering process is perfect.