Sorry but I don't really get the logic behind this (I'm not referring to you in particular FlareBlitz). So deciding when the metagame changes is an arbitrary choice, while allowing people to vote only after their reasons have been evaluated by a restricted number of potentially biased judges isn't?
Much about our suspect process is arbitrary, including the rating threshold. My argument against the "determine when the metagame changes" rule isn't so much based on the fact that it's arbitrary with reference to some ideal suspect testing standard, it's based on the fact that it's arbitrary with reference to its measurement. That is, it's hard to put a number on the metagame itself.
Your comparison to the paragraph system is valid on principle, but not so much when we actually look at the degree of subjectivity involved. A "good" paragraph and a "bad" paragraph are usually pretty clear-cut, at least based on my experiences in the UU nomination thread. Good paragraphs offer reasons that a suspect is broken based on the characteristics of a desirable metagame, while bad paragraphs do not. Perhaps there are paragraphs that are kind of in-between, in which case some subjectivity is required. I'm not a tiering lead, so I do not know how common such a case is.
But regardless of how subjective even the most borderline paragraph is, analyzing a metagame change is far more so. Would an increase in usage of 0.4% be significant enough to warrant re-testing? What about 0.3%? Or 0.5%? Would the change of popularity of a set (say, SD versus choice) be sufficient? And it's not just about the raw data, it's also about the interpretation of that data. What if a counter's usage drops? Would that mean the metagame is less prepared for a suspect due to other threats that require attention or that the suspect is no longer threatening due to the reduced preparation?
In order to make such a system workable, we would either need a very convoluted and detailed system of standards that would require constant revision, or we would need to rely on an opaque system of simply deciding whether the metagame has changed or not every three months for every possible repeat suspect. Neither of those two options would make the suspect process better in my opinion.
But I want to briefly touch on what may be a larger issue here. Ultimately, the entire point of all our other suspect qualification systems is to ensure that voters know what they're doing. If voters decide to nominate a suspect, and then vote it out, that should be that. If we need to introduce further protections such that our voters cannot nominate certain Pokemon, perhaps we need to fundamentally revise our estimation of the competence of our voting pool. I have no idea how we would begin going about this, but if our real problem is that voters make stupid decisions, shouldn't we prevent them from even being voters? And if we do feel that most of our voters are competent, then we should just let their votes speak for whether the metagame has changed enough to nominate and subsequently ban a suspect.
I hope that made sense :)
Which I agree with. Having nominations for previously OU suspects held to a higher standard could potentially be a good middle ground. Higher standards could be stuff like explaining why <pokemon> is more banworthy this round then last round and why this is (what metagame changes have led to this for example).
Yes, that's what I meant. Obviously we don't want suspects re-nominated ad infinitum because it becomes somewhat of a "guilty until proven innocent" type deal. However, opinions can and do change, and incorporating this into the existing paragraph system seems like the best way to go.