About the WCoP and lack of transparency issue

Hi !

I'm writing this thread because the one about WCoP got locked without any warning: in fact, many of us thought that, with the results of zomog's poll, the decision was already pretty clear. Apparently not, as we saw the exact opposite being decided, without actually having time to argue against it.
I'm also posting this here as this situation is actually akin to others that we have seen recently, even though it was generally more about people's bans. There is a clear lack of transparency, recently.

Now, let's be clear: I don't actually care that much about the tiers that are gonna be chosen for the next WCoP. In fact, the probability I will personally get involved in it is very, very close to zero. This being said, I have a problem with decisions being made against the apparent consensus. Let's also be clear on something else: I am not saying that there has been any move by the TDs to actually "force" the decision against the relevant consensus. I'm simply arguing that, considering that the only tangible thing that we saw was zomog's poll, the least the TDs should do would be to show some sort of poll of their own. I can conceive that you TDs don't want to get everybody's opinion on wether SS OU should be the only tier in that tournament this year or not, but why ask all of this in such obscure manner ? And besides, why only ask current teams ? These teams are precisely made out of SS OU players only. For instance, even though I'm clearly not a fan of low tiers in WCoP, it's definitely unfair for lower tier players: how could they ever see a change in the format if the only persons whose opinions are taken into account are SS OU players ?
Finally, even though the spectators are obviously less important than the actual players, I don't think that straight up ignoring their opinion is the right move.


What I propose is this: we should make a poll similar to what is done for the OU surveys. Make two subgroups inside of the poll: one being the qualified voters (setting the bar higher than for OU surveys, so removing the ladder feat, asking for instance "have you played SPL, WCoP or Snake/SCL in the last 2 years", or something like that), and the other being the rest. Then, you analyze the results, and base your decision off that. You can even argue that a slight majority for a format change wouldn't be enough to change it now or something. Just don't let a discussion run with a conclusion that seems obvious to pretty much everyone, then declare that you choose the exact opposite without letting anyone argue there anymore.

tldr: please more transparency in general, and in particular on this WCoP matter.

PS: I've seen quite a lot of discussion on the smogtours discord, but these discussions are far harder to keep track of if you're not there when they happen. I think it's preferable to have clear answers that anyone can find afterwards on this type of issue.

Thank you for reading !
 

Finchinator

Harry's House
is a Tournament Directoris a Top Social Media Contributoris a Community Leaderis a Community Contributoris a Live Chat Contributoris a Top Tiering Contributoris a Contributor to Smogonis a Top Smogon Media Contributoris a Super Moderator Alumnusis a Battle Simulator Moderator Alumnusis a Past WCoP Championis a former Tournament Circuit Champion
OU & NU Leader
The TDs should absolutely be transparent at all possible times. The same should apply to other aspects of the site (like OU did, as you mentioned). Let's start now to make sure everyone is on the same page.

I am a TD and I do not support all CG personally, but I do support finding the best format for our playerbase and finding the best method to do that. I very much accept if that is all CG (regardless of my personal beliefs or what helps my team), but we are doing our best to make sure.

Upon hearing criticism about the prior process that was applied, we worked on another survey that offers more complete coverage of the situation and hard data through a ranked choice vote; this was discussed at length on discord and sent out to the captains recently. We believe this process covers the constructive criticism we received.

Amaranth's efforts were great to get us to this point and he went above-and-beyond talking to everyone, but written-out data is not necessarily the most empirical choice for a situation like this always. This should still not undermine the line of communication he established or some of the progress we made as a community in choosing possible formats. Nobody gains anything by slandering him or any other contributor to these tournaments though, so we appreciate maintaining a constructive tone rather than a critical one. It is not just his decision that he is posting about and it is not just him revising decisions either -- we are a team working with the community on each decision.

The results of the survey will be publicized when possible in an effort to remain transparent as well.
 
On behalf of the Greek team, I would like to express our dissatisfaction about the turn this procedure has taken after Amaranth's post yesterday. While the survey is useful to an extent, we feel the use of numerical data alone to decide the format is not appropriate in this case for a couple of reasons:
1. The difference in preference is not depicted at all. There may be teams that are fine with whatever format and teams that can hardly field a lineup if a certain format is chosen, but both teams' least preferred options are labeled with a "5".
2. Teams are encouraged to vote in a way that doesn't depict their actual order of preference. An example of what I mean by that: our preferred option for the format is 3 SS OU + 7 old gens, so it's obviously our first choice. The problem is that the best way to push for this format with our vote is to order the remaining options not in order of our actual preference, but in order of popularity (the most popular formats placed in the bottom, maybe with an exception for our second favorite format). This severely skews the data and almost invalidates the survey itself and the argument that teams are expected to vote honestly is very naive. I raised that point to Amaranth yesterday and I was told that the TD team will make sure the order the teams voted is no different from the order of preference they previously expressed to Amaranth, which did alleviate my concerns (Greece did vote honestly, for the record), but doesn't this defeat the purpose of the survey ? I mean, if the point of the survey was to make sure Amaranth couldn't misinterpret what the managers told him, then either the managers were not clear enough themselves or there is no trust in the TD team at all.
Last but not least, we are also baffled by the way a certain portion of the community reacted to the decision: criticism made in bad faith to undermine the process and use community backlash as a means to make a last push for their preferred option. When the decision was published, we didn't exactly celebrate it (that format is not our preferred option), but we respected it because we didn't have any qualms with the procedure followed to help the TDs decide the format of the tour (nobody else had expressed any serious complaints either) and because the decision itself simply felt sensible. Anyway, let's see where this survey takes us, but the whole situation feels like a mess now and it certainly didn't have to be that way. :worrywhirl:

P.S.: This post would had been made in the other thread had it not been locked, but that feels like a common practice after a decision is taken and that's indeed problematic.
 

Finchinator

Harry's House
is a Tournament Directoris a Top Social Media Contributoris a Community Leaderis a Community Contributoris a Live Chat Contributoris a Top Tiering Contributoris a Contributor to Smogonis a Top Smogon Media Contributoris a Super Moderator Alumnusis a Battle Simulator Moderator Alumnusis a Past WCoP Championis a former Tournament Circuit Champion
OU & NU Leader
2. Teams are encouraged to vote in a way that doesn't depict their actual order of preference. An example of what I mean by that: our preferred option for the format is 3 SS OU + 7 old gens, so it's obviously our first choice. The problem is that the best way to push for this format with our vote is to order the remaining options not in order of our actual preference, but in order of popularity (the most popular formats placed in the bottom, maybe with an exception for our second favorite format). This severely skews the data and almost invalidates the survey itself and the argument that teams are expected to vote honestly is very naive.
Just so you’re aware: this is untrue and I think you are misunderstanding how ranked-choice voting works.

For example, if format A and format B are the two most popular choices, it only matters which format any given team ranked ahead of the other rather than the distance between the two. Putting format A first and format B fifth means the same as putting format A first and format B second in that scenario.
 
Just so you’re aware: this is untrue and I think you are misunderstanding how ranked-choice voting works.

For example, if format A and format B are the two most popular choices, it only matters which format any given team ranked ahead of the other rather than the distance between the two. Putting format A first and format B fifth means the same as putting format A first and format B second in that scenario.
We had indeed misunderstood how this exactly worked, but there's still a problem with this system, different for each definition of the most popular choices. If the most popular choices are defined by any sort of mean (weighed or not), then tactical votes can obviously skew the result. If the most popular choices are defined by the simpler "this option was ranked first the most, the other was ranked first the second most etc", then you do not have skewed results, but you do not take variance into account and that can lead to sloppy interpretation of your data.
 

Amaranth

is a Tournament Directoris a Site Content Manageris a Social Media Contributoris a Forum Moderatoris a Community Contributoris a Tiering Contributoris a Top Contributoris a Past SPL Champion
Moderator
Here, now that it's fully confirmed, have my full point of view on the events of last week. For transparency.

On the original decision

I started looking at this issue thinking it would be resolved much like most things that need TD resolution to happen: the community gives input, the Tournament Directors look at the input, discuss it, and come out with their own decision.
Somewhere along the way, between z0mog's survey, the general controversial nature of this matter, the inability to reach full consensus within the TD team, and my decision to hear out the opinions of team representatives / captains, the general perception became that this would be decided by a poll of some kind. I understand, in hindsight, why this happened - but this is not how a vast majority of TD decisions are done.
Having 20+ one-on-one conversations is NOT the standard. It's already far above the standard. Most TD decisions definitely don't require a data science minor; they don't require dealing with data much at all.

I defend my original approach as extremely reasonable. Data is cool, but there's a reason we don't generally use it - there are plenty of things that cannot be codified easily into a vote. How many 'votes' is the fact that CG OU is the status quo worth? How many 'votes' should each qualifier team have? Do historic teams that have fallen out of WCoP, such as team UK, deserve to be treated the same as a team that just formed for last year's edition? And what about our own opinions as TDs? Are those worth more than the captains'? Less? The same? Not worth anything at all? And what about situations where the team captains don't have an ordered preference at all, and rate certain formats as equal to one another? Is ranked choice voting still really the best way to do this then?
These are all questions that can be answered. I'm not saying they can't. But if you want to run a poll on anything, you should answer these kinds of questions first - and as things stand, we haven't. If the community thinks captains should decide on WCoP matters through voting, then maybe we can establish some kind of council of representatives in the future, with rules and guidelines to address these points ahead of time. But it doesn't exist right now, and so I treated this like I would treat any other decision - stimulate discussion and engage with the community until I think my grasp on the situation is sufficient, discuss with TDs, and come to a decision using my own brain. That is literally my job as a TD. My conversations were less going around and asking for "votes", and moreso asking for thoughts. It's near unprecedented to take a policy decision as a result of a survey, and there are reasons for that - it's not as simple as "just collect data and present the results", if you want to be thorough. I believe this much acclaimed and requested survey was in fact far from perfect, and that is why I did not elect to start one myself.

The only reason I brought up number of preferences at all is in my post is to try and give some evidence as opposed to no evidence when I said that there is actually considerable support for CG OU, in spite of z0mog's survey and whatever other data points against the format. Evidently providing some evidence was not a good move - it made what was essentially a "we talked and decided this" post read like a "we ran a vote" post, when that was never my intention. I easily acknowledge that as a mistake in hindsight. But if you think back to pretty much any other big decision, you can see that the standard is pretty much "we talked and decided this, the reasons we think this a good idea are as follows, thanks goodbye" - I feel like in that context you can begin to understand my point of view when I (at the time) felt that my work had been easily sufficient.
The bottom line is that I treated this like a normal TD decision. Most likely it was an extraordinary TD decision that required other protocols. It was absolutely a mistake on my part to not realize this, and for it I apologize, but it ultimately comes down to our judgement. As much as it was lacking in not realizing the issue was too hot to answer with a simple TD decision, it was at least good enough to confidently select the answer that was confirmed to be correct by the survey (which, by the way, had its fair share of problems).

Again, these paragraphs are primarily to explain why I thought my actions were reasonable at the time. I still defend some of them, but I acknowledge some others were insufficient, and will try to learn from them. I extend my apologies to anyone negatively affected by my errors.

On "Transparency"

Transparency is a funny word on Smogon. I stated multiple times in the Smogtours Discord server that I would answer any and all questions and provide any and all info that led to the decision in DMs to anyone who requested them. I received one DM from one person and I answered their question to the best of my ability. But for the most part, I was immediately attacked by multiple users and an onslaught of questions began - not with the goal of understanding my process, but with the goal of finding anything they could attack in an attempt to undermine the result of the decision.
Any valid points get ignored and drowned out by other chatters repeatedly, because you are arguing 1 against 5+ and you cannot possibly type fast enough to keep up. And any mistake, no matter how tiny, gets torn apart by said group. They ask questions not with the intent of trying to understand the reasoning, but with the intent of humiliating the decision maker for taking a decision they disapprove of. They can get away with saying literally anything, because they have no responsibilities and, as individuals, are just one in a big group. Meanwhile, we cannot get away with saying anything that is even remotely disputable, because they have more than enough manpower to dispute it to the end of time until you eventually have one slip up. "Transparency" is not achieved through these conversations at all.

I understand the requests for higher transparency, and it's something I strive for myself. I want to be open about our thought process. But each and every single time a TD goes above and beyond to communicate as much as they can to the community, this sort of thing happens - how much transparency can you really expect in such circumstances? I have a fairly high tolerance for this kind of thing and will continue opening up about my process as much as reasonably possible, but this sort of shit is precisely why not many people remain TDs for very long. Why subject yourself to incessant group bullying, verbal abuse etc. when you can just quit?

So, once again, if anybody genuinely wants transparency, you are welcome to contact me in private and ask me about any information you have not received yet or ask me to share with everyone something that you think ought to be public. I have no problems with sharing as much as the community wants. But I don't want to subject myself to toxic people arguing in bad faith any more than necessary, I think I've had enough of that for a full year on Monday. And if you're one of the people who do stay civil and respectful, I cannot thank you enough - I do this for users like you.

That about covers everything I wanted to say. Thank you for reading any of it / all of it. I hope that was transparent enough.
 
Last edited:

Users Who Are Viewing This Thread (Users: 1, Guests: 0)

Top