Rejected Improving the quality of tournaments through seeding

[Isa]

I've been a proponent for seeding in individual tournaments since the start of my Pokémon career, and indeed my first few RBY tours back on RBY2K10 that I joined also had the players seeded. At times I perceived some of the seeds as not optimal, but the system was never in doubt and made the tournaments have a higher quality. Outside of RBY2K10 and select Pokémon Perfect tournaments though, there's been a very saddening lack of seeded individual tournaments within the Pokémon community. After getting paired with top players today in both GSC and RBY Cup, I got frustrated and complained about the draw of the bracket and decided to more actively revisit this subject.

Almost everyone of the ones reading this passage can recall instances of themselves having to face top players early. While I was a TD I spoke to various users about seeding and some remarked that they enjoy playing versus top players at the early stages of the tournament to get competitive games throughout the entire tournament. This is not however something that a tournament should have as its primary goal - the goal of a tournament should always be to create results that can be interpret as an accurate indicator of individual skill. This can be achieved in various ways - double elimination is one way, increasing the games played in a set is another. However I believe that the single biggest change to improve tournament quality as of right now would be to seed tournaments.

As said above, the purpose of having seeded tournaments, and of tournaments in general, is to ensure that the tournament results are as reflective as they can be of the participants individual skill level. Right now however many tournaments on Smogon (especially unofficials, but also OST and individual Classic cups/Slam opens) are running single elimination tournaments without seeding. This results in very high variance in matchups, and players can make it very far into tournaments despite being clearly inferior to multiple prior eliminated players, strictly through the luck of the bracket draw. This is not a feature, it's a bug - tournament results are meaningless if you cannot confidently say that even your second or third placed participant has a skill level that is indicative of their respective placement.

Moreover, multiple Smogon tournaments, such as the aforementioned Classic and Slam, rely on results from the entire bracket and not just whoever places first, making it even more imperative to ensure that these tournaments can fulfill their set task. As of now, even though in theory the "best" player going in to the tournament should always win said tournament, we all know that this is not the case. Even the most elite players face upsets. The odds of getting upset by another player increases with the skill level of an opponent though. You can look at the top 8 of any decently big tour - blunder exemplified it very well in OST last year by comparing the runs of the quarterfinalists, as can be seen here. Malekith had a bracket filled with very skilled players, whereas most of the other quarterfinalists faced only one or two skilled opponents. This variance meant that although it could be argued as "fair" in theory when everyone has the same shot at playing the same individual opponent, in practice it makes individual runs very uneven in difficulty. With seeded tournaments, all players - including both the highest and the lowest seeds - are guaranteed brackets that are similar in difficulty to what other players are experiencing. You would not, as is often the case now, have multiple players in late portions of brackets that have fought very few opponents of significant skill while one or two have fought through very difficult brackets.

The biggest difficulty with seeding tournaments on Smogon would be to identify a system that is fair. For some metagames such as CG OU, there's plenty of tournaments that can be used to draw results from. Not all metagames have this fortune however, as OUs from past generations and various lower tiers simply do not see the same representation within the tournament community. Previous attempts at seeding a few unofficial tournaments have been made, but were done manually by a group of people. While I am personally completely OK with this method (it is used in Super Smash Bros. Melee, the other game community I have a history of hosting and competing in), I understand that many would prefer a more "objective" system than that. My proposal is as follows:

• Design the Circuit Tournaments to be used for seeding. Start X amounts of new tournaments per year and official tournament-relevant metagame that wishes to see seeding. These tournaments would ideally be open to everyone and would run in a format that'd allow for prolonged participation - ideally a Swiss-system tournament where players are not eliminated.
• Create a public database for Circuit Tournament results, managed and updated by the tournament host and/or members of the TD team.
• When hosting a non-live official tournament without a prior qualification phase, create a tournament or bracket, ideally on smash.gg or challonge.com (or a not-yet-developed Smogon system that'd allow for seeding), to ensure that the seeds are all placed on the correct sides of the bracket.

I have chosen to not focus on the specific numbers involved in this suggestion, as I do not believe them to be of significant importance. Tournament hosts may decide to only seed the top 32 players in any given tournament, or only the top 25%, there are multiple ways to track results and handle seeding as a result of those pairings, maybe official individual tournaments should also count for future seeding purposes etc. etc. These are all up for debate. My personal opinion is that every player involved in the seeding-related tournaments should be seeded according to their results and that all win/loss by raw numbers makes for a decent starting position until a better system is developed, but this could be changed. It could also be argued what tours should use this system were it to be adopted - should OST use seeds or not, do you really want to change the format of the first tournament?, - again I feel like this is a secondary question, and right now the primary question is if and how we should implement seeds.

I believe that seeds would vastly improve the quality of tournaments, and I'd like to hear what the community and Tournament Directors have to say about it.

 

[soulgazer]

After getting paired with top players today in both GSC and RBY Cup, I got frustrated and complained about the draw of the bracket and decided to more actively revisit this subject.

I genuinely don't see how you plan to properly, without bias, seed every players for different gens, tiers, etc. in Pokemon. It's really hard to compare Smash and Pokemon on that point. Pokemon being highly side effect (luck) heavy compared to Smash is one difference as well. I could easily add in more and more.

If anything, double elim > seeding

 

[Nails]

While I was a TD I spoke to various users about seeding and some remarked that they enjoy playing versus top players at the early stages of the tournament to get competitive games throughout the entire tournament. This is not however something that a tournament should have as its primary goal - the goal of a tournament should always be to create results that can be interpret as an accurate indicator of individual skill.

i hard disagree on this point, that you base your entire argument on. i don't feel that the purpose of an open tournament should be to create a set of results, i feel that the purpose of an open tournament is to offer a fair, even chance to every player to make it as far as anyone else. we already have spl as a tournament where you have to be known to get in and play old gens, and classic is the only other tournament where old gens have representation in the official circuit. i don't think we should skew literally everything in favor of established players and shut out up and comers.

As of now, even though in theory the "best" player going in to the tournament should always win said tournament, we all know that this is not the case. Even the most elite players face upsets. The odds of getting upset by another player increases with the skill level of an opponent though.

we're playing pokemon, not smash. the best player isn't meant to always win his sets. you know that, but leading a paragraph with that sentiment sort of irked me, and you're framing "upsets happen" like it's a bad thing. top players should have to earn their wins just like bottom players; random brackets are the only way to be fair to everyone. a top player isn't entitled to a disproportionately higher chance of victory than a mid level player aside from what his skill provides him. i feel extremely strongly about this point.

i'm not going to get into the nitty gritty of "how can we ensure that the TDs are seeding the most skilled players?" or "why should the 32nd ranked player have such a massive advantage over the 33rd ranked player?" because while they are issues, i just disagree fundamentally with seeding in tournaments and don't think the discussion should even get that far, at least for tournaments that aren't invitation-only. we don't have stream numbers that will take a hit if established vets aren't making it through to the top 32; i only see down sides to seeding.

edit: removed an unnecessary comma

 

[Finchinator]

While I think we can all acknowledge the pros of seeding, which Isa outlined quite nicely in the OP, I do not think that it is optimal to resort to for current generation tournaments or tournaments that are particularly large in size. In addition, I think we, as a community, would need a partial philosophy change to adapt this to any official tournaments at all, regardless of the tier or size, which I do not view as particularly likely.

As for why I feel the seeded tournament structure is not optimal for current generation tournaments, I think that it has a lot to do with the nature of playerbases and metagames; if you look at any of the communities, the "top players" are often pretty stable and set, but the group of "tournament players" and respectable tier mains often has a fairly high level of variance/turnover as time elapses. In addition, there are always the hidden gems in the mix and I feel like there is no real way to seed accurately and consistently because of these factors. I think that a great example of this is with the NU community; personally, I have always kept ties with the metagame, but I tend to frequent it playing wise much more during NUPL/SPL/GSlam time and during those times I consider myself to be someone who would be seeded near the top, but during other parts of the year, I would often favor many more dedicated, yet less established tier mains over myself. This goes for a number of other tournament players in lower tiers and it is not always easy to gauge where they are at at any given point in time. Continuing on the NU comparison, over the last year/handful of months alone, a number of players rose to relevance and if a tournament like Grand Slam were to include a seeded NU Open (totally using this as an example, not proposing this as an application), the seeds established at the start of the tournament are very likely to seem very off by the middle stages, let alone the end, considering the continuous evolution of the playerbase and public perception of the players of the tier.

As for why I feel the seeded tournament structure is not optimal for larger sized tournaments, it is pretty simple -- we do not know who a lot of people are and sorting out the bottom half (or even potentially more if we're talking OST/classic cups) of seeds can basically become some glorified guessing game where funny names end up getting higher seeds and the rest is, more or less, randomly picked. Sure, you can argue that the top/middle-end is what truly should be emphasized upon, but the fact of the matter is that every portion of seeding/player pool matters equally as every single pairing holds the same weight in a tournament. To show this, I want to use another example from NU, but this time I want to hone in specifically on their seeded tournament that was held last year. It was actually a very enjoyable experience to play in this tournament, but that was largely because I liked the metagame as opposed to the seeding aspect. If anything, the seeding aspect proved to contradict what it truly intended to do with regards to rewarding top players and stacking the middle-of-the-pack up against each other. To exemplify the former, a seemingly random user who got seeded dead last by pure coincidence because he was one of the dozens of randoms in the field ended up defeating FLCL, Z+V, and Disjunction, all of which were seeded relatively high (you can see the full bracket here and diif's run (the user I am alluding to) here). The fact of the matter is that the "randoms are dangerous" quote from the whole BlooStyle fiasco very much applies to seeding of large portions of fields of any larger tournament and I do not like the consequences this has on the tournament, especially when it has large potential to undermine the whole purpose of seeding in the first place. Seeded tournaments are best for fields where the seeders/system has formal data or experience with all of the players involved and this simply is not applicable to almost all Smogon tournaments, especially individual officials.

Finally, I have noticed that Nails alluded to this in his post and I couldn't agree more -- nowhere on Smogon have we established that we want to give any priority or preferential treatment to any type of players, be it middle-of-the-pack or top players. This type of seeding where a certain player gets an easier draw, especially in the early rounds, makes a lot of sense when there is some qualification phase, which is why we see seeded brackets in pretty much every playoff, but it does not make as much sense to me when we are giving officials/tournaments a fresh slate and a lot of new users are joining, as I touched on above. All things considered, unless we want to have a pretty drastic departure from our current philosophy in this regard, which admittedly is not really outwardly stated so much as strongly implied by current tournament structures, then the seeded structure is not a fit for our flagship tournaments, in my opinion.

edit: also, it's worth noting that in the Smash community, seeding is often utilized in tournaments where there is an entry fee or a significant monetary prize involved, which means that RNG match-ups can be quite annoying in the earlier stages when people who normally would be serious contenders are not in the money. Our only money tour is OST, only one person of well over a thousand wins it, and there is no entry fee, making the monetary opportunity cost of joining being absolutely nothing compared to potentially noteworthy amounts for a Smash tournament depending on specific quantities/economic factors.

 

[Tony]

Both Nails and Finchinator have touched on most of what we feel holds seeding back with regards to Smogon Tournaments, but I wanted to add that there's there's no objective and uniform way to fairly seed either, which makes the concept massively impractical to implement. I can empathize with getting a rough bracket, but seeding isn't something we're planning to implement anytime soon.

 

[McMeghan]

Hello everyone,

With the beginning of the 6th Smogon Classic, I got reminded at how unfair this tournament can be to some people. You have players coasting through the early rounds off activity wins and facing people who don't know the mechanics while some other are pitied against specialists in half the cups they joined. This is an issue that can be seen in Grand Slam as well. I think it's dumb how big of an impact it plays on any given player's chance at making Playoffs and after venting about it on Discord, I started thinking and discussing about solutions, which is why I'm making this thread.

In almost any sports or other games, the way to fix this is to seed the participants. For people unaware with the concept, you can read the wikipedia page about it. I believe seeding would certainly benefit our tournament structure, especially considering the inherant variance of the game we play. I don't mean to gatekeep our tournaments by introducing seeding. The barrier of entry will stay the same, anyone can participate and make a deep run given they play well enough. But I think tournaments are meant to represent who is the best player, and while it is true that, to win the whole thing, you still gotta beat good players, this will still happen with seedings, just not as randomly as right now. It's not like we won't have plenty of upsets anyway, once again due to the nature of the game.

Before moving to the next point, I really want to reiterate how almost any sports/games working with a bracket system like we do use seeding because it's how you create more chances for the tournament to crown the best player on a consistent basis. Given the variance of Pokemon, any measure done to reward consistency without being too intrusive/being a culture shock should be considered. So if you accept seeding should be looked into, let's move on.

How to implement Seeding in Smogon Tournaments?

Now here comes maybe the heart of the problem. How should we seed players? If we look into other communities, the answer is simple. They use previous results to establish some form of ranking and then use this ranking to determine seeds. I will list some tools we could use to do our ranking and seedings. A key point to remember though, before delving into them is that no solution we can take will solve everything at once and entirely, but it will make the situation better, especially overtime, which is why they're worth exploring and taking. We shoulddn't reject ideas just cause they won't work fully on the first go.

1) Previous Tournaments results
This one is obvious. Using previous tournaments results to get a ranking on who's better/worse and seed accordingly. When this was brought up on Discord, a Tournament Director said he had issue with it because those very tournaments had random seedings themselves to begin with. While this is true, the point of seeding is that as time goes by, this issue gets resolved naturally as the better players will more often place high and everything will be sorted out.
For this proposition, I think we should certainly at least use the previous results of the tournament we're seeding (previous Classics for Classic for example), but this also could be a nice way to reward activity and results in our unofficials like our multiple championships/circuits which currently don't attract a big part of our tournament playerbase.

2) Seeding Tournaments
An idea I thought of is to organize an unofficial tournament which purpose would be to help us seed players prior to the Big tournament itself. For example we'd have a Classic Tiers Bo5 tournament and the final results could serve as data (& training) for seeding. The idea is that, the more recent datas we have on who's doing well in (a) given tier(s), the more accurate our seeding will be and the better it'll serve our purpose and the players.

3) Ladder
Using ladder as a seeding tool has a lot of interesting aspects. It rewards consistency in the tier first and foremost (you win more => you rank high => your seed is better => better chance to go far in a tournament). It rewards actively playing a tier. It rewards involving yourself in a tier. It promotes ladder activity. Basically I feel like this solution embraces all the things we aim for as Pokemon players. Getting good and consistent.
As for how this would be done, I think people should ladder under their Smogon name, and on a given day or within a declared time period, you take screenshots of the ladder (much like in OLT) and the position of each player is added to our seeding formula.
----

The way I see it, those solutions don't give unfair advantages to anyone, nor does it force players to participate in them. Look at it this way: if you try to get a higher seed, you raise your chance of not facing established good players, and as a result raise your chances of going far in a Tournament. If you don't try to get seeded, well it's just like it is right now. Basically, everyone is happy.

I want to repeat here that, just because even with a high seed you could be paired with a good unseeded player, that doesn't mean we shouldn't start implementing seeding in our tournaments. The idea is to make the situation better and better overtime. Just because it won't solve everything right away doesn't mean we have to accept doing nothing forever.

The other advantages of the aforementionned methods is that they don't put strain on the players or the hosts because they don't impact the lenghts of our tournament, which is already a "problem" in itself. Our tournaments are long enough as is, and anything ressembling a qualifying phase would only make this issue more of a logistical nightmare than it already is. On the other hands, using previous tournaments results and/or ladder for seeding will help us reward consistency, continuous involvement and activity, which is -I think- a competitive community like ours should strive for.

PS: should this be accepted, I will happily contribute to the final result if I can help in any way.

 

[Triangles]

This post just speaks to show how out of touch McMeghan is with the experience of a newer player getting into the tour scene. Imagine you are signing up for your first ever Smogon tournament after having played on the ladder for a month or so. In an unseeded tour, you can maybe get through a few rounds of close hard-fought series with other newer or decent-but-not-great players, and I can speak from being in discords with newer players that they really appreciate even a round 4 or 5 run because it feels like a small triumph and genuine progress, and they enjoy playing the games for their own sake. And the main thing is that after they are out, they want to sign up for another tour.

In a seeded tour, the new player experience is instead facing Finchinator using stall for 2 games and then wanting to quit Pokemon forever.

There's a lot of other issues with this proposal too, such how to collect 'accurate' data from a small sample size of matchups that are originally random themselves anyway, and then use that to assign a numerical value to someone statistically. And even if you figure out a way to do it, I'd imagine there is just a huge amount of man hours and difficulty that go with phsyically processing the data.

But really, the most prescient issue is how badly seeding tournaments would gatekeep new players and make their experience miserable.

 

[LuckOverSkill]

I am against seeding in tournaments.

You speak of fairness, but how is seeding fair to anyone but the top players? What makes you think some of your draws were unfair, compared to the others?

Seeding would discourage new players to join tournaments. As much as playing against better players is one of the best ways to improve at the game, it is also true that you can only lose in round 1 so many times before giving up. This is the biggest issue of the method in fact, because we need new players, and yes, you could argue that certain tournaments (like Classic) aren't aimed at newcomers, but I don't see a reason to make it even harder for them to join and get involved.

We should focus on making it easier for new players to join and have fun, not harder.

 

[Finchinator]

I understand why this idea is being proposed, but I do not think there is a perfect solution here and I believe the status quo of randomized pairings should be defaulted to. It is frustrating to see top players getting paired up against each other and lesser players getting by with seemingly easy runs to the later stages of tournaments and it is valid to have grievances about this, but I would much prefer this than to potentially suffocate newer players like the above posts outline (so I will not touch on this anymore) and to open up for a potential seeding-based slippery slope.

What I mean by this is that if we start making Classic seeded, then when do we stop? Grand Slam is another official that subscribes to the same exact format, so will we apply it to this as well? It would only make sense to me. All of a sudden, we are creating these seedings for almost all of the major tiers and seeing as we have them on hand and are updating them for the sake of official use, why not apply them to everything short of live tournaments? Now all of a sudden doing anything possible to get the highest seeding in your best tier(s) becomes an arms race and it gets out of hand. Seeding will begin to play too big of a role and it will end up with us creating a mess that we can avoid simply by keeping our current status quo in tact.

 

[Stratos]

Now all of a sudden doing anything possible to get the highest seeding in your best tier(s) becomes an arms race and it gets out of hand. Seeding will begin to play too big of a role and it will end up with us creating a mess that we can avoid simply by keeping our current status quo in tact.

This is a strange argument for two reasons.

1. Pokemon being the variance madhouse that it is, seeding is almost irrelevant by round 3. DOU seasonals already all seeded, and I don't feel any safer with the 1 seed than the 31 seed in a 128-man seasonal. Having a seed helps me avoid Ezrael in round 1, and that's basically all it does.

2. If we design the seeding structure to make sense, then why is "people competing to get a better seed" supposed to be a bad thing? Oh shit, people will play more Pokemon--that's the worst case scenario!

I think the best argument against seeding is the one that Triangles has presented. However, having hosted a DOU seasonal or two in my time, I'm not sure it's super relevant. Usually about half of the entrants are tied for absolute zero seeding points. Seeding does not come even close to guaranteeing every random gets crushed by a killer in round 1, it just reduces the odds of two killers facing r1.

If the fear is that seeding will not accurately reflect a player's skill, because low-skill players will metagame for high seeds while higher-skilled players with no motivation will get low seeds, I'm not sure that's a bad thing. For one, encouraging people to actively play the meta is a goal. If I got to choose between someone who ladders 50 DOU games per day and posts regularly winning a seasonal, versus some outsider who's a better player but doesn't contribute to the community, frankly I'd pick the first guy. But even so, that vastly overestimates the influence that seed has on your final standing.

In the (seeded) DOU invitational in 2018, nvakna tied for 16th seed with 3 other players after playing only two tours all year, won the play-in, and ended up winning the whole thing from bottom seed. He played 10th seed Croven in finals. This is a tournament where I'd expect seeding to matter MORE than usual, because it's not full of mooks in the early rounds. If you're the better player, who doesn't have many seed points because you didn't enter many tours: just win, lol. I don't see any problem with rewarding a combination of activity and skill with softer r1/2 opps.


As for how we should seed, I love the ladder points idea. I think at one time there was a "feature" to link PS / Smogon accounts labeled "coming soon!" that never happened? If we could bring that back, the bracketmaker could handle seeding automatically with a few API calls.

 

[Finchinator]

I am glad seeding is working well for DOU, Stratos. I hope it continues to serve its purpose and perhaps we can look to there for an example if need be. I just want to clarify one thing because I think you did not fully understand what I was going for in the sentence you quoted.

2. If we design the seeding structure to make sense, then why is "people competing to get a better seed" supposed to be a bad thing? Oh shit, people will play more Pokemon--that's the worst case scenario!

I do not think we will be able to create a universally agreeable seeding method; that is part of what I meant when I said there was no "perfect" solution to be found. If seeding weighs one thing heavier than another, then people will shift focus to that and so on -- or vice versa for things that may not be weighted well (don't even get me started on handpicked seeding -- this would create a whole other mess, but I trust we are above that as a community). I agree there is no downside in people participating, but we need to be very careful in what we incentivize as this is a hobby and people can only dedicate so much time/join so many tournaments, so everything we incentivize comes with a something likely being deincentivized. Also, since you brought it up, I echo'd the sentiment that Triangles (and LuckOverSkill) included in their posts in my first paragraph. I just saw no point echoing an already well-stated argument anymore. Was not trying to dismiss what they said at all.

While we are on the topic though, could you explain the seeding method DOU uses? I am actually curious to see how it looks and perhaps it could serve as a point of reference for those interested.

 

[Excal]

Regardless of what happens, I think that top players running into each other super early in classic (ie. BKC vs Asta r1 of ADV cup last year) strongly questions the competitive integrity of classic as a whole. I kind of like the idea of seeding if it's possible, as it would strongly reduce bracket luck and would give more committed players a bit more insurance and perhaps more motivation to try qualifying for this already gruesome and demanding tournament.

That said, I definitely understand not wanting to suffocate newer players, but I'm also not sure how much more suffocating it would be for them, and perhaps seeding's benefits outweigh the cons. In terms of how many people enter classic, "top players" make up a smaller fraction of the entrants, meaning newer players are still decently likely to encounter opponents that they can realistically beat. While the top players on smogon are incredible, I don't think competitive pokemon is played anywhere close to as high of a level as other competitive games like Super Smash Bros Melee, wherein every tournament is seeded. I agree with McMeghan's point that we'd still see plenty of upsets with seeding due to the nature of our game.

I think seeding vs randomizing depends on how much work we'd be willing to put into seeding (because I assume it would be a tremendous amount), and how much we value inclusivity within the community compared to how competitive we want our tournaments to be. It seems like so much of smogon policy surrounds making the game as competitive as possible (ie. tiering policy) while with tournaments we seem to consider how much fun a new player would have in their first tournament running into an established player stalling them more often than they normally would. I think in this way there's a lot of contradiction.

I think newer players are inevitably gatekept anyway, and when I was a newer player I got absolutely stomped regardless. Bad Ass's seeded tournament last year was awesome imo, even though I was an unseeded random and got destroyed round 1. I cannot speak for the policy regarding seeding and I'm not trying to imply that I'm strongly for seeded tournaments over everything else, but I think that classic's current format is quite terrible and if seeding were less controversial and easier to implement given our resources, then I would be for it over what we have right now.

 

[pasy_g]

If we care about competitive tournament integrity, this is indeed a very good thing to talk about. And PLEASE, PLEASE, dont just stamp this off as "lmao noo stupid idea we cant do that, wont happen."

Seeding players one by one is imo indeed very unrealistic, but that is no the only way how we can seed. Let's say we have a tour with 512 players, now we seed out like 30? 40? players, based on ladder requirements (like suspect tests), previous tournament results, spl starters, circuit points and what not. If we arrange it so that those players dont play each other in early rounds and dont even have 10% of the playerpool here we could already drastically improve the chances for the good players to not steal each other points and give more realistic playoff chances for the good players. With that method you absolutely dont line up the "noobs" at a wall to get shot down by BKC, Ojama and Soulwind. They have pretty much the same odds to get away with worse opponents than the absolute topplayers, and can still make r4/r5 runs Triangles (or some statistic genius calc that out for me and tell me if the chances are way off?) Maybe i am wrong).

Further we also increase the general level of play since we endorse playing the game outside of the official tournaments, where some people enter with having free access to the newest teams from whereever they got them from. This also helps the newer players to stick to the game (a deep run in a DPP seasonal feels pretty good too), ladder games gain a lot of quality and actually help people to find quality games, so they can improve and not only play at a high level in tournaments, because they dont have the right connections to get testgames vs good players. Sounds like a better argument for me to help the new players?

Overall, a simple implementation of a seeded group and the ways to get into this seeded group can be very beneficial for every aspect of the completitive playing scene and this is a good chance to talk about this, think about it, dont just disapprove for the sake of it. Thanks for reading!

 

[lydian]

Ok so I'm not a tournament player at all, I don't have much to base my opinion off of, other than personal experience and opinion but I still feel like I want to weight in on this matter. So please bear with me and my 0.02$.

To seed or not to seed?

So, I definitely feel like there's a problem with the lack of seeding in most cases. In theory, seeding is not needed in a tournament where only the first place gets rewarded (say OST but without the prize money), as in theory the best player would beat anyone one-on-one and go on to win the whole thing, obviously that's not how Pokémon works since we have hax and matchup to account for, but I don't see the lack of seeding as much of a problem in standalone tours.

The problem arises when you take the other results into account, be it to qualify for Classic Playoffs, the Grand Slam Playoffs, or any circuit there is. Because, with no seeding, the in-theory second-best player could get knocked out as soon as the first round should they go up against the potentially best player, when they could have had a much better result, going ideally as far as the finals, had they matched up with anyone else in the first round. This way, you're not giving an accurate ranking of the players in your player poll as a qualifying tour should do.

Obviously, there are a ton of issues with seeding, and those mostly come up with the method of seeding and how it's done, but not exactly the purpose of seeding itself. I'm not a huge fan of the gatekeeping argument, as in general, competitive Pokémon has a very steep learning curve, as I wouldn't say winning against Charizardfan69 and Umbreonlover23 in rounds 1 and 2 only to lose in Round 3 to an actual good player would really motivate too much on playing ADV or something. I feel like Pokémon intrinsically has a really steep learning curve and seeding doesn't alter it very much. Especially since, as Stratos said, there are about as much unseeded players as seeded players, so if I'm fresh on ADV OU and don't have a seed that doesn't I'll immediately play Finchinator or whatever Round 1. I'll elaborate on this point later in the next section. And I don't really like the point that it only benefits top players because well, that's exactly the purpose of seeding, if a tournament is used as a metric to determine the best players of a certain tier for them to qualify to playoffs of something, then yeah, you do want the better players to qualify.

How to seed?

Thinking about it, to me, the most feasible way to do this would be to implement a bit similar to that of tennis, I'll quote Wikipedia here, so we're all on the same page:

In a tennis event, one version of seeding is where brackets are set up so that the quarterfinal pairings (barring any upsets) would be the 1 seed vs. the 8 seed, 2 vs. 7, 3 vs. 6 and 4 vs. 5; however, this is not the procedure that is followed in most tennis tournaments, where the 1 and 2 seeds are placed in separate brackets, but then the 3 and 4 seeds are assigned to their brackets randomly, and so too are seeds 5 through 8, and so on. This may result in some brackets consisting of stronger players than other brackets, and since only the top 32 players are seeded in Tennis Grand Slam tournaments, it is conceivable that the 33rd best player in a 128-player field could end up playing the top seed in the first round. [...] Rankings of tennis players, based on a history of performance, tend to change positions gradually, and so a more "equitable" method of determining the pairings might result in many of the same head-to-head match-ups being repeated in successive tournaments.

I also feel like we should implement a method a bit inspired by this, seeding every single participant would be a huge hassle and wouldn't bring much usefulness out of it, so I think we should just focus on avoiding to make the stronger players face each other in the early rounds, something akin to what Tennis does here, so imagining ADV Cup had 32 participants for the sake of simplicity and for this example, every round could be randomly generated just following these procedures:

• In Round 1, divide the players into groups of higher and lower seeds, 1-16, and 17-32, and don't allow players of the same seed group to play each other. So if I'm one of the best players I won't be paired of someone of similar skill level, but rather someone of lower-level allowing me to progress further into the tournament, something that wouldn't happen if I faced off against the 1st seed or the 2nd seed, for example.
• For Round 2, do the same, players of seed 1-8 can only play players of seed 9-16.
• For Round 3, players of seed 1-4 can only play players of seed 5-8.
• For Round 4, players of seed 1 or 2 can only play seeds 3 or 4.
• And Round 5 the remaining finalists obviously play one another.

This solves the problem of strong players facing off versus one another early on, and also make it so the very weaker player isn't always guaranteed the strongest player and has the freedom to advance to the next round. And of course, means with a low skill level can't get too far in by just having some lucky pairings. I honestly feel like this is the best approach and still retains some level of randomness to avoid being exploited. I don't really see any problem something like this would bring, but if anyone wants to provide a different perspective please do so!!

The only problem I see myself is substitutes, as an "in as a sub" could come from either a newer player or someone really good that just didn't join before for some reason, so giving a bye to the higher seeded player could jeopardize them if a good player signed up, and in general, I don't think higher seeds are entitled to byes, so personally I would just randomize the players that get byes separately, and seed the pairings after that. That's just a workaround I thought of in 5 minutes, though, and I'd love to know how exactly DOU handles this issue.

And even if this is too harsh, we can just go with seeding a quarter of the users every round, instead of half, so instead of seeds 1-16 can't play each other in a Round of 32, we can just make seeds 1-8 not be able to be paired against each other, then 1-4 for the Round of 16, and so on... This would be almost identical to the normal random format, but just avoids the strongest players meeting each other in the same round that Charizardfan69 and Umbreonlover23 play one another.

Ok, but where do we get the seeds from?

And now, I feel like this is the main problem with the idea of seeding. Going off of the previous tournaments is probably the best we can do, but the main problem is that the latest edition of these tournaments was not seeded to begin with and, in general, Pokémon is a game full of variance and matchup involved, so going only off of one previous edition of the tournament doesn't make much sense to me. As a good player can get haxxed in like Round 2, and now they have a really shitty seed. Using several editions of the tournament in the past is also not ideal as metagames evolve very fast, and a player that was really good in one year can be crust now, so imagine that with three or four years instead.

Ideally, you might want to pull unified seeds for a certain tier, and include all official tournaments / circuit tournaments in it of the past years, but then that kind of forces a player to sign up for OU Autumn Seasonal if they want to have a shot in OST which feels really backwards to me as well. Ideally, we could make seeding have a Best Finish Limit format, so that your best—say 2 or 3—results in OU tours in the past year would count towards your seeds, and the others would not, and that also creates a bit of a problem since smaller tours will obviously be worth fewer points. On average, your OST Quarterfinalist is probably better than your average OU Autumn Seasonal winner. We also run into a bit of a problem with team tours should they count for seeding, as one could be in a really big handicap if they didn't get drafted for a team tour like SPL or Snake Draft, because of—say—poor attitude / chat presence, which fundamentally has nothing to do with your Pokémon skills.

The idea of doing a prior tournament is just quite a waste of time in my eyes, especially if it's not seeded either, then the "BKC vs Asta in Round 1" thing could happen again, and suddenly either of them has a really shitty seeding for no reason. And if these tournaments ARE seeded, then why not just use this very seeding for the actual real tournament? Either way it's a bit of a waste of time in my eyes.

I don't dig the idea of using ladder either, as the ladder and the tournament scenes are starkly different as anyone can notice. Tournament has much more preparation involved, especially in a higher level, and again, forcing players to ladder to have a shot in OST also feels completely backwards to me. Last but not least this is a personal point of mine, but I assume a lot of people can resonate with it, as someone with ADD (attention deficit disorder), playing multiple games on ladder is NOT easy at all, so maybe a good player that doesn't fare well in the quantity > quality format of ladder could be severely handicapped if ladder was used for seeding.

In my opinion, this is the main problem that lies with the idea of seeding in Pokémon, fundamentally it works, but the main problem is where to get the seeds from. If someone can come up with a fair way to seed people, then I'd be all in for it, but because of how tricky something like this would be, I'd say I'm a bit on the fence about it. I'm happy this thread is seeing discussion again though, and I hope my lengthy post (sorry) has contributed at least a bit to the discussion!

 

[Texas Cloverleaf]

Any competitive integrity argument is pure arrogance. You're playing a tournament, you have no inherent right to have an easier matchup and no tournament is inherently better or worse because someone "deserving" did or didn't make it. You play a variance game, get over it.

 

[Bughouse]

Seeding makes sense when you have a fair base to perform seeding with.

No one objects to VGC seeding the top cut based on Swiss results.
No one objects to the Tour, Slam, or Classic top cuts being seeded based on results in the individual tours.
No one objects to seeding a Doubles Seasonal based on Doubles circuit results.
These are all good examples of being able to seed using reasonably fair predictors of how someone will perform in the future based on relevant results in the immediate past.

However, I think seeding the individual tours for Classic or Slam may be substantially different from seeding for example a Doubles circuit tour based on points within the Doubles circuit. For the Doubles circuit example, the user bases between these individual tours throughout the year has a relatively high rate of overlap. You actually can suss out with some degree of accuracy who is better than whom with historical results among a narrow group of people to go off of. I'm not sure that this is the case for Slam or Classic.
Many people who do not play the (insert tier here, going with UU as an example) UU circuit at all throughout the year sign up for the UU Slam as their only large UU tour of the year.
Last UU slam was a 768 man tour (technically 724 + byes). The most recent UU circuit points had only 226 players with any points on it. If that was the basis of seeding Slam, only 30% of the players could be seeded, and that max assumes everyone who was on the UU sheet also signed up for Classic. I actually did do that comparison, since what else am I gonna do during coronavirus lockdown... in reality around ~10% of the people on the UU sheet didn't also sign up for the UU Slam. One example is Earthworm, who is only on the UU sheet because of his participation in UU Classic, and wanting to play GSC UU (and skill at it) don't have much to do with playing SS UU.

That's just barely comparable with what the grand slam tennis tournaments do, which is 32 seeds out of 128 entrants or 25% seeded, 75% randomized. And at least all of those 128 players are full time tennis professionals, and being a tennis professional means playing the same exact game as everyone else. In our case the other 70% that would be unseeded ranges from 1 post signup wonders to someone like McMeghan casually playing LC for the first time all year, but is getting fed teams by a friend, and some of the seeded players might have never even played the current gen if their only participation in the circuit came from a Classic.

In my opinion, if we can't seed these tours fairly, there's no benefit to seeding, and substantial cost.

 

[McMeghan]

I think Stratos/Excal/Pasy explained why seeding isnt a way to gatekeep new players. The proportion of strong players/new players in tournaments is so skewed toward the latter group that the vast majority of the new players wouldn't get the top players and get executed in the first round.

There are such subjective statements here like "new players actually like facing other noobs so they can get by in the early rounds" when I could very well just answer that new players like facing big fishes and making a name for themselves by having a good game against them, either for their own experience or for their reputation. Lol@calling me out of touch too, you do realize that maybe I play and follow other games competitively? Where I see the benefits of seeding for their overall structure ? I also join other games tournaments and there are so many other "low lever players" like myself there that you play people around your level much more often than the top player who's going to stomp you. On top of that, there are so many avenues on Smogon for new players to play in, seeding would just help make (some of) our official tournaments more representative of the winners' skills.

This isn't about top player arrogance either, it's about having our tournaments reward consistency and painting a better picture at who's better at the game, but based on the responses here and on Discord it seems a large part of the community/staff prefer to embrace the randomness & easiness of our current structure rather than trying to establish some sense of consistency reward.

 

[Stratos]

While we are on the topic though, could you explain the seeding method DOU uses? I am actually curious to see how it looks and perhaps it could serve as a point of reference for those interested.

We seed all our seasonals based on circuit points so far this year (first tour of the year is seeded based on last year's circuit). It works for our uses, but I think in the general case, there's a better idea, which I'll detail below.

Any competitive integrity argument is pure arrogance. You're playing a tournament, you have no inherent right to have an easier matchup and no tournament is inherently better or worse because someone "deserving" did or didn't make it. You play a variance game, get over it.

Seeding reduces the impact of bracket luck on a tournament's final standings.
Seeding encourages increased participation in the competitive scene.
Seeding backweights highlight matches to the later rounds of a tournament, which have higher stakes, making a better spectator sport.
All of these seem like admirable goals to me.

In my opinion, if we can't seed these tours fairly, there's no benefit to seeding, and substantial cost.

I'm confused. What's the cost? So even if you have a seed, you might pull McMeghan r1. That's true when unseeded to an even greater extent. If there are 50 killers in the pool of UU open, and 20 of them have seeds, I think we have strictly improved from the scenario where 0 of them have seeds. Thank you for providing some numbers to the argument that "new players are not getting screwed by seeding" though.

If the cost is to host time, let me outline my preferred method of seeding, and how I think it solves that and many other issues: ladder-based seeding. The general idea here is something like follows:

1) Players link their Smogon account to (maybe multiple) PS alts on their profile (this would require authentication on each end).
2) When using the Smogon Bracketmaker, it automatically pulls the Elo scores for the relevant format for each player and picks the highest one. Anyone without an Elo is 1000, of course.
3) Players are grouped into power-of-4 sized "seed tiers" based on their Elo: top 4, top 16, top 64, top 256 etc. Seeds within each tier are randomized.
4) The bracket is made using these seeds.

There are a number of advantages this proposal has:

It requires no manual work from the host. Manually seeding a 128-man seasonal bracket has taken me about half an hour in the past, that obviously balloons to multiple hours for a massive tour like UU open. This is all automated after a bit of programming on the front end. I'd even be willing to do the programming work to get this up and running myself, though I'm not sure if access is limited to admins or something.

It doesn't require much work from the player to secure themselves an accurate seed. Seeding based on previous tours means you are guaranteed to have a low seed on your first tour. If you seed based on ladder, anyone can have a high seed. On the flipside, for experienced players, this isn't something they have to grind out all year to keep a good seed. After they enter a tour, they have a week to play some ladder games and get a high seed.

It encourages ladder participation. I think it's self evident that a stronger ladder is better for the community; it's a great place to test new teams, it's where new players go to learn the game, and it's the only place to find competition 24/7. Encouraging ladder participation seems like a blanket win to me.

It limits the advantage of "jockeying for seed." To address Finch's earlier point, one of the main tools that seeded tournaments have to combat this is the aforementioned seed tiers. There's no advantage between 1st and 4th, 5th and 16th, 17th and 64th. You can't guarantee yourself a specific matchup r1, and the marginal gain of grinding out seed points is minimal (except at a cutoff). This also benefits players who don't have a seed, because 1/2 to 3/4 of the tour is guaranteed to be in the lowest seed tier, so you arent going to get matched against first seed because you are last seed.

Of course it's not perfect:
Players might be discouraged from laddering on their highest alt right before a bracket is made, but that's literally day of / two days of the tour starting, maybe a few times a year. Don't think this is a huge deal.
Allowing people to affect their seed right up to a cutoff advantages people who are free right before that cutoff. If a bracket is made at say 10 PM EST, Europeans who have been asleep for hours might get their seed bumped and there's nothing they can do about it. This is the biggest problem I've thought of with the system, but as I've said above, the impact of a low seed is kept to a minimum, so this shouldn't be a huge deal.

However, I strongly believe the positives outweigh the negatives, and I can't think of a better seeding system at this time.

 

[Boat]

I'm undecided on whether seeding should be implemented (except randomized seeding for double elimination), but I do want to offer thoughts on how it could be most effectively implemented if it was. One of the many concerns brought up in this thread has been that if you seed tours, you are lining up new players to be destroyed by the top players. I think that an effective resolution to this problem would be to only seed a certain number of players, say 10%/15% of the bracket. That is well beyond the percentage of top players in a bracket, but it ensures that new players still have a fair shot. It still accomplishes the same goal (ensure top players don't fight in early rounds), but is way more fair to the new people. I do have concerns about how this will be implemented in the bracketmaker; to make this work would require either serious changes to the bracketmaker or to use the system outlined in my post here. To quote, "Rather than using the bracketmaker for each round, a host would use it for R1 pairings, and use those pairings to assign seeds for the rest of the tour. From there, continue like a standard bracket (highest seed plays lowest seed, second highest plays second lowest, etc). There is no room for host manipulation either, because it would be clear if someone was assigned a seed they shouldn’t have gotten from the random pairings.". This would work, but would require some creative problem solving since some seeds are guaranteed and some are random.