CAP and learning about the metagame

[DougJustDoug]

Ginganinja sent me the following question via PM:

Anyway I was reading the CAP Mission Statement and I was wondering when CAP will return to its "roots" so to speak and make CAPs to actually learn about the OU metagame. Its something that gets thrown around a lot, but after asking CAP newbies, CAP veterans, competitive OU players involved in CAP, competitive OU players outside of CAP, the amount of people that actually learn about the OU metagame based on our CAPs is actually pretty small.

Instead of answering via PM, I would like to respond to this in a bit of detail here in the main forum. I think it's a good idea to revisit some of these "core values" type of questions from time to time in CAP. If anything, to help keep everyone on the same page.

The short answer is -- Yes, CAP is, and always has been, about learning about the metagame. So we never really departed from our "roots", as Ginganinja mentioned. However, as longtime CAP diehards know, some CAP projects are better than others in this regard. Looking at some recent CAP projects, I agree with many people's subjective assessment that the learning aspect of CAP has been suffering a bit lately, for whatever reasons.

That is not a knock on the leadership of recent projects, and it doesn't mean recent projects have not been engaging and enjoyable for the community. There are many ways to define a "successful" CAP, and we've had plenty of success on recent CAPs. But when it comes to learning, I use this simple definition of success:

A good CAP project exposes fresh and interesting aspects of the metagame, battling, and Pokemon in general.​

We have tried to structure the CAP process and rules to encourage this to happen, but we can't guarantee it will happen, and sometimes when it does happen, we go too far with it. That's the tricky part of the whole learning aspect of CAP -- it requires us to strike a precarious balance between doing "Too Little" and "Too Much".

Novelty and Experimentation

The CAP community likes "new stuff", and we tend to put a premium on novelty. For example, that is a big reason CAP typing polls overwhelmingly favor unique or nearly-unique typing combinations. Nobody wants to make yet another Normal pokemon (never in CAP history, actually), since Gamefreak pretty much has that covered in the actual game. Sure, we could do our own Normal mon, and maybe we will in the future -- but we'll have a hard time making something that stands out, since there are so many varied Normal types already.

CAP's love of novelty is not restricted to Typing. CAP loves Abilities that are underrepresented, Moves with low distribution, and even wonky stat spreads that have little or no analog in the actual game. We've had to make all sorts of rules on almost every step of the process to prevent the community from making extreme choices, such that our creations would not adhere to the general precedent and example of the actual game.

Hand-in-hand with our love of novelty is CAP's desire to experiment with things. For example, we often want to see how certain mechanics would play in OU, if given the proper surrounding Pokemon build to support it. Or maybe we want to see if a combination of mechanics could become a viable battle strategy. Sometimes we strive to impact the usage of certain pokemon or playstyles by making a pokemon that supports or counters other pokemon and playstyles.

But as much as we like to experiment with new things, there is also the opposite tendency to stick with the status quo. Resistance to change is as commonplace here in CAP as it is everywhere else in life. People tend to revert to tried-and-true solutions to problems, rather than thinking outside of the box. No one ever does this intentionally or explicitly, but the gravitational pull of the familiar is often hard to notice and even harder to resist.

On every CAP project there is a tug-of-war between the opposing tendencies for "too much" and "too little". Ideally, we end up with a balance -- but recently we seem to keep ending up on one extreme or the other. And both extremes typically result in little meaningful learning on the project.

Too little Novelty and/or Experimentation = We make a CAP that doesn't cover much new ground. Maybe it starts out looking like a fresh new thing. But a couple of polls into the process, we find ourselves with a creation that looks like something CAP has already done before. Or worse, the CAP starts looking a lot like something that already exists in the metagame. People get bored, participation wanes, and discussions get stale because the answers to most questions are obvious to most intelligent participants.

Too much Novelty and/or Experimentation = We make something so bizarre that people don't understand how it relates or applies to real metagame play. Too many new things thrown together, or a few new things taken to an extreme, makes it hard for CAP participants to understand and predict the competitive implications. This leads to disjointed decisions every step of the way, because everyone is following a different drumbeat. Scientists are great; Mad Scientists are dangerous.
​

Concept and Leadership

Looking back on all the past CAP projects, I would say Concept and Leadership are the two biggest factors that determine how much we learn on a creation project. The community at large is fairly constant, and, as I mentioned above, typically has warring tendencies on all sides. So it comes down to Concept and Leadership to tip the scales, positively or negatively.

The selected Concept is, of course, vital to almost everything that plays out on a given CAP. That's the big reason we have recently completely restructured how we create and develop Concepts. Creating and choosing good Concepts makes everything better on CAP. Conversely, a bad Concept makes everything harder. The problem is that every chosen Concept looks great at the outset of a project, that's why they win in the Concept Poll! The problems with bad Concepts don't become apparent until later in the process, sometimes too late for us to do anything to right the ship.

Technically, all Concepts are supposed to encourage us to learn about the metagame. The format of all Concepts requires the author to pose questions to be answered at the conclusion of the CAP. Recent CAP projects have not formally answered those questions, and perhaps we need to put those questions more at the forefront of our process. But whether the questions are formally answered or not, ALL concepts are designed with learning in mind to some degree. Of course, some Concepts are more conducive to learning than others. And if we as a community want to focus more on learning, we need to author and choose more Concepts with a better learning bias.

Leadership is the great equalizer for Concepts -- both good and bad. Leadership can make bad concepts manageable, or even wildly successful. Leadership can also start with a great concept and end up with a project in ruins.

When I refer to "Leadership", I am not just talking about the Topic Leadership Team or CAP Moderators. I am also referring to the Policy Review Committee, CAP veterans, respected battlers, talented submitters, and intelligent persuasive discussion participants. Everyone who takes initiative to actively shape CAP policy and creation projects -- you are a "Leader" in CAP. And you need to own the responsibility that comes with that.

I wrote a very long compendium on CAP Leadership a while back, and it is stickied in the CAP Policy Review subforum. I won't rehash the points in that compendium here. Anyone that has read this post this far, obviously has some interest (and stamina) in CAP leadership, so you probably should check it out, if you haven't already.

Further Discussion

This thread is not a Policy Review, it was opened in response to a question. I see this more as a collective survey on the current state of the Create-A-Pokemon project, as it pertains to learning about the metagame. I have presented my opinions on the foundations that encourage and/or discourage learning on CAP projects. What do you agree or disagree with? What did I miss? What did I not emphasize enough?

I'd like to hear more opinions on the topic, and I think others would too. Dare I say, we all might learn something here...?

 

[ginganinja]

Actually, I was wondering if we could take greater stock in the questions every concept has listed within it in its submission. Generally, we have a concept, and quite a lot of people (myself included), immediately zero in on the concept itself in the following concept discussion thread, while the actual questions just get shoved into the background. It would be nice, I think, if at the end of a CAP, or near the end, we look at the finished product and see if we can actually conclusively answer some of the main questions that were initially asked at the start of the process, since most "Final Product" threads tend to look at how the created CAP relates to the concept itself, not how well it actively seeks to answer our initial discussion questions.

 

[FireArrow]

^This so much. Changing our main goal to answer the questions rather than manufacture a shiny new CAP is probably the biggest step CAP can make to gear its discussion in the right direction.

 

[jas61292]

I want to start of my response here with a quote. I posted the following as part of my opening post in the CAP16 Concept Submission thread. It is specifically about what kind of concepts I see as good, with regard to the learning aspect of CAP. While obviously this was written with a specific project in mind, rather than the general case like we are discussing here, I think it is still very relevant to what this thread is about:

Probably the most important thing to me in a concept is that it clearly has goals in mind with regard to learning something about the game of competitive Pokémon. In the past we have done projects with concepts that were more about taking certain things, be they moves, abilities, typing, or playstyle, utilizing them, often in a new way, and seeing what happens. While concepts like these can certainly teach us things, we really have no idea what going in. This puts us much more at the mercy of the result than we otherwise would be. Sure, they can have interesting discussions along the way, but often times these discussion are simply about how to fit the Pokémon in OU and not anything specific to the concept.

What I’d like to see are concepts that focus clearly on learning goal. This could involve interacting with a current metagame force, trying to learn about a certain concept in competitive Pokémon that we have never thoroughly explored, or possibly something completely different. There are infinitely many ways to go about this, and I’m sure you all can think of many more than me.

That being said, the most important part of the concept submission to me is the questions section. I want to see concepts that know what they are setting out to learn. Simply saying “How will doing X effect Y?” is not going to cut it. Additionally, I would like to see concepts that have such learning goals for both the process discussions and for the actual product. While the bulk of what we get out of a CAP project comes from the great discussions and debates that we have, I consider the playtest an integral part of the process, and I fully expect that going into it there will still be more to learn.

As is pretty obvious here, I think the questions of a concept are pretty damn important, and I think, as ginganinja suggested, that it would be very wise to put greater stock into these. The questions, to me, are the part of the concept that best shows what it has to offer when it comes to the actual discussions, yet, once the concept is chosen, more often than not, we simply ignore them. If sometime well into a project (but not necessarily at the end) we had a stage where we stepped back and discussed the questions themselves and not just the impact on the specific Pokemon we are making, I think the average participant would get a lot more out of the project. At the same time though, these questions are, ideally, something that should be being looked at throughout the entire project. If we are failing to address the specifics as they apply to the current project, then I feel that we are failing to really follow through with the concept in the first place.

What I personally would like to see would be two different things. One would be, like ginga suggested, having a time later in a CAP where we look back and discuss the questions head on, and how we applied them to our project. I think this would probably be best to do, not necessarily right at the end, but possibly something that opens up in the later parts (maybe around the same time as movepool or something) and stays open through the end, as that would allow us to not only discuss what answers we have found, if any, but also to realign for the end of the project what we still really need to focus on. The second thing I would like to see would be simply a shift in priorities from our Topic Leaders. While the TLT could certainly help on this, I see this kind of issue right here being exactly what the TL position is designed to take on. I believe that our Topic Leaders really should be focusing more on these questions. The TLs main job is to help lead discussions, and I believe that one of the best ways they can do that is by using the questions provided in the concept as a starting point. If we can make people discuss the actual important issues that concept is trying to address, and not simply how to make it function in OU, I think we stand to have much more engaging and informative discussions. With that said, this is not something that needs to be left to the TL alone. While I believe it should very much be a part of their job description, every single person who is a community leader in one way or another here (as Doug described) should probably be placing this kind of thing much higher up on their priority list than they have in the past.

To put this more simply I think this project very much can have a great emphasis on learning in the competitive Pokemon environment, however, in order to do so, we need to make sure that the project leaders have that fact foremost in their minds. We are the people who are leading discussion, so if we make sure that the entire project, from concepts submissions through each and every discussion step, has a focus not just on the result, but on the what we want to learn, we will be more likely to actually get there.