Policy Review Concept Submissions

Birkal

Birkal

We have the technology.
is a Top CAP Contributoris a Site Content Manager Alumnusis a Battle Simulator Admin Alumnusis a Top Artist Alumnusis a Senior Staff Member Alumnusis a Top Community Contributor Alumnusis a Metagame Resource Contributor Alumnusis a Top Smogon Media Contributor Alumnus
Hello, Policy Reviewers! I'm here to present what I consider to be the root of numerous problems within the CAP process: Concept Submissions. As someone who has observed this stage keenly for the past seven projects, this is the stage that influences any given CAP the most. Which can be a real tragedy, considering how hit and miss our concepts can be. Let's look at some examples:

I'd argue that the concepts that are inherently the most popular are the ones that allow voters to dream and consequentially project their individual vision onto the entire project. Quanyails' "Show Me Your Moves!" encouraged the community to dream up its favorite wonky moves and create a Pokemon based on that. Yilx's "Einherjar" presented an interesting dilemma that was sort of two concepts jammed into one. Dummy007's "Major Third" also encouraged voters to dream up two Pokemon to make a pair. The issue with these concepts, in my opinion, is once we decide on a trajectory, the rest of the project can be a bore. If you don't get what you envisioned at the concept submission stage, it can be difficult getting engaged in that particular process.

Vague concepts can go on to plague an entire process. "How does this relate to our concept?" and "What are we trying to do again?" can become commonplace in conversations. While confusion will always be a part of a process this big, I can't help but worry about the levels of confusion, especially how they can lead to frustration. Anyone can submit a concept, which fits in with our democratic nature. That being said, do we want any individual to be able to walk in and essentially define what we do for the next three to five months? Particularly with no guidance?


There are numerous ways to address this issue, but the one I've grown most fond of is Concept Blueprints.

Proposal:
We add a subforum for quality control of all concepts, similar to C&C. In this forum, anyone can submit a concept idea at any time. Anyone is allowed to comment and critique the concept as they see fit. After discussion, a Quality Control team of CAP veterans cleans up the concept. Things they look for could include learning goals, proper questions, and practicality within the CAP Process. Once approved, that concept could be slated in any future project by the Topic Leader for Concept Polls. Finally, these discussions would also serve as our Concept Assessment stage. The Topic Leader would look at the blueprints set out by the QC team and move forward as the concept dictated.
​
(This conversation dates back to mid-August, for those wondering.)

[2014-08-18 16:03:57] <DougJustDoug> I really think we need to take a hard look at how we handle Concepts. I think our policies for Concepts needs to be completely reworked.
[2014-08-18 16:15:35] <DougJustDoug> We need much better Concepts in CAP.
[2014-08-18 16:15:50] <DougJustDoug> And not because Concept authors suck or anything
[2014-08-18 16:16:59] <DougJustDoug> The more I think about it, the more I realize that Concepts need a LOT more vetting and assessment before we launch a project based on them. TOO MUCH vetting and assessment for us to do it right at the start of a project.
[2014-08-18 16:17:29] <DougJustDoug> I'm thinking Concept Submission and Concept Assessement should be evergreen sub-projects within CAP.
[2014-08-18 16:18:00] <DougJustDoug> Basically, treat Concepts kind of like C&C analysis (kind of).
[2014-08-18 16:18:24] <DougJustDoug> Submit a Concept, have it validated by some Concept Validation Team.
[2014-08-18 16:19:03] <DougJustDoug> Then valid concepts would go through some team process to flesh out into a full-blown "Project Skeleton".
[2014-08-18 16:19:31] <DougJustDoug> Any fully completed Project Skeleton is eligible for the Concept Poll at the beginning of a CAP
[2014-08-18 16:19:54] <Birkal> you know me, Doug
[2014-08-18 16:19:59] <Birkal> concepts are easily my favorite part of CAP
[2014-08-18 16:20:05] <Birkal> so this all sounds like heaven to me, lol
[2014-08-18 16:20:07] <DougJustDoug> Unused Concepts/Project Skeletons stay eligible for the next CAP.
[2014-08-18 16:21:21] <DougJustDoug> Basically, we stop reinventing the wheel every CAP. Invest in Concept Assessments by people that are into Concepts, just like C&C is done by people that get off on that kind of team creation process.
[2014-08-18 16:22:25] <DougJustDoug> And that way we can throw out shitty Concepts without running afoul of the normal "everyone can submit" philosophy for CAP polls.
[2014-08-18 16:22:42] <DougJustDoug> But we don't leave Concept up to the whims of the masses every CAP
[2014-08-18 16:23:37] <DougJustDoug> We guarantee that any selected Concept has been thought about, assessed, and fleshed out by a bunch of people that know what it takes to have a Concept sufficient for framing a three-month CAP community project.
[2014-08-18 16:23:41] <Birkal> I wouldn't mind if we chose leadership based on leaders who took interest in particular concepts, even
[2014-08-18 16:26:17] <DougJustDoug> I envision we would make a Concept Workshop subforum in CAP, and we do all the C&C work there.
[2014-08-18 16:26:44] <DougJustDoug> One thread for submitting "Concept Abstracts".
[2014-08-18 16:26:52] <DougJustDoug> Open to almost anyone
[2014-08-18 16:27:28] <Birkal> I'd agree with that, yah
[2014-08-18 16:27:40] <Birkal> and ecstatic to head it up too
[2014-08-18 16:28:06] <Birkal> then we vote on C&C'd concepts
[2014-08-18 16:28:15] <DougJustDoug> Those Abstracts will be vetted and approved for becoming full blown "Concept Threads" -- which is where everyone turns it into a full Project Skeleton. Or, perhaps gets closed, if we realize it just isn't viable after further thought.
[2014-08-18 16:28:16] <Birkal> and make a CAP based on the blueprints they left behind
[2014-08-18 16:28:25] <DougJustDoug> Which is what we CAN'T do now in CAP
[2014-08-18 16:28:45] <DougJustDoug> WE get stuck with Concepts that kinda "dissolve" upon further assessment.
[2014-08-18 16:29:34] <DougJustDoug> Concept Blueprint is a better term than Project Skeleton. I like it.
[2014-08-18 16:29:50] <Birkal> btw, I think this would fit well into giving CAP more creative flexibility
[2014-08-18 16:29:57] <Birkal> moving into banned abilities / further topics / etc
[2014-08-18 16:30:10] <Birkal> anything that we've banned due to "screwing with the process" could easily be fixed with a C&C'd blueprint
[2014-08-18 16:30:14] <DougJustDoug> Absolutely.
[2014-08-18 16:30:47] <DougJustDoug> When we vote on Concept, the blueprint would line up exactly how the steps should be handled.
[2014-08-18 16:31:06] <DougJustDoug> If the process order needs to be tailored a bit, so be it.
[2014-08-18 16:31:06] <Birkal> with dates / timelines / key questions / etc
[2014-08-18 16:32:04] <DougJustDoug> The concept C&C team would have people that "sign off" of areas where they are experts.
[2014-08-18 16:32:20] <DougJustDoug> Battle folks would sign off on "competitive relevance"
[2014-08-18 16:33:02] <DougJustDoug> Process guys could sign off on it having "sufficient community choice options".
[2014-08-18 16:33:05] <DougJustDoug> Stuff like that
[2014-08-18 16:33:18] <DougJustDoug> The semi-equivalent of GP Checking in C&C
[2014-08-18 16:33:28] <Birkal> yep
[2014-08-18 16:33:44] <Birkal> I have no doubt that something of that ilk would work wonders
[2014-08-18 16:42:28] <DougJustDoug> The concept thing I have been ruminating on for several days. Ever since the Concept Assessmen this CAP
[2014-08-18 16:43:00] <Birkal> again, I'd be elated if we added in Concept Blueprints
[2014-08-18 16:43:17] <Birkal> and I think it could be pretty popular
[2014-08-18 16:44:16] <DougJustDoug> Keep it in the back of your mind, and we'll pick this up later as this project starts winding down.

I believe there are many advantages to a system like this. First, it allows for greater voice of competitive players. In a C&C style forum for discussion, anyone has a voice. But being a Quality Control member holds more weight -- you can mandate changes and take part in higher level discussion. If you're not on QC team, but you're a competitive player, you're much more likely to be heard by a seasoned QC member than a random you're trying to argue with in a process thread. Second, it allows for greater project flexibility. Have a great concept that mandates an ability? We can outline that in the blueprint. Need to require a specific grouping of moves? That can be blueprinted as well. As long as submitted ideas have great learning goals and hold merit for the process, we can change up the process by removing or adding stages. Finally, it allows for greater quality in every process. By being checked by a QC team, we have a much greater chance of pulling up concepts that hold merit throughout the entire process.

For what is arguably the most important part of CAP, our current concept submission stage is surprisingly lax. Anyone can come in with an idea they find intriguing and submit it, no filter. While we shouldn't remove our accessibility, we can increase quality in a non-intrusive way. In fact, I'd argue that making CAP Blueprints would be a lot of fun. It's a way to get creative juices flowing while setting useful standards for the CAP Project. Should we implement something like this? If not, what can be done about fixing the current state of concepts? Do concepts need to be fixed? Let's hear your thoughts.
 
Last edited:
They certainly do, Birkal. Similar to the dilemmas brought forward in the Retention Issues thread, it's clear that the voters have not been fully aware of what they've been voting for, and how likes have swayed the ultimate results. With ambitious concepts that lack a clear direction, the journey can potentially become a jumbled mess of discussions.

Concept Blueprints would be a great way for concepts to improve while maintaining the democracy of the CAP project. It indeed works like analyses in this sense- everyone can participate, but the best players get their voices heard more. As such we get the best of both worlds. With a guiding direction possibly coming before the concept assessment, it would be easier for us to stay on track during the process.
 
to clarify, this concept workshop would be open all the time, and then whenever a new project is starting, we take whatever blueprints are ready to go and put them in a poll?

Birk-a-dit: Concepts are open for submissions whenever. In terms of what gets polled, we could either submit 'em all or have the TL make their slate as they do currently. Either would work, I s'pose.
 
Last edited by a moderator:
I think that concept slating has not been particularly well handled, and concept assessments have been handled even worse. I think this sounds good in theory, but I am definitely concerned by the details, so I'm curious to see it fleshed out more.
 
I'm going to want to think on this a lot more before having a full response, but with regard to --
Birk-a-dit: Concepts are open for submissions whenever. In terms of what gets polled, we could either submit 'em all or have the TL make their slate as they do currently. Either would work, I s'pose.
Click to expand...
I believe that in a system like this, it would be absolutely 100% critical that the TL still make the slate. We should absolutely not have a slate everything mentality for two major reasons.

Firstly, from a polling perspective, we have already had a PR thread last time around about why doing something like this on a competitive poll is not such a good idea. Furthermore, here is already a ton to read, and just having it all open to dozens of concepts makes it super unlikely that most voters would take the time to really know what a concept they are voting for is really about.

Secondly, from the perspective of leadership, not all leaders are fit for all concepts. When I was TL, I certainly do not think I was lacking for topics to pick from, and I made a slate that most people seemed fairly happy with. However, the few complaints I did see was that none of the Stall based concepts were slated. There were many good stall concepts, and it was not simply an oversight that none made it on the slate. The fact was, they didn't make it on the slate because I was terrible with stall, and as the chosen leader for the project, I felt it would have been irresponsible to allow the project to take on a concept that I would not be able to lead properly. So, unless we completely overhaul our leadership system (which I would not really be in favor of), I think it is absolutely necessary that the TL still be able to choose which concepts are slated for a given CAP, as not every TL will necessarily feel comfortable with every concept, no matter how high quality said concept is.
 
Last edited:
I am completely fine with this proposal. In fact, it seems quite similar to how some of the concept discussions have happened for my side project Fuji Labs. Having a single, open-forum place for submissions (and accompanying discourse) is extremely helpful. This fosters discussion about the concepts and provides a good place for newer players and veterans alike to have a back-and-forth on what makes a good concept, and helps point out what to avoid. Currently, having the single thread of concept submissions at the start of the CAP process really just encourages people to submit their ideas, but it does not promote actual discussion of the concepts. I really think this proposal can change that for the better.

Also, I'm completely agreeing with jas that the TL should still be able to slate which approved topics he/she wishes to. A TL will be naturally more knowledgeable on some subjects that relate to certain concepts and less knowledgeable on others.
 
In all honesty I don't have much to say here other than I find the proposal awesome. My only question is how the CAP veterans would be chosen to be the "Quality Control" team. Like would we do it by application or invitation? I would find it awesome, though, because often times I would like to give feedback, but it's all too jumbled and I'm often too tired / lazy to look through all of them, read them, and provide feedback, while this would be MUCH easier because it's more organized and all.
 
I like this proposal, but creating a Concept Blueprint subforum to manage concepts is probably going to be harder than it seems. We're going to need a good plan for policing of the subforum, a place to keep track of approved concepts, rules and deadlines regarding when concepts can be slated, and blueprint submission guidelines among other things. This seems like a good solution, but if it is approved, we're going to need to work out in more detail how we want this subforum to work.
 
The OP says that these discussions would serve as the concept assessment stage. By this wording I'm assuming that this means that the concept assessment stage will be cut out entirely in favor for pre-slating discussion. I agree with the rest of the proposal but this part seems weird to me.

I understand that the point of the QC team is to make sure that the concepts are unambiguous and that they are clear in expressing their goals. However, what the OP has failed to mention is whether or not the QC team is going to set out an outline for what direction each concept would be headed toward or if they are just going to make sure that each concept has clearly asked questions and is practical. If it is the former, then the discussions between QC members and general posters about each concept is not at all going to be suffice in serving as the concept assessment stage. However, even if it is the latter, a concept assessment stage should still be held. I currently view this concept workshop as a forum that will be occupied by many submitters but with few discussers outside of the QC team. And even with those discussers, there will likely, at first, be such a huge backlog of concepts, that it will be difficult for anyone to follow every discussion of every concept. Even if the direction which the project will head towards is clearly mapped from the QC stage (pre-slate), it is still important for those who did not participate in the formation of the concept direction to be able discuss the concept ( / cover our asses from people who say they had no voice in choosing the direction of the project). This new concept assessment stage however, should be short, likely only 1-2 days at the most.
 
So, after reading this and discussing it a bit on IRC, I have to say that I find this proposal very worrisome, as I believe it is trying to mash two separate ideas together and pass them off as one. One of these ideas, concept development and quality control, is something that I think is an absolutely wonderful idea. Concepts are hard, and while lots of people can have good ideas for a project, not everyone is able to take their ideas and formulate a great concept from it. Having a system in place to help develop great concepts is a fantastic idea.

However, as I mentioned, I feel this proposal is trying to take that great idea, and mash this whole "blueprint" thing together with it, and, quite frankly, I think the blueprint idea is absolutely terrible, and very much against the principles of this project. The idea that we should be taking what is now the point of entire discussions and letting them be decided by a QC team before a project even starts is not only anti-democratic, but also anti-discussion, two things that are very important to the project mission, as it now stands. For anything about a Pokemon to be decided before the project starts would mean one of two things: either the concept itself is all about having it, or the QC team is making a decision that no one on this project should ever be allowed to make without a vote. While we can always get in a debate about whether a concept about something specific is good or bad, the fact is that any mandated ability/move/etc. removes discussion and decision from the hands of the CAP community. You can argue that they still get a say by voting for the concept, but that doesn't bring back the discussion that was stolen from them. Furthermore, by allowing such concepts as these, I fear we are doomed to move away from interesting, thought provoking concepts forever.

This may be more of an aside, but beyond my initial reaction of blueprints being anti-CAP in general, I believe their main driving factor is to promote certain kinds of concepts that I believe are unhealthy for the project. Birkal has said more than once (and I agree with him) that one of the major problems with the Cawmodore process was that enthusiasm and quality of discussion dropped significantly after concept assessment. While a lot of people love the idea of a concept that let them utilize their dream niche move, once a move was chosen, most of those people lost interest, since it was not the dream niche move they had hoped for. Should concepts specifying abilities/moves/whatever become allowed like this, I fear we will see a similar phenomenon. While its true that the more specific nature of these concepts might make it harder to win a concept poll, I think you are delusional if you do not think that these kinds of concepts would absolutely fill the concept subforum. We would be swarmed with an endless supply of "Pokemon with X" concepts. While most might end up too silly to make it past the QC stuff, any that do will be likely to get chosen. People love shiny new stuff, and any concept that has a promise of doing something unique will always do well in the polls. But, what do these concepts really do? While the specifics may be different, overall, any "Pokemon with X" concept really just boils down to "Give it X and don't fuck it up." There are no discussions of what we need to do to make the concept work. Only discussions about how not to break the damn thing. While I know not everyone dislikes this as much as I do, with concepts like that being legal, I would be shocked if we are not doing practically the same thing at least every other project, if not more.

And don't even get me started on the things that we currently have banned. We ban things because they screw with the CAP process. Just because we go in knowing we are fucking up the process before hand does not make doing so any better. In the past we have determined that even minor inconsistencies, such as switching stage order, can prove inconvenient. It is for that very reason that we, as a PRC, established a specific order of events and explicitly banned adjusting the process for each project. Concepts involving these banned elements would not be exceptions to this. They would be the very thing we wanted to prevent. They are no good for CAP, and trying to adjust our policy to allow for them is a terrible idea.


In summary: Concept QC is a great idea, and we should totally go through with it. However, we should otherwise stick with current concept rules, as anything along the lines of the blueprint idea steps over the line as to what should be allowed of any small subsection of our community.
 
jas61292 said:
So, after reading this and discussing it a bit on IRC, I have to say that I find this proposal very worrisome, as I believe it is trying to mash two separate ideas together and pass them off as one. One of these ideas, concept development and quality control, is something that I think is an absolutely wonderful idea. Concepts are hard, and while lots of people can have good ideas for a project, not everyone is able to take their ideas and formulate a great concept from it. Having a system in place to help develop great concepts is a fantastic idea.

However, as I mentioned, I feel this proposal is trying to take that great idea, and mash this whole "blueprint" thing together with it, and, quite frankly, I think the blueprint idea is absolutely terrible, and very much against the principles of this project. The idea that we should be taking what is now the point of entire discussions and letting them be decided by a QC team before a project even starts is not only anti-democratic, but also anti-discussion, two things that are very important to the project mission, as it now stands. For anything about a Pokemon to be decided before the project starts would mean one of two things: either the concept itself is all about having it, or the QC team is making a decision that no one on this project should ever be allowed to make without a vote. While we can always get in a debate about whether a concept about something specific is good or bad, the fact is that any mandated ability/move/etc. removes discussion and decision from the hands of the CAP community. You can argue that they still get a say by voting for the concept, but that doesn't bring back the discussion that was stolen from them. Furthermore, by allowing such concepts as these, I fear we are doomed to move away from interesting, thought provoking concepts forever.

This may be more of an aside, but beyond my initial reaction of blueprints being anti-CAP in general, I believe their main driving factor is to promote certain kinds of concepts that I believe are unhealthy for the project. Birkal has said more than once (and I agree with him) that one of the major problems with the Cawmodore process was that enthusiasm and quality of discussion dropped significantly after concept assessment. While a lot of people love the idea of a concept that let them utilize their dream niche move, once a move was chosen, most of those people lost interest, since it was not the dream niche move they had hoped for. Should concepts specifying abilities/moves/whatever become allowed like this, I fear we will see a similar phenomenon. While its true that the more specific nature of these concepts might make it harder to win a concept poll, I think you are delusional if you do not think that these kinds of concepts would absolutely fill the concept subforum. We would be swarmed with an endless supply of "Pokemon with X" concepts. While most might end up too silly to make it past the QC stuff, any that do will be likely to get chosen. People love shiny new stuff, and any concept that has a promise of doing something unique will always do well in the polls. But, what do these concepts really do? While the specifics may be different, overall, any "Pokemon with X" concept really just boils down to "Give it X and don't fuck it up." There are no discussions of what we need to do to make the concept work. Only discussions about how not to break the damn thing. While I know not everyone dislikes this as much as I do, with concepts like that being legal, I would be shocked if we are not doing practically the same thing at least every other project, if not more.

And don't even get me started on the things that we currently have banned. We ban things because they screw with the CAP process. Just because we go in knowing we are fucking up the process before hand does not make doing so any better. In the past we have determined that even minor inconsistencies, such as switching stage order, can prove inconvenient. It is for that very reason that we, as a PRC, established a specific order of events and explicitly banned adjusting the process for each project. Concepts involving these banned elements would not be exceptions to this. They would be the very thing we wanted to prevent. They are no good for CAP, and trying to adjust our policy to allow for them is a terrible idea.


In summary: Concept QC is a great idea, and we should totally go through with it. However, we should otherwise stick with current concept rules, as anything along the lines of the blueprint idea steps over the line as to what should be allowed of any small subsection of our community.
Click to expand...
Or each TL slates the concepts that he likes working on, and if he doesn't like Move centered concepts he won't slate them. Or we create very strict policy about those kind of concepts, and only one of them or something can be stated each time. There are many solutions to the problem you mention, and to me, this seems as a rather small thing, so i would rather discuss it after we find out how to get started with the C&C CAP thing.

Perfect idea and i am on board with it. Find a QC team with people that have both CAP and OU knowledge, create a subforum, and let's get started.

Also, i agree with jas the the slate of concepts for each CAP should be decided by the TL. The TL is supposed to be the driving force of the CAP, so he should be working with something that he feels comfortable with.

Finally, i disagree with jas that having a subforum for vetting concepts is a bad thing. Everyone should be able to post and comment about the concepts in the blueprint subforum, and i am sure that without most people caring about submitting their own ideas, we will have even more quality discussion regarding concept criticism.
 
alexwolf said:
Finally, i disagree with jas that having a subforum for vetting concepts is a bad thing. Everyone should be able to post and comment about the concepts in the blueprint subforum, and i am sure that without most people caring about submitting their own ideas, we will have even more quality discussion regarding concept criticism.
Click to expand...
Just going to quickly respond here because I think I may be being misunderstood. I actually agree with the idea of having a subforum, and I agree that everyone should be able to post in there to help people improve their concepts. The thing that I dislike is the idea of blueprints that plan out more than our current concepts do. Concepts should be just that: concepts that a project should strive for. This blueprint idea goes beyond that, to actually start planning details of the Pokemon, and that is something I cannot get behind.

A lot of what I am saying comes from IRC discussions, as I don't think the OP was as clear as Birkal was there. Looking at the OP, the whole "greater project flexibility" is the specific part I am talking about. Planning things out for the project on each concept before it was ever selected was what blueprints are for, and that is what I object to.
 
jas61292 said:
Just going to quickly respond here because I think I may be being misunderstood. I actually agree with the idea of having a subforum, and I agree that everyone should be able to post in there to help people improve their concepts. The thing that I dislike is the idea of blueprints that plan out more than our current concepts do. Concepts should be just that: concepts that a project should strive for. This blueprint idea goes beyond that, to actually start planning details of the Pokemon, and that is something I cannot get behind.

A lot of what I am saying comes from IRC discussions, as I don't think the OP was as clear as Birkal was there. Looking at the OP, the whole "greater project flexibility" is the specific part I am talking about. Planning things out for the project on each concept before it was ever selected was what blueprints are for, and that is what I object to.
Click to expand...
In this case, i mostly agree. This subforum should be for polishing concepts and maybe adding a few more discussion provoking questions, but that's it. Anything more takes away from the CAP process.
 
This is just a thought I had, but why don't we come to a compromise between Jas's stance, and Birkal's proposal? I'm just throwing out numbers here, so they can be changed, but I propose this: Every time a process-breaking CAP (One that mandates a specific ability, move, typing etc that would cause us to have to skip a stage or go in with an absolute) occurs, there must be at least a three CAP cooldown period before another process-breaking CAP is allowed.

Effectively, this would satisfy both sides - Jas's worries of us getting a Necturna every other CAP would be dispelled, as it would only be able to occur every fourth CAP at the very most, and Birkal's proposal still gets to go through, and we get to take on Necturna-concepts we otherwise would not be able to take.

----

On the note of the idea of a subforum to vet concepts, however, I have full support. It'll lead to a increase in concept quality across the board, for only a small cost in time and management. However, I do think that the standards to be allowed to vet concepts (the proposed team of CAP veterans) be somewhat strict - being a veteran alone should not be enough. As the goal is to increase quality, we need posters who know how to vet, and while a lot of our veterans do have skill in various aspects of CAP, not all of them have skill in C&C.
 
I feel that the consensus here is to move ahead with a Concept Blueprinting subforum here in CAP. However, since there are numerous opposing view on how this forum should look (all are justifiable, by the way), let's make this as bare bones as possible. Since this is a significant change, let's not add on a plethora of new rules and regulations. This proposal would operate as follows:
  • Concept Blueprinting forum is created, with a Rules OP and an Index of WIP Concepts.
  • Anyone can post a concept at any time. Anyone can comment on a concept at any time.
  • A Quality Control team is set up to work on improving concepts to be suitable for the CAP Process.
    • Improved areas include adding more relevant questions and making it clear for the CAP Process.
    • QC team is allowed to reject concepts that will not lead to a process full of learning. Rejected concepts are locked.
    • I will personally lead the QC team with three or four other members to start. This number could grow with active contributors to the forum making their mark.
    • Applications for the Concept QC are available to anyone, but members are expected to be very knowledgeable about the CAP Process and helpful.
  • At the start of a CAP, the TL slates any concepts they choose that were authored between that date and the previous date of selection.
  • After voting, we host a brief Concept Assessment to solidify the chosen blueprint (not change it), and move forward with the project.

Is this agreeable with everyone? If you have any final comments, please make them within the next 48 hours. As always, if you need more time to write up a post, simply ask for it.
 
Birkal said:
  • At the start of a CAP, the TL slates any concepts they choose that were authored between that date and the previous date of selection.
Click to expand...
Why the previous date of selection though? Why not just choose from the whole pool of concepts available? I agree with everything else btw.
 
Birkal said:
[*]I will personally lead the QC team with three or four other members to start. This number could grow with active contributors to the forum making their mark.
[/LIST]
Click to expand...

4-5 people is not going to be enough. You've seen how many concepts we get submitted Birkal. Maybe I'm paranoid, but I think we should start more than just 4-5 people total.

Otherwise I don't have anything else to say, looks good.
 
Unfortunately I've been psychotically busy this week, but I feel an urgent need to weigh in on this before tomorrow rolls around.

Concept Blueprinting sounds like an excuse to massively poll jump. While I'm usually the first person to air my discontent at recently selected concepts, I really do not see how this addresses the core issues of concepts. The chief benefit to this policy is that it almost entirely removes bandwagoning during the few days we currently have the concept submissions, and allows for better analysis of each individual concept.

What I think is difficult to discern here, and to use a recent example, is the core selection of Major Third. Would we have selected a better core because we would be assessing it before it was selected, or would we essentially repeat the process we ended up with where the topic consisted mostly of boosting one's favored core over analyzing the presented options.

I will submit, however, that the benefits to eliminating the Free-for-All environment that plagues our current concepts would be a major improvement on project quality. I think we should be careful just how far we allow this "blueprinting" to go. Ideally we end up eliminating *unnecessary* vagueness in our concepts while not dictating a predetermined outcome. The reason we have so many rules on so many stages is that so many different aspects of this game can break our process, so in the end I think if this will relax that restraint by allowing more forethought, it will be more beneficial than harmful in the end.
 
alexwolf said:
Why the previous date of selection though? Why not just choose from the whole pool of concepts available? I agree with everything else btw.
Click to expand...

This would be my question as well. I can certainly understand wanting to keep things relevant, but I'd rather not force us to go QC the same things over and over cause the concepts "expired" after that project.

On a different note, I would just like to suggest that, if we go with what Birkal's last post suggested, we drop the whole "blueprint" word. As is, that proposal just uses concepts as we have them now, and puts them through some QC. That is nothing new. Its use in this thread has caused a ton of confusion for me personally, as I have found it difficult to differentiate what suggestions are part of concept QC and what are about changing concept rules themselves, and where blueprints fit into all of them. And if it confuses me that much, I fear the effect it could have on people who want to submit. Unless we are changing something significant about concepts (which I don't think that last post suggested), we should just leave them as "concepts." A "Concept Submissions" or "Concept Quality Control" subforum gets the point across without introducing any new terminology. Cause god knows we have more than enough unique terminology as is.
 
I support "Concept Quality Control," because we can then call it CQC, further adding to CAP's long, proud tradition of TLAs (Three Letter Acronyms).

What I think is difficult to discern here, and to use a recent example, is the core selection of Major Third. Would we have selected a better core because we would be assessing it before it was selected, or would we essentially repeat the process we ended up with where the topic consisted mostly of boosting one's favored core over analyzing the presented options.
Click to expand...

In this case, absolutely. Major Third is somewhat of a poor example to use for that argument, because we would more than likely have selected a better core if we had more assessment and overview on it before it was selected (with all due respect to DetroitLolcat and his work on that project, that was a major dun goof'd decision on his part that could have likely been averted, had we more competitive oversight).
 
I'm honestly still not sure what this is supposed to solve - it's not like we don't already give the TL advice when they are selecting their slate, and I'm sceptical with regard to what a dedicated QC team could do to solve a perceived problem with untenable concepts without actually breaking the rule about not specifying particular aspects of the build.

But oh well.

If we do go through with this, it would be very helpful to have a coherent proposal as opposed to the vague promise of a QC team of some nature doing something over an unspecified period of time - let me say that one thing that stands out to me in particular is that this has the potential to stretch out the Concept stage if it isn't timed properly, if we go with the "authored between <x> and <y> dates" model, and so I would very much recommend that the total pool of concepts be always available for consideration.

"Making it clear for the CAP process" needs to be clarified a bit - what exactly does this entail that the current system of simply weeding out badly worded concepts does not (Well, ok, with the exception of the last CAP)? Aside from rewriting bits of prose and adding more questions... certainly there's the potential for more explanation in the justification section which C&C can help with. As such, this should be spelled out clearly as well as what "making it clear for the CAP process" does NOT mean (e.g. specifying type/ability/exact stats).

Just so I'm not misunderstood, this is a good idea, if simply because of the fact that badly worded but interesting concepts can be renovated to actually stand a chance at the polls, but I do think that the proposal can be elaborated upon a little more at this stage, given that nobody seems to have (yet) disagreed with the fundamental premise.

As a last note - I would suggest that one more point be added to the proposal, namely that
  • The name of the user who originally submitted the concept is not included in the concept poll
There was a lot of discussion in the past about whether or not we could divorce names from poll entries in order to prevent the Shiny Badge Effect, but as I recall it was deemed not worth the effort when most people would be following the discussion thread in general. Now, however, there would be a cumulative effort behind every concept and they could be polled with much less hassle. Oh, and we could still keep the concept winner's name on the CAP site, but simply forgo the name for the polls.
 
Sorry for not posting in this thread until now, considering my involvement with Birkal in bringing it up in the first place. But I have been traveling internationally the past two weeks and I had far less free time and internet connectivity than I expected. Also, ORAS came out the day I got home, and I've been spending quite a bit of time... y'know... playing Pokemon (gasp!). So even though this PR thread is important to me, I've only just now gotten a chance to make a substantial post.

In order to separate my role as a policy leader from my role as a discussion participant, I'll post some framing guidance in this post and I'll give my personal input later in a separate post.

After reading this thread, despite some concerns raised, it appears that the general proposal posted in the OP and the refined proposal made by Birkal are both still good. However, several posters have commented about lack of clarity as to what exactly we will be doing with this new proposed "Concept Blueprinting/QC/Workshop/Whatever Subforum". In this PR thread, we need to work this out a bit further. So to frame this for further policy discussion:

What exactly will be we be doing in the new Concept Subforum?​

This is the key question we need to focus on a bit more. To give some further guidance to PRC discussion participants, I think we have three "camps" emerging in this thread. For each camp, here's some more framing sub-questions (?) to consider:

Blueprints
If you like the general idea of Blueprints, then the following questions need some commentary to make them a bit more "real" to everyone, beyond the somewhat vague ideas mentioned so far.
  • If we make these "Blueprints", what will they look like?
  • How do Blueprints differ from regular Concept Submissions or what do they add to Concept Submissions that are not already there?
  • How will Blueprints be constructed? (ie. What is the process to make them?) (ie. Who does what?)

Concept Improvement
If you are NOT generally liking the idea of Blueprints per se, BUT you think Concepts in CAP could be improved via a dedicated subforum "workshop" of some kind, then consider these questions:
  • What are the problems with current concepts that can be solved with dedicated attention in a subforum?
  • What will be the process to improve concepts in the subforum and who will be involved?
  • What interaction or conflict will the subforum process have with the normal CAP construction process?

Anti-Proposal
You are against this proposal overall, if either of the following are true:
  • You think CAP concepts are just fine as is. Sometimes we get good concepts, sometimes we get bad concepts. That's just how it goes in Create-A-Pokemon.
  • You think concepts need to be improved, but we don't need any policy changes to improve them. You think we just need to do a better job with Concept Submissions, Concept slating, and Concept Assessments in the existing CAP process.

I don't consider any of these camps to be "right" or "wrong", and feel free to say where you land in this. But based on my read of the thread so far, these seem to be the three positions that are emerging. If you fall in the gray area somewhere between these positions or have another position entirely, I'm happy to hear about that too. I just want us to frame the poilcy discussion more clearly, so we can make distinctions as to what is being discussed and what will ultimately be decided.
 
DougJustDoug said:
What exactly will be we be doing in the new Concept Subforum?
Click to expand...

If it wasn't obvious from my comments in the IRC logs Birkal posted in the OP -- I am in "Blueprints" camp (from the previous post). But I'm not really sure what a Blueprint will look like exactly. So to work towards a more concrete notion of a Blueprint, I'll discuss the problem with Concepts today and discuss a couple of ways that some form of Concept QC team could work on improving concepts.

Policy Proposal Inspiration
I've been unhappy with CAP Concepts for quite some time, but I never could put my finger on exactly WHY. Then on this last project with Plasmanta, during the Concept Assessment the problem became clearer.

We pick Concepts that SEEM good on the surface, but they fall apart under closer scrutiny.

On Plasmanta, I thought the concept was pretty good when I first considered it. In a nutshell, I thought the concept meant we would be experimenting with fainting mechanics. That seemed like interesting, unexplored territory with several avenues that could be pursued, depending on what the community favored. That's pretty much the textbook definition of a good CAP concept, right?

Well, the key word above is "seemed". When we went into Concept Assessment, other people immediately began posting facts and interpretations that dramatically changed my read on the concept.
  • People correctly noted that Fainting, no matter how you slice it, can never really be "good" for your team or "bad" for the other team. You can make the act of fainting "better" than normal, but it will never be a good thing, except for some highly contrived or rare situations.
  • Many of the moves and abilities that pertain to fainting are gimmicks, and can't really be the focal point of an entire pokemon.
  • Most of the genuinely useful fainting-related roles or strategies (like Suicide Lead) are incredibly straightforward to build and don't provide many options for the CAP community to discuss and decide.

I'm not trying to completely rehash CAP 19 here in this post, I'm just illustrating that very quickly after we began assessing the Concept in detail, we discovered two key things:
  • Some interpretations of the concept were fundamentally flawed and not "competitively viable"
  • Some competitively viable interpretations were not "project viable", in that they did not provide a range of interesting choices throughout the construction process.

This happens often on CAP projects, but rarely does it happen so early in the process and almost never does it happen in the Concept Assessment, before anything has been decided. Normally with so-called "bad concepts", we get a concept assessment that seems decent enough, but after a couple of polls, we find ourselves somewhat "stuck". As in:
  • We have something that is not competitively viable (overpowered or underpowered), and requires massive compensations in later steps that are inconsistent with the spirit or direction of the project
  • The trajectory of the project becomes so narrow that the last few steps of the project are pretty much a foregone conclusion with very little real discussion or interesting choices to be made

On CAP 19, as we were going through the Concept Assessment, I realized we needed to spend more time vetting concepts in more detail before slating them. That led to my discussion with Birkal which led to this proposal.

Vetting Concepts
In order to decide what the proposed subforum should do, let's define what we are trying to achieve. In the previous paragraph, I mentioned that we need to vet concepts better and somewhat prove that concepts are "good" before slating them. Using terms and descriptions from above, I propose the following definitions:

A "Viable CAP Concept" should have BOTH of the following characteristics -
  • "Competitively Viable" - Every step of the CAP process should be able to be implemented with normal competitive game mechanics and battle strategies. Mechanics and strategies should not be likely to be overpowered, underpowered, or overly contrived (ie. "gimmicks")
  • "Community Project Viable" - Every step of the CAP process should have multiple viable options to intelligently discuss, debate and decide.
Notice those definitions refer to "every step of the CAP process"? That's the hard part of all this. How can we determine how a concept will hold up across every step of the CAP process in advance? Obviously there is no way to prove how things will happen across multiple steps that execute over several weeks of calendar time. But we can do simulations to see how a concept MIGHT play out or COULD play out. Those simulations are what I propose we use as a basic building block of doing "Quality Control" on a Concept Submission in the proposed subforum.

I imagine a Concept QC process looking something like this:

A concept submission is made as the OP of a thread in the Concept Subforum.
Perhaps there is some pre-approval process required before a thread is actually made. Maybe there is an open "cattle call" thread where everyone posts concept submissions as posts, and we approve certain posts to be made into full-blown "Concept Threads". Or maybe we allow anyone to make threads and we close the threads that are crappy from the outset. I don't really care too much at this point HOW the "potentially good" concept threads get created. Perhaps we can look to various techniques used in C&C to manage who makes analysis threads and whatnot to guide this part of our process. But regardless of how we do it, I'll assume, without too much trouble, we can come up with SOME WAY to get threads with Concept Submissions as the OP of the thread, and the postings in these threads will be the "QC workshop" for the submitted concept.

QC Phase 1 - Concept Discussion
Discuss the concept as a whole and see if it warrants deeper analysis. This is intended to get immediate or high-level impressions out of the way. My guess is that most bad concepts will have problems emerge right from the outset. In this phase, QC posters can focus on specific aspects of the concept like:
  • Wording or grammar
  • Clarify ambiguous parts of the submissions
  • Add, remove, or clarify Questions To Be Answered
  • Specific mechanics referred by the concept
  • Etc.
Or QC posts can focus on broader issues like:
  • Is the concept interesting?
  • Does the role already exist in the metagame?
  • Has this been done before in CAP (can be a good or bad thing)?
  • Etc.
Phase 1 Pass/Fail Determination
I don't really care how this happens, but we'll need a way to determine which concepts will proceed to Phase 2. If it fails Phase 1, the thread is closed.

QC Phase 2 - Simulation / "Play it forward"
This is the hallmark of the vetting process I propose. QC posters will do a "CAP Simulation" of the proposed concept in a single post.

A Simulation will "play forward" the concept across the key areas of CAP -- Assessment, Typing, Abilities, Stats, and Movepool. A Simulation would consist of each of those five high-level areas presented, in order, as if it were a single step in CAP. The poster will predict and present the multiple viable options that would emerge in each simulated high-level step of the process. On each simulated step, the QC poster will also select one or more options as the presumed "winning options" for that area of the process. Subsequent steps are presented dependent on the previous winning options.

Basically, a Simulation is a simplified "mini-CAP" with five sequential steps that correspond roughly to the actual CAP construction process.

The goal here is to have a few intelligent CAP veterans play forward a CAP concept using "CAP process theorymon" to predict the outcome. Then others can comment on the simulation according to the "competitively viable" and "community project viable" definitions mentioned above, as in:
  • Are the predicted options in each simulated step all competitively viable?
  • Would multiple options in each simulated step have a real shot at winning or is one option pretty much guaranteed to win?
With a bad concept, it will be hard to play out, even in just a theoretical simulation. Or even if one simulation is reasonable, it will be hard to come up with multiple good simulations. With a good concept, there should be multiple simulations predicted, and each one should result in a significantly different outcome.

Phase 2 Pass/Fail Determination
Similar to above. I don't know or care how this happens right now.

Blueprint
I'll leave this part TBD. I have presented a lot of foundation stuff to discuss, and I really need to hear some feedback on everything before I get into Blueprints themselves. Maybe I'm off-track completely, or maybe I'm onto something workable. We'll see.

But I will say that I DO NOT envision Blueprints being a specific dictation of the outcome of CAP steps. Although I advocate simulations mentioned above, I do not see those being part of the Blueprint really. I see them as discussion and analysis tools that help us vet concepts, and they won't be included with the Concept Blueprint or any other artifact of the Concept QC process that makes its way into the actual construction process.​
 
Last edited:
1) The simulation aspect is exactly what I had in mind for concept vetting. Too often I see a concept where there is exactly one viable (or at least one clear best) path to take, and that's just not interesting.

It happened on the Doubles CAP that Pwnemon and I ran until interest plummeted. I could easily pull up logs of me talking to Pwne where I said "Grass/Ground with Sand Rush" almost a week before that typing was picked and the ability was all but forced. The next step would have been stats comparable to Venusaur (probably a bit stronger) and eventually Sleep Powder. Basically to make Doubles Sand look like Doubles Sun. And that was obvious to anyone who knew Doubles and who knew CAP (basically Pwne and me). Perhaps we shouldn't have slated it for that reason, but we did.

That's just not an interesting concept when there's one path so clearly the best. Any sort of concept vetting absolutely needs to ensure that there are multiple viable paths.

2) Personally I don't want the blueprint to actually happen. I just want to ensure that only viable and doable concepts are able to be slated. Personally I'd prefer that the forum where the QC happens to be a hidden forum. I don't think that the QC team's "blueprint" should overly influence discussion, but it's there for mods and project leadership to see to help keep the project on track if the public is way off base.

3) Realistically speaking, however, I am concerned that there just isn't enough quality manpower to get everything in this thread accomplished, when it's already hard enough to find 5 quality members for the TL/TLT.
 
Couple of quick points.

Firstly, I agree that there has to be some form of foresight going on. In every concept I personally see submitted, I can look at it, and immediately try and think as to whether its possible, or thought provoking enough to answer important questions of the OU metagame. Obviously, not everyone who votes does this, so having some sort of "play it through" for the QC team is important. Agreeing with srk though that it should be largely private, the last thing we want is a visible QC "playing it forward", and then having people bandwagon submissions because "The QC team mentioned it" when they played it forward.

Of course, I'm not certain if CAP can actually gain a large selection of concepts that the QC team would agree upon as acceptable. I'm really wondering how many quality submissions we would actually get, this idea falls pretty flat if the TL only has something like 1-2 quality submissions to pick form, and judging from previous concept submissions, there are quite a few concepts that are either illegal or don't offer much in either competitiveness or community viability. Furthermore, CAP process tend to take quite a while, and certain concepts risk becoming outdated even if the QC team gives them a pass. For instance, say the metagame experienced a period where pokemon x, y and z were particularly dominating, or style x, y, z were particularly popular due to defending against x. In (say) January this concept might be valid but by (say) March, might be no longer relevant to the metagame. I guess my worry is not so much 'can it be done' but more 'how many quality submissions will we actually end up getting".
 
You must log in or register to reply here.
Back
Top