Readability Indices for Smogon Articles?

[X-Act]

Long story short: in the postgraduate course in education I'm reading, I learned about a few readability indices. These are used to provide an estimate of the readability of a piece of text. The output of these indices is invariably the minimum grade level, based on the U.S. education system, that a person needs to have in order to understand the text. For example, if the readability index of an article is 10, then Grade 10 students or older (14-15 year olds) would be able to understand the text.

I think that these readability indices can be used in our articles or analyses. We need to keep in mind that the average people that are reading our analyses or articles are probably Grade 10 students. That means that we shouldn't write text that is more complex than this.

This website provides readability indices for any text that is pasted in it, together with their average. I think it would be a good idea to paste the article or analysis we're typing or editing in this website to check whether we get a Grade 10-level text or lower. If it's higher than Grade 10, here are some suggestions to simplify the text:

• Split long sentences into shorter ones.
• Try to replace words having 3 or more syllables with other words having 2 or less syllables without altering the meaning of the sentence.

Here are some average scores for some of our articles:

Attacking Types (from The Smog #3): 8.68
Normalized Base Stats: 8.72
Introduction To Competitive Pokemon: 10.12
Mastering the OU Ladder (from The Smog #5): 10.54
The Art of Hail Stall: 10.58
Bronzong Analysis: 11.36
Jolteon Analysis: 11.60

I don't know if the analyses I chose are exceptions, but it seems like analyses seem to be beyond the comprehension of Grade 10 students. Maybe they require a little bit of simplification?

As an aside, my articles' non-mathematical parts can be oddly understood by Grade 9 students, which is somewhat of a shock to me.

(This post's readability index is 8.22, so it should be clear to every Smogon member!)

 

[Synre]

I have two thoughts here that go in completely opposite directions:

The first one is that this is a good idea based on the direction the ship has been turning. This is another one of those things that, five years ago, would likely not have been looked at positively, but as things are we already write the content of our articles more for people in that age group. We explain things we'd have always taken for granted years ago, trying to be more complete and inclusive. It's probably a logical next step to change the prose to hit a wider audience, as well.

However on a less logical, more personal level I have to object. I'm not sure if this is my pre-Smogon botter elitism talking or my English minor or what, but I don't think capping the quality of our writing to be more readable is a good idea.

There are a couple major reasons for this:

- I think first and foremost the demographic we want to hit, even when aged in 9th or 10th grade, has a reading level higher than 9th or 10th grade. I would wager that even though a majority of our staff members joined when they were that age or younger, they'd have had little trouble reading the articles because they were reading at a much higher level than what is average for their age group. I would wager this is true for almost everyone who ends up being reasonably competitive at this game; it's complicated enough(though perhaps not so complicated as we'd like to think) that the type of people who latch on to the game and become the type of players we're marketing(originally marketed?) to typically have above average reading/writing/mathematical ability anyway.

-This more goes with the degree bias, but I don't think it's such a bad thing to make people think a little harder to read our posts. We're giving them free information anyway, I don't see much harm in giving them something that's at a slightly higher reading level than what they might be used to. Challenging people is usually a good thing. I'd be interested in knowing a bit more about how that rating is calculated, anyway, since looking over that Jolteon analysis, for instance, there doesn't seem to be anything even remotely confusing about the writing based on the chosen prose to me.

I think that either way we should make the choice that best hits our audience, but that maybe we should think a bit about what our intended audience is.

 

[Bologo]

Heh, that site is awesome. I personally don't think that there are any articles or analyses on our site that are a ridiculous reading level. The only one I've been able to find so far is the current Heatran analysis. The reading level of that is 12.06, which is ridiculous, considering it seems to take a hell of a lot for the text to reach that level. I know that analysis is one of the ones going through concision, but basically, if anyone wants an example of something that might be really difficult to read on this site, there it is.

In particular with this analysis, I looked at some of the individual sets that looked hard to read:

Stealth Rock Lead - 12.06
Life Orb - 14.06
Choice Specs - 12.28
Rest + Sleep Talk - 12.52
Burn Support - 12.20
Taunt - 12.46
Team Options - 13.32

The average of these sets makes the current Heatran analysis seem on par with a 1st year university paper than something that grade 10/11s could understand.

The other parts of this analysis are under grade 11 level, which I think is fine.

 

[diinbong]

wow I can't believe my Togekiss analysis scored a 12.90... I'll keep this in mind when writing future articles.

 

[twash]

This is a pretty cool site. Arceus' second set managed to score 16.26, which is the highest I can find. Most seem to be over 10, if not 11 or 12...

 

[Arkeis]

Wow, thanks X-Act, this is really interesting. Just for fun, I typed in the following

"A is for apple. B is for bird. C is for cat."

And got an average reading level of -1.34.
lol I guess that would be kindergarten or something.

I then typed in random gibberish

"sddjkjakfsiasfifosoisaoaooiaafjasiosfaasm"

and got a level of 114.08!

 

[EspyJoel]

I agree with the idea of making some sets more readable but I think we've got to be careful about how we take these grade scores, for example, the Jolteon SpecsJolt analysis got a 11.82 and I don't see the language used to be complicated, and I don't see why a 14-15 year old couldn't understand it when according to these scores, they wouldn't be able to. Also, on the idea of being careful with grade scores, although its not a Pokemon article so a bit off topic, a simple A2 Politics conclusion (which would be around the age 17-18) got a 16.38 meaning you would apparently have to be 20-21 to understand this. The word Impeachment gets 21.24 grade, so apparently I'm not supposed to understand what I'm writing.

Also, adding to bojangles point below, Dragonite and Salamence get 17, and Tyranitar 19.36.

 

[bojangles]

I'd just like to note the fact that the names of Pokémon seem to drive up the scale, because they're not real words. For example

"I attacked the opponent."
Scored a 6.60 on the Flesch-Kincaid Grade Level. This remains the same with changing "I" to "He". However, changing it to:

"Celebi attacked the opponent."
Scored a 12.5 on the same scale, which is a massive jump of 5.9 grades.

I repeated this with "He has many moves." It scored a 0.7 on the Flesch-Kincaid Grade Level. When I changed this to "Rayquaza has many moves", it jumped to 6.6, which is another jump of about 5.9.

It seems to elevate the grade level when Pokémon names are used, for the reason I stated before. We should keep this in mind when using it with our articles. It may very well be that certain analyses are getting high grade levels because of the use of Pokémon names/terms.

 

[diinbong]

I tested that too bojangles, but I concluded that it had to do mostly with number of syllables. I'm fairly sure that since "Rayquaza" is three syllables compared to a sentence of 7 it makes a big deal compared to "he" which is 1 to 5. Alien words did have some small effect on the grade, but in entirety I don't think that is a big deal.

 

[TAY]

I understand why we would want to make the analyses and articles more readable, but I do not want The Smog dumbing down its articles just so sixth graders can understand them.

And just for reference, that sentence scored a 14.82. No Pokemon names or anything. Thank god I'm into my junior year of college and I can understand that sentence now! There is no way in hell The Smog is splitting that sentence in two.

This may not be a particularly reliable resource. For one thing, syllable count seems to be more important that anything else. Using the word "yellow" as opposed to any of the one syllable colors raises a simple sentence by a whole grade level....

I do understand the sentiment though. As someone who does occasionally make edits to on-site writing, I will take readability into account in the future. I don't think we should make a big deal about this is C&C though. People have enough trouble understanding the writing guidelines as it is.

 

[X-Act]

I'm not saying that this is a hard-and-fast rule that everyone should employ. Rather, I'm saying that this is a tool that writers for the site can use if they want to simplify their text.

Also, as a word of precaution: these readability indices work best if the article has at least 30 sentences (at least that's what they told me in my course). So typing in a few sentences just to test the tool doesn't work too well I'm afraid. Sometimes, if the text is long, a few samples of 10 sentences each are tested, though, for some reason, text from the introduction and from the conclusion are not chosen. It has something to do with the different way they are written compared with the rest of the text, I'm told.

 

[Glen]

aw but i like being able to show non-monners the heatran analysis and go 'SEE?'

 

[X-Act]

Also, as a response to TAY, I think it really depends on what you want to do with your article. If an article wishes to teach something about a Pokemon, such as a Pokemon analysis, I think it should be understandable by the majority of people that wish to use such a Pokemon. If, on the other hand, an article is just for general information, such as some of the Smog articles, then this restriction should be alleviated.

I repeat that I am of the opinion that Pokemon analyses should be understandable by most people. It's useless to write something about a Pokemon that the average player cannot understand. It's not how intelligent you are and how impressive your prose is that counts when writing a Pokemon analysis; it's how clear your point is to the average user that counts.

 

[Hipmonlee]

For a lot of these analyses with the high scores it isnt so much hard to understand as it is poor writing..

The sentences in that Rayquaza analysis really do meander.. There's no need for that.

Have a nice day.

 

[chaos]

I don't like the idea of associating a number with "readability." While some might not readability < 10 as a hard and fast rule, I imagine many will... I don't want the effectiveness of the writing to suffer because it uses "advanced sentence structure" (or whatever)

edit: I think their implementation is broken. I got a 102 in reading ease once, when the score is supposed to go from 0-100

 

[X-Act]

I don't like the idea of associating a number with "readability." While some might not readability < 10 as a hard and fast rule, I imagine many will... I don't want the effectiveness of the writing to suffer because it uses "advanced sentence structure" (or whatever)

I don't like the idea either, but apparently many people from all walks of life use these, from educators to magazine editors, to usage in health care literature and military agencies. In fact I was surprised that these are so widely used, so I kinda bit my lip when I myself was doubtful about their usefulness. And again, I want to reiterate that these should only be used as guidelines, not hard-and-fast rules. :(

edit: I think their implementation is broken. I got a 102 in reading ease once, when the score is supposed to go from 0-100

If you're referring to the Flesch Reading Ease score, it can actually go to about as high as 120 in theory, but it almost never does in practice. Also, remember that these measures should only be used on texts having 30 sentences or more.

 

[obi]

Increasing readability is not the same as dumbing down. It's the difference between

Celebi utilizes Grass Knot.

and

Celebi uses Grass Knot.

The second sentence is not dumbed down in any way, but is more readable. "Dumbing down" can help with readability, but that's not what anyone is proposing we do. Consider the following from the Lynch Guide to Grammar and Style:

There's nothing inherently wrong with long words, but too many people think a long word is always better than a short one. It doubtless comes from a desire to impress, to sound more authoritative, but it usually ends in imprecision and gracelessness — and, what may be worse, if you use long words improperly you sound like an ass. (Look up malapropism in your dictionary, or, better yet, read Richard Brinsley Sheridan's play, The Rivals.) Words like functionality and methodology have their proper uses, but they're not the same as function and method. See also Anticipate, Utilize, Obfuscation, and Vocabulary.

 

[Synre]

"Utilize" is almost always wrong in that context, though, since utilize connotes more something you can take advantage of to improve your situation, whereas use is a more general term. I think in my previous post I was unnecessarily pretentious, but I think the general jist of what I wanted to say was that using the smallest words possible to reduce reading level isn't ideal. Using the biggest word possible isn't any better and isn't what I meant to imply either, because no matter what we should be using the most precise word available, regardless of its size.

Using the use/utilize thing, you could write Celebi use/utilizes Grass Knot and be "right" with either word, but use would make more sense because utilize just makes the prose purple, it doesn't make it any more descriptive(if anything it makes Grass Knot sound more valuable than it is). On the other hand, something like "Salamence utilizes the hole Magnezone puts in your opponent's team by removing their Steel-types, allowing it to use Outrage and Draco Meteor relatively unchecked" would be a situation where "utilize" is still better than "use" since the difference in connotation makes it fit the situation a little better. Though, that's a very "4 AM" example since "exploits" would make more sense instead of utilize...

I just think in general we should be trying to be as precise as possible, as with any writing, without worrying too much about whether or not what we're writing is at a reading level that is "too high." If something is too complicated to understand, or written lazily enough that the meaning is convoluted, it shouldn't have gotten on the site to begin with, since that's something that should be unacceptable already. I don't seen any reason to expand from simply using the most accurate word for the situation, there's isn't much benefit in going any farther than that. I'm not a fan of this tool since it gives us an arbitrary level to aim for. Robots don't tend to evaluate writing, which is always pretty subjective, very well, and the amount pokemon and move names inflate the rating is a good example of why.

 

[cantab]

I'd just like to note the fact that the names of Pokémon seem to drive up the scale, because they're not real words. For example

"I attacked the opponent."
Scored a 6.60 on the Flesch-Kincaid Grade Level. This remains the same with changing "I" to "He". However, changing it to:

"Celebi attacked the opponent."
Scored a 12.5 on the same scale, which is a massive jump of 5.9 grades.

I repeated this with "He has many moves." It scored a 0.7 on the Flesch-Kincaid Grade Level. When I changed this to "Rayquaza has many moves", it jumped to 6.6, which is another jump of about 5.9.

It seems to elevate the grade level when Pokémon names are used, for the reason I stated before. We should keep this in mind when using it with our articles. It may very well be that certain analyses are getting high grade levels because of the use of Pokémon names/terms.

In which case, that is the wrong index. The Gunning-Fog Score seems insensitive to Pokemon names - "He attacked the opponent", "Muk attacked the opponent", and "Rayquaza attacked the opponent" all score 11.60.

For a longer text, I tried the first Arceus set, and got a 17.50. Then I searched and replaced Arceus with a real name (Simon) and got no change, 'Muk' and got no change, and 'a beaver' which got a .2 increase. I then made some more substantial simplifiying changes, to the following text:
Changes are in bold, but I may have missed some.

Of the many options Arceus has, this set is perhaps one of the most fearsome that exist. With Arceus’s bulk, it should have no problem finding a niche for one Swords Dance, and maybe even two or three. Because it doesn’t particularly care about paralysis from Thunder as ExtremeSpeed’s priority advantage negates the Speed loss (in fact, it is arguably good for this flavor of Arceus to be paralyzed because then it can’t be burned and have its attacking ability reduced), one may switch this Arceus on weak attackers such as Blissey with impunity. After even a single Swords Dance, ExtremeSpeed will OHKO all Darkrai, Palkia and Latias unless they invest significantly in Defense, and has a very good chance of OHKOing the standard 40 HP EVs Mewtwo, assuming that one has at least a Silk Scarf boost. Earthquake smashes Dialga and Metagross into the ground without question. For the last move, Shadow Claw is the preferred choice since it lands super effective hits on the Uber physical walls of Lugia and Giratina; however, Overheat is an acceptable move to kill Skarmory and Forretress.
The EV spread for this Arceus provides for a combination of bulk and power; Speed isn’t necessary as ExtremeSpeed, this set’s main attack, will attack before most other offensive moves anyway. Silk Scarf will net ExtremeSpeed 20% more power, a boost that is critical to OHKOing many Uber threats at full health, whereas Life Orb will provide even more power to all of Arceus’s attacks at the expense of some health every time it does damage. While Life Orb does contradict this set’s bulky sweeper theme, it will allow Arceus to OHKO the standard Mewtwo all the time and often 2HKO Giratina with a Swords Danced Shadow Claw, OHKOing guaranteed with a critical hit.
While this set often rips through half a team or more, it is often better to only use ExtremeSpeed when it is needed, as its meager 8 PP is easily used up hitting Lugia and Giratina that don’t care about it; what worsens the PP problems is that many Ubers, including [words deleted] Giratina and Lugia, have the Pressure ability, which doubles PP usage for direct attacks. With its main asset gone, this Arceus is made useless. As long as one doesn’t play badly, however, these statements should never stand to discourage one from taking advantage of the divine terror that this set is.[/code]which dropped the score to 16.40.

Conclusion: If we are going to use a readability index, the Gunning-Fog index is probably the one we should use

EDIT: I've read up more on how the index works. The key with the Gunning Fog index is that it IGNORES jargon and proper nouns which are assumed to be known (and also cannot be changed for anything simpler). For us, the Pokemon names are assumed known, while terms like OHKO, HP, and so on we have a glossary for.

 

[jc104]

I think that, purely for the name, we should use the SMOG index for readability (it also stands for "Simple measure of gobbledygook.")

On a more serious note, the Gunning-Fog index removes "familiar jargon" and proper nouns, which would in theory work better. However, there is also a difference in that it does not distinguish between one-syllable and two-syllable words - so the question becomes "is a two syllable word harder to read than a one syllable word?" My instinct says yes. I dont know if it would be practical to remove pokemon names when carrying out the calculation of the Flesch-Kincaid Grade Level, but this would probably be the best indicator.

The most important thing rather than the length of the words is the sentence length. In some of the analyses these are frighteningly long - this doesn't indicate an advanced level of english, it simply indicates that it is difficult to understand (or even, simply, wrong.) Complex words can make the analyses more interesting reads for the more able among us, and some are necessary "jargon". Therefore I suggest that we use the scale that leans the most towards sentence length. One issue I can see is that sentences split with dashes and semicolons are easier to read than those split with commas or not split at all; I see, however, no obvious solution to this.

EDIT: My post got an average readability grade level of 14.78 - was it really that hard to read? This has somewhat diminished my trust in the scales themselves.
EDIT 2: There were some full stops missing - I could see this having a large impact on some of the analyses if they have not been carefully checked for this seemingly minor detail (particularly notable at the end of paragraphs.) It is probably a good idea to ignore all titles and movesets as well, as these have no full stops but are extremely easy to read. Running the calculator again, it gave me 11.82 - more reasonable in all. The word "Gobbledygook" can't have helped either. Also, I support Teifu in his protest against the excessive use of the word utilise. (Also, the word "abuse" instead of "use" is abused excessively!)
EDIT 3: You should definitely check if the calculator counts decimal points as full stops - this may have contributed somewhat to the apparent readability of the OP. Having checked, it apparently does.