First off, let me start by saying: "Just because it has counters, doesn't mean it's not broken." This applies to Dory really more than any other pokemon in OU.
Now, if one was to consider Dory broken (I'm not extremely adamant about it), the biggest issue I have with it is the dynamic to the speed game it brings to OU, which is completely unhealthy. Dory Outpeeds Deoxys-S and every scarf-user in the game without doing anything! Yes you can say it loses that speed should sand break, but let's face it-- Sand will always be the primary weather due to Tyranitar and Hippowdon's completely superiority as individual pokemon to Politoed.
Also, Rain v. Rain becomes something of a crap shoot with Rain teams having little means of good anti-rain support available to them for mirror matches. This flaw in the "rain style" to me, means that amongst "good players" (note: not "great players" this is a different player group, the great players are exceptional exceptions), the players who create the standard of a metagame, sand will always be the more popular weather.
Now in this sand weather, Dory completely changes the dynamic of speed. scarfGarchomp can't touch it. scarfLatios can't touch it. Dory:
1) Doesn't need Choice Scarf.
2) Doesn't need to lock itself into attacks
3) Has bulk statistically almost on par with Machamp
4) Has room for a defensive item that makes it immune to its most common weakness, and a host of resistances
5) Packs Swords Dance
All while being faster than every single Scarf user in the game. Am I missing something, or is the above . . . well, insane?
Dory is perfectly built for late game sweep-- That speed, bulk, and sweeping power is extremely hard to touch, and while it might be lacking in coverage moves, but that weakness becomes more and more irrelevant with the passing of the game. Furthermore, Balloon removes problems with Spikes. If you thought Flygon was an amazing early-revenge-to-late-game-sweep transition poke in 4th Gen (it was AWESOME), Fory is everything Flygon was and so, so much more.
To me, excluding Balloon pokemon, the only really relevant counter is Gliscor-- Gliscor's presence is what makes Dory somewhat palatable. Unlike Skarmory, Gliscor actually has a means of killing last-poke-dory, and with Poison Heal, SR neutrality, and immunity to Spikes and sand, it has the durability to keep up with Dory the whole game.
Frankly though, I really don't think that Gliscor should be on every team just to beat Dory, nor do I think it's great that Gliscor is the go-to-check when Gliscor is already over-relied on to handle Lucario, Roob, Terrakion and Lando. Gliscor can fall to the "Double Ground/Fight" strategy, and unlike other sweepers that have a chance of being revengers, when Dory's path opens, it opens WIDE open.
