With all those common weaknesses, low hp, mediocre def, and lowish spe, he's going to have a hard time sweeping even if you do manage to set up 6 calm minds.
Forretress frequently carries earthquake and can just explode to screw your setup. Nattorei has leech seed, payback, toxic, and can also...