The effect is extremely minor, but if you, say, had an even amount of HP before EVs (so, 4 EVs bringing your HP to 303 instead of 302) you'd be unable to come in on Stealth Rock an additional time. If you had an odd amount of HP before EVs you'd just redistribute it to one of the defenses if you're not running a mixed attacker or another offense if you are (so, think Mega Latios wanting the 4 Attack on top of the 252/252 Attack and Speed). If the mon you're using has the same base Defense and Special Defense, it's basically always better to dump the 4 EVs into Special Defense (assuming you don't have to hit an odd HP number) so you don't have an unfavorable matchup against an uncommon Download user.
Basically, it just varies from mon to mon, there are cases where you may want to speed creep (so, while you'd normally see a SpDef Toxapex you may see a Bold, mixed-bulky Pex that runs a little bit of Speed investment to speed creep stuff, etc.) even if you're inherently tanky. And in the most standard situations, like the scenarios I listed above, it doesn't hinder you whatsoever to prepare for situations like that since they're so inconsequential in most situations but can potentially make the difference in some.