And this is precisely why SwagKey is overhyped. It is not as good as people say it is, because you are relying on luck to win. You're hoping the opponent will stay in, hit themselves with confusion or get full para and you hitting them with Foul Play. SwagKey is starting to feel like that set that a scrub throws at a more skilled player after getting thoroughly outplayed and says "here, I know you outplayed me, so I'm going to just throw this at you and throw caution to the wind. I might get a win out of this, but if I don't, I was going to lose anyway".
That being said
November Blue, SwagKey works best late game with multiple hazards down to reduce the luck needed to win (you still need to get lucky for SwagKey to "sweep"). Neither of which your opponent did. This is just another effect of the overhype of SwagKey: players think they can set it up whenever they want and autowin. And, yes, SwagKey does pretty terrible against stall (you don't even
need a cleric) :/ .