Though I agree with most of the points raised by the last posts, I think the "jerk" rivals being the more popular ones is correlation rather than causation. The rivals who are generally considered jerks for whatever reason tend to be the ones with the highest amount of story involvement: Blue is the Champion of the region and is pretty much the only familiar face you see at different locations in the region; Silver is Giovanni's son and has a special investment in the story of taking on Team Rocket as a result, often showing up at climactic parts of the story such as Radio Tower; Hugh is the catalyst for the entire story of BW2, with his personal antagonism toward Team Plasma putting you the player in situations where you find out what's really going on, such as when you and Cheren follow him onto the Team Plasma headquarters ship at the Pokémon World Tournament; and Gladion it goes without saying has heavy story involvement, originally being staged as a part of the enemy before later coming round to your side. Your exposition to all of these characters in situations that actually feel important to the player and the wider context of the adventure makes them a lot more memorable to players, as the characters are only one part of a larger, more significant whole -- and therefore the player is thinking about the interactions they had with those particular rival characters quite often throughout the story.
I think this theory serves to offer an explanation as to why Barry and to an extent Dawn are the most popular of the friendlier rivals. They also have really heavy story involvement, such as when the three of you team and split up to investigate one of the three lakes of the region each, and you actually fight in a multi battle WITH both of them at different points in the story (Dawn twice, and Barry once but right at the climax) which makes them extremely memorable and in some ways endearing.
Compare that with the friendly rivals. Brendan/May to my recollection never even meets the villainous team, and the extent of your interaction with them is "Hey [player]! Have you been filling out your PokéDex? Let me see!" phrased in various different ways throughout the adventure. Dawn has the same function, but then a lot more content and characterisation on top of it as a result of her further story involvement. This just leaves Brendan/May deathly forgettable, especially because the one way they could have saved the character -- giving them a truly challenging team that you battle at a climax to make for at least one really intense, memorable moment -- never happens and is given to Wally instead.
Wally himself has the makings of a good character but you only see him three times during the adventure. Once in the catching tutorial, which let's be real most players turn off for, which is where you get introduced to his defining character trait of "he's weak lol". Then you fight him outside the third gym, where the character development of "he's still weak lol but he's gonna TRAIN and BEAT YOU" occurs. And then you don't see him for 5 more badges and an entire evil team takedown until the end of Victory Road, where he gets the climactic rival battle for the adventure before the Champion. If we actually got to witness Wally's progression over the course of the adventure I think he would be a beloved fan favourite, his accruing of personal strength that he then bestows upon his team training them up too is an endearing story in concept. Hell, he could have even had story involvement, using saving the world from Team Magma/Aqua as his motivation for cultivating that personal growth -- because his growing strong has impacts that are far bigger than him. The issue is he had to try and share the spotlight with Brendan/May, and even though you only see him three times during the adventure he still overshadows his counterpart, which goes to show the missed potential.
Cheren in BW is portrayed really dubiously, half of the time as a kind-hearted friend and half of the time as a self-involved "IMMA BE THE CHAMPION :(" character, and I think it gets lost in translation. I accredit this to the heavy lathering of "truth vs ideals" the story writers tried to put into BW, where Cheren was supposed to represent one of them and Bianca the other (to this day I'm still unclear which is which and feel like Bianca actually encompasses both concepts better than Cheren). This made him a really weird mix of in some ways a jerk character, but you're already established childhood friends with him so you can't hate him because he has your back, you just don't like talking to him. Plus, the pacing with Cheren is extremely top-heavy -- most of your interaction with him occurs at the start of the story to a suffocating degree, and then it trails off until you fight him at Victory Road. I think he also turns up at Dragonspiral Tower and so has some story involvement, but he has no special stake in the events of the plot like Silver and Hugh do, nor does he actually accomplish anything by turning up there because he's accompanied by a team of Gym Leaders who fight Team Plasma for him.
Bianca from my experience is one of the more popular less friendly rivals because they actually gave her a full-bodied characterisation. Pokémon is a kids' game at heart and so even seeing fairly routine things like a teenager arguing with her parents can leave a lasting impression -- that NEVER happens in Pokémon games, the only other occasions of it happening being the extra piece of dialogue between Silver and Giovanni added in HGSS, and in Sun & Moon with Lusamine and her children.. Even more impressively is that both Bianca and her unnamed father get full character arcs, where the player has sympathy for both characters, but that the player witnessing the long-lasting effects of Bianca's Father's overprotectiveness still makes the player side with Bianca in the end. It's a really well done character arc that needed way better pacing to be effective. You can optionally visit Bianca's house right at the start of the adventure to have the character arc set up, and then the rest of the arc happens in one massive conversation in Nimbasa City. If they stretched it out over the course of the journey, I think Bianca would be a fan-favourite. As it is, it was just a missed opportunity, and Bianca has little involvement in the game outside of it.
However, a character who the other posts have left out is N. N is the real rival of BW, and is possibly the most popular rival in the history of the series. N also is not a jerk. He starts out on the opposing side, like Gladion, but from the player's very first meeting with them it is made very clear that N has an affinity for the player and not an antagonism. Sure he fights you, but because he wants to examine you as a trainer and hear your Pokémon's account of what it's like to be cared for by a truly kind trainer -- N frankly has no intention of beating you at this point, and it's unclear if he ever actually wants to beat you in a battle until Ghetsis' manipulation of him reaches even more dangerous levels. Eventually he also gets a full redemption arc. N is a shining example of a full-bodied, well-paced and -written character that doesn't have to be a jerk to be endearing, and is actually more endearing as a result of his initial victimhood, but also player relatability as a result of his express compassion toward Pokémon, combined with his development of inner strength over the story. It also goes without saying that he has heavy plot involvement, being actually the most central figure to the story arguably moreso than Ghetsis.
XY rivals were caricatures and that doesn't work for the character who you're meant to have grown with over the course of the journey, because none of them get any growth. At the start of the journey Trevor sets out to complete his PokéDex and then by the end he is still trying to complete it. Dancing guy likes dancing at the start of the adventure and by the end of it likes dancing. Serena has a little bit of plot relevance -- she's there when the player receives mega-evolution, and was originally going to be the only character of the 5 who ended up with them to receive a Pokémon but the player's mum is famous so Prof Sycamore just decided to give out 4 more rare Pokémon. Shauna has the closest thing to a character arc, but she has Wally syndrome of where you only see the checkpoints of the arc and not the actual work and living that went into reaching those checkpoints, plus the overall conclusion is a little ambiguous in what it actually is. They were just written badly, and had little to do with Team Flame (who are admittedly one of the weaker teams anyway which gave the rivals an even harder attempt to stand out).
Hau is, as others said, just a wet blanket. You see him eating enchiladas sometimes, and he battles you once in a blue moon. No further progression, no real plot involvement other than that he's the son of Hala but their relationship is never explored. The game would be no different if Hau wasn't in it -- have the player character fight Lillie instead and change her characterisation a bit, and it would be exactly the same. That's not what you want out of a rival (I'd like to point out this is also true for Brendan/May or Wally, Cheren, and 3 of the 4 XY rivals).
tl;dr So I think that's your answer. It's actually little to do with whether the character is a jerk or not, but rather just how involved the character is to the plot and/or the player. I think the original premise was faulty because Barry and N are as popular as the 4 jerk rivals we've had and they're not jerks, but I agree that in general the nice rivals have the most potential to be forgotten about. I think it could be put down to nothing more than the more time and investment that's being put into the story of the game as a whole, the more is also being put into the rival, and making your rival(s) just a nice friend who doesn't do much is almost the default for games which haven't had much focus on plot. This means that there aren't as many climaxes for those rivals to be involved in, or that the climaxes they are involved in the player simply cares about less because the plot has been less enticing as a whole, which leads to fewer pressurised environments with which to develop a true relationship between the player and rival characters. All of the 6 rival characters who are head and shoulders above the others (Blue, Silver, Barry, N, Hugh, and Gladion) have specific memorable climaxes over the course of the journey that stay with the player. The other rivals don't have those, with the exception of perhaps Wally and Bianca, where the issue is moreso that the player hasn't seen the events leading up to the climax and so simply cares less.
Edit: Apparently Hau likes malasadas not enchiladas and dancing guy who likes dancing is called Tierno. Who knew, who cared. That's very much case in point.
