I can see a quick rebut to that coming: they haven't been abandoned. Gen 8 has given us a slew of them, from species like Obstagoon and Cursola in Galar to the likes of Wyrdeer, Sneasler, and Overqwil in Hisui, and Gen 9 will probably give us more. Cool. But they don't really count. At least, not for what I mean. In conceptual terms it's pretty plain to me that those Pokemon are considered regional variants first, cross-gen evolutions second.
What I'm talking about is the cross-gen evolutions Gen 2, 3, 4, and 6 gave us. Just straightforward "this Pokemon has a new relative now" add-ons like Crobat, Kingdra, Azurill, and Yanmega.
I would actually personally argue in terms of the Gen 8 evolutions that Wyrdeer, Kleavor, and Ursaluna count as "regular" cross-gen evolutions in the same vein that Gen 2 and 4's cross-gen evolutions are. Those three in particular are by all means "this Pokemon has a new relative" add-ons to old Pokemon, as they are not evolutions of a regional version of Stantler, Scyther, and Ursaring, but rather a straight up new evolved form of the Stantler, Scyther, and Ursaring we know. I can understand what you're getting at with most of the Gen 8 evolutions not necessarily being in the same boat as older cross-gen evolutions, and for the vast majority of them, that's true that they are regional variants. But there are three who still more or less are in the same boat of "this Pokemon has a new relative now" add ons in the same boat as the likes of Kingdra or Electivire: Wyrdeer, Kleavor, and Ursaluna are more or less that because they are the evolved forms of the original mon themself, and are pretty much the only three "traditional" cross-gen evolutions that have been introduced in years. Granted, it's a small amount compared to Gen 4, but it's something.
Considering one of them is based on using a move, and two of them are item based, it's likely they will remain as permanent new members of their evolutionary lines for all of their future appearances, and I imagine Psyshield Bash will remain a part of Stantler's moveset from here on and the Black Augurite and Peat Block will return in any game that has Scyther and Teddiursa. Granted, the method to evolve them may change, in that Scyther will need to be traded holding the Black Augurite to become Kleavor in a traditional mainline title, and Ursaluna may become "level-up Ursaring while holding a Peat Block under a full moon", and Stantler may just become "level up while knowing Psyshield Bash", but I imagine they will always have the ability to evolve into these new forms no matter what.
Part of it may have to do with the overall shift in design ethos. Lots of people thought that Alomomola would be a Luvdisc evolution but of course it couldn't have been, because part of Gen 5's modus operandi was to break with the previous gens entirely and do its own thing. And when you're shifting regions it makes sense to come in with a new design team, who will have from-scratch concepts rather than building on the old ones.
Gen 2 was designed as a sequel to Gen 1 so naturally it added on. Also, there are less "filler" Pokemon now, in that even weak or single-stage Pokemon tend to have some sort of niche to make them stand out. So there's less need for evolutions.
But still. It's just odd comparing the huge amount of cross-gen evos from earlier gens to now. Every time that the new games arrive and get stripped down by hackers people tend to ask "any new evolutions?" on leak threads and while yes, Scarlet and Violet might bring new evolutions they'll almost certainly be regional ones. But wouldn't it be cool if they just unexpectedly threw something like Thievul or Pincurchin a new evolution too?
I think the fact that Gen 5 itself introduced the Eviolite may also play a factor as to why they haven't introduced any straight up cross-gen evolutions since. Aside from BW1's gimmick being that it is an extremely self-contained experience with just its new roster of mons, the Eviolite is coded to be compatible with Pokemon who can still evolve. Obviously this applied to every previously existing cross-gen mon at the time since by the time of Gen 5, every mon who had a cross-gen evolution was already recoded to be not fully evolved in a prior generation so were instantly compatible with the Eviolite from the start. However, with the advent of this item, this means any mon who was already fully evolved by that point was programmed as "fully evolved" and thus incompatible with Eviolite, and I am not sure how the coding for it works but they might not have wanted to mess with the coding in the scenario that a Pokemon coded as fully evolved gains a new evolution and thus has to be recoded as not fully evolved and thus Eviolite compatible. Maybe they also had apprehensions about how a Pokemon gaining the ability to use Eviolite would have an effect on and influence their viability in the competitive metagames. If a mon who was fully evolved at one point suddenly now has the ability to boost its own defenses, that might be something they feared would make some of them too strong. Of course, this is all speculation, and any possibility could arise, but I do think the Eviolite item existing may have played a part in why fewer traditional cross-gen evolutions have been a thing since: after all, the only one after that in Gen 6 was Sylveon, who was an alternate evolution to one of Eevee's many evolved form, and there was no need to recode a mon to NFE since Eevee was already capable of evolving anyway, and only recently have they bucked the trend by bringing three traditional evolutions with Wyrdeer, Kleavor, and Ursaluna.
Moreover, I also get the feeling that after Gen 2, having cross-gen evolutions in Gen 4 was a thing because that was really one of Gen 4's gimmicks and something they felt was a distinct part of Gen 4's identity. The fact that Gen 4 introduced so many of them and they comprised the bulk of the dex makes me convinced that in addition to the gender differences thing, the new evolutions was a Gen 4 thing mainly and they felt that that's something that makes DPP unique to itself: after all, it was in the region that has a Professor who studies evolution, thus making it a part of Gen 4's gimmicks.
I also wouldn't assume we'd get regional evolutions in Gen 9 either. That seems to be a focus of every other generation mainly, as the only ones that did it in any capacity are 2, 4, and 8 (and 6 I guess if you count Sylveon). The fact that we got regional evolutions in Galar likely was a push towards hinting the existence of a new Sinnoh game (ie PLA) coming next after SwSh, and I don't think that is guaranteed to carry over to Scarlet and Violet. I do think we will get new regional families but I wouldn't be so quick to assume that we'll see new evolved forms to Gen 9's regional forms just yet, they might even try something different considering their weird and bizarre philosophy with rotating ideas and reinventing the wheel ad infinitum with every generation.