African Swallows, Unlike PO Servers, Are Non-Migratory

[Antar*]

tl;dr: We're moving to a new server, hopefully early next week. The only thing that should change for you guys is the Advanced Connect, but we're hopeful that the new server shouldn't have the lag issues of the last server or the disconnect issues we've been encountering this week


Many of you (especially staff members) have come to me complaining about the rash of mass disconnects currently plaguing our beloved PO server. I figure I should go ahead and tell you all what's going on.


As most of you remember, the computer currently hosting our PO server is *not* the same machine that was hosting it a little over a month ago. On that computer, our PO server was so badly plagued with lag that it was basically unusable for large chunks of the day (what some of you may remember is that, two months before that, we were on yet another server, which had the unfortunate habit of going offline for days at a time each month). In response to these lag issues, I moved us to MY OWN PERSONAL COMPUTER. Luckily for everyone, I built it to have the power of a gaming rig, and I don't even notice that you guys are on it, most of the time.

I anticipated issues with lockups and having to take PO offline while I did things like restart my computer or run updates. What I *didn't* really forsee was that I'd have issues with my home internet connection, which had been fairly reliable up to that point.

Not that I could probably ever get Comcast to admit it, but I'm pretty sure what's going on is that I'm being throttled as I rapidly approach my more-than-generous bandwidth cap of 250 GB/mo (in most months, I don't even tend to break 80. Right now, I'm past 230). That throttling, for some reason, is manifesting as my internet just up-and-giving out for a few seconds-to-a-minute at a time (it's not just PO--I find I can't even surf the web when this happens, and my wife says it's not just my computer). When that happens, some people are randomly disconnected, and when the internet comes back, the server lags as it tries to fix everything, and people overwhelm the server by trying to log back on (or so the theory goes).

Luckily, we just got a new dedicated PO server that's being configured as I type. It'll take me a few days to set up, but the plan is to migrate us over to the new server on Monday, April 2. All your accounts, passwords and ladder rankings will carry over. The ONLY thing that you'll have to do differently is to use a different "advanced connect," which we'll hopefully be able to set up as po.smogon.com (the advanced connect of the last server). I'll keep all of you posted. This new server is more powerful than the last server, but it isn't quite as powerful as my personal machine, so it's possible that lag *might* be an occasional issue. I'll go ahead and say this: if it's a MAJOR, CONSTANT issue, I have a fallback plan (hint: it involves not using PO).

If you have any questions, feel free to post them. In the meantime, enjoy the last two days of the month, and get ready for updated tiers, come April!

[cookie]

What's the airspeed velocity of an unladen PO server?

[firecape]

Did anyone else read the title as "Africans swallow, Unlike PO..." at first o_o (thanks Antar ;o)

[Lockeness]

Calculations for Antar's SWAG ladder ranking:

(New Server + Monty Python reference + No more random disconnects) x (The velocity of an African Sparrow) + (Even more Monty Python References)=

(16:32:31) *** Antar's rank in Standard SWAG is 1/51207 [9001 points / 3 PO Server Changes]!

[Antar*]

New server is online. Advanced connect: 173.233.95.2:5080

[Adam Lambert]

I appreciate it, Antar.

Originally Posted by Fat firecape

Did anyone else read the title as "Africans swallow, Unlike PO..." at first o_o (thanks Antar ;o)

I did...

[blarajan]

Thanks Antar. We all love you lots.

Lots.

[November Blue]

Awesome stuff Antar! Thanks for the advanced connect info, I use it all the time.

Got a question. What does a server migration involve? Is it just a file/program transfer from wherever the server was hosted before you took it?

[Antar*]

Originally Posted by Fat November Blue

Awesome stuff Antar! Thanks for the advanced connect info, I use it all the time.

Got a question. What does a server migration involve? Is it just a file/program transfer from wherever the server was hosted before you took it?

In this case, it involved configuring a new server from scratch, building a major software dependency from source, building PO from source, and moving all the configuration stuff / player databases over.

[Metal Bagon]

Originally Posted by Fat firecape

Did anyone else read the title as "Africans swallow, Unlike PO..." at first o_o (thanks Antar ;o)

Me...

So are the ladders reset or they stay the same?

[jonnyrolo]

As said in the OP they stay the same Metal Bagon. Also as of today I'm having trouble getting on the server. When I click on it in the server list or Advance Connect it brings up the chat window and says "Connected to the server!" but that is it. No user list, no password registration, etc so I came back to my PC after I ate and still its the same issue. Its not saying what the issue is either.

[Antar*]

Originally Posted by Fat jonnyrolo

As said in the OP they stay the same Metal Bagon. Also as of today I'm having trouble getting on the server. When I click on it in the server list or Advance Connect it brings up the chat window and says "Connected to the server!" but that is it. No user list, no password registration, etc so I came back to my PC after I ate and still its the same issue. Its not saying what the issue is either.

Could have sworn I responded to this.

I get this problem too, occasionally. Just hit "Exit" and try logging in again. At least, that works for me.

[Aurora.]

apologies for the bump

it's probably pretty obvious already, but we're having this same issue again
even where there's less than 150 players logged in, the lag is horrendous
and when i logged in during a 200 player session, i couldn't bear it
no idea why; i'm also getting the pink "the server isn't accepting any more connections" message occasionally

[Desolate]

That message along with a disconnection just means that our lag control system is enabled and you got unlucky and connected during a time when a good amount of others were. I'm usually the only one who uses this so I keep a close eye on it and if I see more problems than good arising from the server refusing connections then I disable it or lower the threshold. However getting back to your original point (sorry for the rant; the need to explain was bound to come up some time): there has been an influx of lag lately even when the player count is low, but remember player count is only one variable that affects the server.

[Aurora.]

Originally Posted by Fat Desolate

That message along with a disconnection just means that our lag control system is enabled and you got unlucky and connected during a time when a good amount of others were.

usually if i make another attempt to connect again it works, so probably...
the lag keeps fluctuating too; it drops for a little while occasionally - but for the majority of the time, it's terrible, no matter what connection speed i'm using

[Antar*]

Originally Posted by Fat RoryReloaded

it's probably pretty obvious already, but we're having this same issue again
even where there's less than 150 players logged in, the lag is horrendous
and when i logged in during a 200 player session, i couldn't bear it
no idea why

what do you mean by "horrendous?" Do you mean your battles were timing out? Or do you mean consistent 3-4 second lag, which is what *I've* been observing.

As far as I can tell, lag is tied to how many users are attempting to log in at a particular moment. When a user logs on (and potentially when a user logs off), PO has to send the updated list to all users of who's logged on and off. So when the player count is high, it's usually worse, but the real issue is how many users are logging on or off in a given stretch interval.

By limiting the number of users that are allowed to log on over a specified amount of time, we are able to limit the lag to a few seconds, on average.


If this isn't working, and lag is actually "horrendous" (vs. you and other users having unrealistic expectations of what constitutes acceptable lag), I'll have to look into other solutions.

[bearsfan092]

Originally Posted by Fat Antar

I have a fallback plan (hint: it involves not using PO).

Please tell me I'm not the only one hoping for the fallback plan. Outside of its team builder, I'm not fond of PO, as it seems to give a bunch of people (including but not limited to Smogon) server headaches. Also, PO's stubbornness in not adding the new damage formula information found by OmegaDonut, Kaphotics, and company is infuriating.

[Aurora.]

Originally Posted by Fat Antar

what do you mean by "horrendous?" Do you mean your battles were timing out? Or do you mean consistent 3-4 second lag, which is what *I've* been observing.

As far as I can tell, lag is tied to how many users are attempting to log in at a particular moment. When a user logs on (and potentially when a user logs off), PO has to send the updated list to all users of who's logged on and off. So when the player count is high, it's usually worse, but the real issue is how many users are logging on or off in a given stretch interval.

By limiting the number of users that are allowed to log on over a specified amount of time, we are able to limit the lag to a few seconds, on average.


If this isn't working, and lag is actually "horrendous" (vs. you and other users having unrealistic expectations of what constitutes acceptable lag), I'll have to look into other solutions.

at some point today, i was getting lag upwards of 7 seconds
again, referring to player count, this is when <150 players were logged in

[Aurora.]

at some point today, i was getting lag upwards of 7 seconds
again, referring to player count, this is when <150 players were logged in

edit: i had channel events on, didn't see too much login-logout fluctuation

[Antar*]

I'm closing this thread, which was started to discuss the migration and is NOT supposed to be a catch-all for complaints about PO.

Originally Posted by Fat RoryReloaded

at some point today, i was getting lag upwards of 7 seconds

Gosh! Seven whole seconds! The horror! To be clear: lag of 7 seconds is NOT "the same issue" that we had on the old server. Talk to me when we're having lag upwards of a minute and people's battles are timing out.

Originally Posted by Fat bearsfan092

Please tell me I'm not the only one hoping for the fallback plan.

The backup plan is to migrate to aesoft's Pokemon Showdown. But considering it lacks support for previous generations and (last I checked) Doubles/Triples, I'd rather not go this route.

Also, PO's stubbornness in not adding the new damage formula information found by OmegaDonut, Kaphotics, and company is infuriating.

The nice thing about PO is that it's open source*. PO didn't care about VGC timers either, but my personal fork of PO--the one our server is built from--supports this and a bunch of other differences (mostly battle log stuff, some zalgo text blocking). Really the only thing that's preventing our PO server from having the updated damage formula is that I haven't bothered to code it in yet.

*As is PS, just for the record