@tbnuser said:
The recovery wasn't too abysmal thankfully.
Just need to wait for the network to settle down now though, getting anywhere between 600-1000ms on the first hop running mtr to google
Oof, yeah definitely need the network to settle down in LA. Right now it's bouncing between normal 10-20ms latency, spiking up to 250-300ms latency, or rocketing up to 1000-3000ms latency with random timeouts. It'll be fine for 10 minutes, then a couple mins of barely reachable. It was pretty solid last night into the early morning, with none of those problems, I was thinking how smoothly switching IPs had gone for me, this is probably my fault for jinxing it haha
@bakageta said: It was pretty solid last night into the early morning, with none of those problems
What nodes? That's important. Last night into early morning it was smooth because we made some changes, now I just need to know which nodes had problems at what time so I can go through the monitors.
@bakageta said: It was pretty solid last night into the early morning, with none of those problems
What nodes? That's important. Last night into early morning it was smooth because we made some changes, now I just need to know which nodes had problems at what time so I can go through the monitors.
Oh right, that might help eh? I noticed it on LAXA023, taking a quick look at the rest of mine and I'm not seeing any issues on any other nodes personally.
(edit)
And this is why I didn't mention it in the first place, because it seems to have calmed down now and is behaving normally.
@VirMach Please check that the computer being migrated from Denver 02 to LAXA005 has power. The new IP is displayed, but the hard drive is not. The machine won't boot and can't connect to VNC.
@bakageta said: It was pretty solid last night into the early morning, with none of those problems
What nodes? That's important. Last night into early morning it was smooth because we made some changes, now I just need to know which nodes had problems at what time so I can go through the monitors.
Oh right, that might help eh? I noticed it on LAXA023, taking a quick look at the rest of mine and I'm not seeing any issues on any other nodes personally.
(edit)
And this is why I didn't mention it in the first place, because it seems to have calmed down now and is behaving normally.
I do see what you see around that time, maybe half an hour before your post. Unrelated to the first issue, it was just some network abuse (maybe?) Someone sent out like 500,000 PPS / 6Gbps for maybe an hour. I'll go spank him later.
It does make me concerned about the switch though, it should've handled that better. Adding it to my list.
If I ever end up going to the datacenter for the Seattle servers... I'll also plan out finally finalizing the cabinet in LAX. It was never meant to be on this switch for this long, maybe I'll take down a QFX5100 & EX4300 or something. I have EX4300's everywhere around the office, I think I bought a few too many.
@VirMach said:
If I ever end up going to the datacenter for the Seattle servers... I'll also plan out finally finalizing the cabinet in LAX. It was never meant to be on this switch for this long, maybe I'll take down a QFX5100 & EX4300 or something. I have EX4300's everywhere around the office, I think I bought a few too many.
The new IP that was already in effect dropped out.
After checking, the original correct ens3 in /etc/network/interfaces was rewritten to eth0, I manually changed it back to ens3 and the server came back online immediately.
It happened on servers LAXA009 and LAXA026 successively.
I am using Debian 11.
Have the honor of being the crybaby who pays $20 for a 128MB VPS at VirMach in 2023.
@FAT32 said:
Sometimes I always wonder why unfortunate things happen to good people. Although VirMach is doing a business here, but a lot of times I feel like it is a charity, things just broke randomly and VirMach still tried his best to keep things afloat.
When VirMach migrate away from CC, a lot of people say VirMach probably cannot survive with those insane deals (come on, $0.95/yr with IPv4? That cannot even cover the cost of IP), but yet they did. Now when DediPath encountered issue, VirMach still managed to resolve it so quickly and everything seems to be smooth on my side as a user.
Sure, there are times where support can be slow or poor, but based on my ~6-7 years of usage I don't really encounter major hiccups. Even with all the inflation, as far as I know VirMach never increased recurring price of any services. (Keep in mind the pricing are rock-bottom, literally unsustainable)
Thanks VirMach, just want to show my appreciation here after following probably thousand pages of forum posts (OGF/LES), hope that one day everything can back to normal and we can see VirMach 10th anniversary deals
I bought my first ever VPS from VirMach in 2017. Had only used shared hosting before that. Then I got introduced to insane VirMach deals on BF/Cyber Monday. Sometimes I think Sohail is a very rich guy and he just wants to keep us happy that is why he is still going.
LAX2Z019 update, kind of. The situation didn't improve at all, and at this point it's basically 0% chance the NVMe can be brought back to life. The capacitors flew off the PCIe riser so there was probably a voltage spike, and another capacitor on the NVMe looks pretty bad. Going to call it later today, and we'll get back up the portion unaffected and recreate the rest.
Chicago2 is routed properly now. Still waiting on Tampa.
@VirMach
My VPS on node "RYZE.LAX-A026.VMS" not received a new IP address. The SolusVM panel listing only one old: "47.87.153.XX" (ZenLayer / Alicloud(?)) IP address. Is this a mistake? or normal?
I ask it only because if I understood correctly, the new IP assigning process on all LAX VM's has been ended (Networks status page info). Sorry, if I understood something wrong. Thx for the answer!
@titus said: @VirMach
My VPS on node "RYZE.LAX-A026.VMS" not received a new IP address. The SolusVM panel listing only one old: "47.87.153.XX" (ZenLayer / Alicloud(?)) IP address. Is this a mistake? or normal?
I ask it only because if I understood correctly, the new IP assigning process on all LAX VM's has been ended (Networks status page info). Sorry, if I understood something wrong. Thx for the answer!
That's one of the nodes that ran out and need to be moved around, you'll get a separate notification and/or it'll be posted on network status page. We haven't got to that yet but trying to do it in a way where you still get some notice. So you're part of the lucky 2%~ of people affected by that. Doesn't go by node for that, but probably like a dozen people on some nodes, something like that.
Tampa new IPv4 ready. ⌛ You may manually change this to your main IPv4 address in the full VPS control panel early, or continue using the old IPv4 address until September 29th.
My IP was not pinging and I reconfigured yesterday so I went to reconfigure again.... now VNC not loading (Starting VNC handshake), Status Unknown, Operation Timed Out After 90001 Milliseconds With 0 Bytes Received. Will re-check back in like 20 minutes to see what have I done and maybe it's SolusVM feature. :-D
Few minutes later server rebooted, but no network access on new IP (-:
[Mod edit: Now that it is fixed, I have removed the screenshot showing Jab's IP addresses.]
@VirMach said:
LAX2Z019 update, kind of. The situation didn't improve at all, and at this point it's basically 0% chance the NVMe can be brought back to life. The capacitors flew off the PCIe riser so there was probably a voltage spike, and another capacitor on the NVMe looks pretty bad. Going to call it later today, and we'll get back up the portion unaffected and recreate the rest.
Chicago2 is routed properly now. Still waiting on Tampa.
@Virmach: For machines that used to have more than one IP address but are now only allocated one is this a ticket occasion? It's not an issue for me as I wasn't currently using it but wanted to make sure there wasn't an automatic process which was going to follow up if I did nothing.
Comments
I'm betting on 141.11.xxx.xx
For staff assistance or support issues please use the helpdesk ticket system at https://support.lowendspirit.com/index.php?a=add
The recovery wasn't too abysmal thankfully.
Just need to wait for the network to settle down now though, getting anywhere between 600-1000ms on the first hop running mtr to google
(changed) ✅ Tampa has new IPv4 assigned but ❌ it has routing issues, we are waiting on the datacenter. Do not switch to new IPv4 yet.
Seems like my complaining worked, this is nice middle ground I would say
Haven't bought a single service in VirMach Great Ryzen 2022 - 2023 Flash Sale.
https://lowendspirit.com/uploads/editor/gi/ippw0lcmqowk.png
Oof, yeah definitely need the network to settle down in LA. Right now it's bouncing between normal 10-20ms latency, spiking up to 250-300ms latency, or rocketing up to 1000-3000ms latency with random timeouts. It'll be fine for 10 minutes, then a couple mins of barely reachable. It was pretty solid last night into the early morning, with none of those problems, I was thinking how smoothly switching IPs had gone for me, this is probably my fault for jinxing it haha
What nodes? That's important. Last night into early morning it was smooth because we made some changes, now I just need to know which nodes had problems at what time so I can go through the monitors.
Oh right, that might help eh? I noticed it on LAXA023, taking a quick look at the rest of mine and I'm not seeing any issues on any other nodes personally.
(edit)
And this is why I didn't mention it in the first place, because it seems to have calmed down now and is behaving normally.
Where are the Atlanta Club fans? :-|
@VirMach Please check that the computer being migrated from Denver 02 to LAXA005 has power. The new IP is displayed, but the hard drive is not. The machine won't boot and can't connect to VNC.
I do see what you see around that time, maybe half an hour before your post. Unrelated to the first issue, it was just some network abuse (maybe?) Someone sent out like 500,000 PPS / 6Gbps for maybe an hour. I'll go spank him later.
It does make me concerned about the switch though, it should've handled that better. Adding it to my list.
If I ever end up going to the datacenter for the Seattle servers... I'll also plan out finally finalizing the cabinet in LAX. It was never meant to be on this switch for this long, maybe I'll take down a QFX5100 & EX4300 or something. I have EX4300's everywhere around the office, I think I bought a few too many.
give us one EX4300, we are missing one. LOL
George Datacenter LLC
www.georgedatacenter.com
Owner Hardware
Looks like RYZE.LAX-A022.VMS is stable again, it was a bit on and off yesterday
The new IP that was already in effect dropped out.
After checking, the original correct ens3 in /etc/network/interfaces was rewritten to eth0, I manually changed it back to ens3 and the server came back online immediately.
It happened on servers LAXA009 and LAXA026 successively.
I am using Debian 11.
Have the honor of being the crybaby who pays $20 for a 128MB VPS at VirMach in 2023.
man, virmach update is better than sex
Fuck this 24/7 internet spew of trivia and celebrity bullshit.
It's all about the anticipation.
I bought my first ever VPS from VirMach in 2017. Had only used shared hosting before that. Then I got introduced to insane VirMach deals on BF/Cyber Monday. Sometimes I think Sohail is a very rich guy and he just wants to keep us happy that is why he is still going.
Fast as fuck Core i9 VPS (aff) | Powerful AMD Ryzen VPS (aff)
Hello, haven't you made a push ticket recently?The push I did last Thursday is not finished yet.
lol.
On sad note: Tampa IP addresses still not routable worky
Haven't bought a single service in VirMach Great Ryzen 2022 - 2023 Flash Sale.
https://lowendspirit.com/uploads/editor/gi/ippw0lcmqowk.png
From what I can tell, there are still problems with:
RYZE.LAX-A007.VMS
TPAZ004.VIRM.AC
Stop the planet! I want to get off!
MIAZ011 has both iPs usable, but it seems that they have different routing policy.
The destination is TunnelBroker.net Miami router.
The old IPv4 can reach there directly through FL-IX.
The new IPv4 takes a detour through Ashburn.
Accepting submissions for IPv6 less than /64 Hall of Incompetence.
LAX2Z019 update, kind of. The situation didn't improve at all, and at this point it's basically 0% chance the NVMe can be brought back to life. The capacitors flew off the PCIe riser so there was probably a voltage spike, and another capacitor on the NVMe looks pretty bad. Going to call it later today, and we'll get back up the portion unaffected and recreate the rest.
Chicago2 is routed properly now. Still waiting on Tampa.
@VirMach
My VPS on node "RYZE.LAX-A026.VMS" not received a new IP address. The SolusVM panel listing only one old: "47.87.153.XX" (ZenLayer / Alicloud(?)) IP address. Is this a mistake? or normal?
I ask it only because if I understood correctly, the new IP assigning process on all LAX VM's has been ended (Networks status page info). Sorry, if I understood something wrong. Thx for the answer!
That's one of the nodes that ran out and need to be moved around, you'll get a separate notification and/or it'll be posted on network status page. We haven't got to that yet but trying to do it in a way where you still get some notice. So you're part of the lucky 2%~ of people affected by that. Doesn't go by node for that, but probably like a dozen people on some nodes, something like that.
My IP was not pinging and I reconfigured yesterday so I went to reconfigure again.... now VNC not loading (Starting VNC handshake), Status Unknown, Operation Timed Out After 90001 Milliseconds With 0 Bytes Received. Will re-check back in like 20 minutes to see what have I done and maybe it's SolusVM feature. :-D
Few minutes later server rebooted, but no network access on new IP (-:
[Mod edit: Now that it is fixed, I have removed the screenshot showing Jab's IP addresses.]
Haven't bought a single service in VirMach Great Ryzen 2022 - 2023 Flash Sale.
https://lowendspirit.com/uploads/editor/gi/ippw0lcmqowk.png
@Jab fixed
Can confirmed, worky worky, much many thanks!
Haven't bought a single service in VirMach Great Ryzen 2022 - 2023 Flash Sale.
https://lowendspirit.com/uploads/editor/gi/ippw0lcmqowk.png
LMAO what drive + riser model was that
👍
@Virmach: For machines that used to have more than one IP address but are now only allocated one is this a ticket occasion? It's not an issue for me as I wasn't currently using it but wanted to make sure there wasn't an automatic process which was going to follow up if I did nothing.
EDIT: Nevermind, I'm an idiot.