LAXA017 had a slightly different configuration than others, so it's not playing well with the network changes and causing heavy packet loss to nearby servers LAXA018 and LAXA019 as well. Having them attach KVM and should be corrected relatively painlessly after that.
It is possible for it to be complicated further if the reason for it being different than the others is something I recall correctly, in which case this node may need a motherboard swap or at the very least further attempts at BMC re-flash. In my brain though that was either LAXA027 or LAXA017, so we'll know in about half an hour.
@host4cheap said:
Is there any ETC for DALZ004/DALZ007 (Network status says - waiting for equipment release.)
It's all waiting on one guy at Flexential that's apparently the only person that can deal with the problem, and he's so far decided that he doesn't very much like doing anything about it.
@cornercase said:
So I guess my SJCZ005 was moved to LAXA007 however currently shows up with no OS installed. Should I be expecting a backup to load in here?
I don't believe that should have happened. Make a ticket so I can check.
MY MISTAKE! It was originally at DENZ001 not SJC.
Still curious if there will be a backup restore coming.
@cornercase said:
So I guess my SJCZ005 was moved to LAXA007 however currently shows up with no OS installed. Should I be expecting a backup to load in here?
I don't believe that should have happened. Make a ticket so I can check.
MY MISTAKE! It was originally at DENZ001 not SJC.
Still curious if there will be a backup restore coming.
Power down, and power back up. I'd need to know how exactly it shows with no OS installed to answer you further after that.
@cornercase said:
So I guess my SJCZ005 was moved to LAXA007 however currently shows up with no OS installed. Should I be expecting a backup to load in here?
I don't believe that should have happened. Make a ticket so I can check.
MY MISTAKE! It was originally at DENZ001 not SJC.
Still curious if there will be a backup restore coming.
Power down, and power back up. I'd need to know how exactly it shows with no OS installed to answer you further after that.
@cornercase said:
So I guess my SJCZ005 was moved to LAXA007 however currently shows up with no OS installed. Should I be expecting a backup to load in here?
I don't believe that should have happened. Make a ticket so I can check.
MY MISTAKE! It was originally at DENZ001 not SJC.
Still curious if there will be a backup restore coming.
Power down, and power back up. I'd need to know how exactly it shows with no OS installed to answer you further after that.
Done and no change. Ticket submitted.
Thanks
You had an old ISO mounted. We've been having lot of people not checking this, a lot and we've already added it to the reset button. I was going to tell you to use the reset button initially but told you the power down and power on specifically to make sure I could ask you this next series of questions.
How can we improve this further to where you do not miss that? When you initially saw it was offline, you just used the power on button instead, right? Or rather, when you were told to power it down, and then power it on, which is intentionally the wrong way of doing it, did you notice the message:
The virtual server has been booted successfully. If that did not work, make sure to unmount any ISO.
Or were you just doing all this through the full control panel?
@VirMach said:
How can we improve this further to where you do not miss that? When you initially saw it was offline, you just used the power on button instead, right? Or rather, when you were told to power it down, and then power it on, which is intentionally the wrong way of doing it, did you notice the message:
The virtual server has been booted successfully. If that did not work, make sure to unmount any ISO.
Or were you just doing all this through the full control panel?
I didn't pay attention to that message because there shouldn't be an ISO mounted. I have booted and rebooted this VM in the past (since I first set it up) and there has never been any issue with an ISO (either because one wasn't mounted or it wasn't first in the boot order). Maybe change the message to indicate that the boot order and/or presence of an ISO may have changed without the customer's intervention.
@VirMach said:
How can we improve this further to where you do not miss that? When you initially saw it was offline, you just used the power on button instead, right? Or rather, when you were told to power it down, and then power it on, which is intentionally the wrong way of doing it, did you notice the message:
The virtual server has been booted successfully. If that did not work, make sure to unmount any ISO.
Or were you just doing all this through the full control panel?
I didn't pay attention to that message because there shouldn't be an ISO mounted. I have booted and rebooted this VM in the past (since I first set it up) and there has never been any issue with an ISO (either because one wasn't mounted or it wasn't first in the boot order). Maybe change the message to indicate that the boot order and/or presence of an ISO may have changed without the customer's intervention.
ANYWAY: There is still no OS present...???
Installed OS: None Detected
None detected means none detected, not none.
It means the system doesn't know what's on there. This is most common when an ISO is mounted to install the OS, or an ISO was mounted but never used and SolusVM lost track of the original operating system name before the ISO mount. I've added more tooltips everywhere to improve explanations.
You had an ISO mounted from a long long time ago, and it doesn't exist anymore so it was trying to mount it still on reboot, you probably would not have noticed that previously if the service has been online since then or if the old ISO still existed on DENZ001 but not on LAXA007.
I do see this is on "hard disk only" for boot order, but I don't think that matters, this is based on SolusVM's code, not logic.
Are the new IP's for LAX that got made available today not routing yet?
45.15.143.1 seems to hit 10.254.0.7 IP when tracerouting (which is strange, I didn't think you could see a 10.x.x.x on the public Internet)
@VirMach said:
It means the system doesn't know what's on there. This is most common when an ISO is mounted to install the OS, or an ISO was mounted but never used and SolusVM lost track of the original operating system name before the ISO mount. I've added more tooltips everywhere to improve explanations.
You had an ISO mounted from a long long time ago, and it doesn't exist anymore so it was trying to mount it still on reboot, you probably would not have noticed that previously if the service has been online since then or if the old ISO still existed on DENZ001 but not on LAXA007.
I do see this is on "hard disk only" for boot order, but I don't think that matters, this is based on SolusVM's code, not logic.
Got it! I made it to a place where I could VNC in and it appears the IP (which is still the IP from DENZ001) is not being routed.
So is this expected and we are now just waiting for new IP addresses?
@cornercase said: So is this expected and we are now just waiting for new IP addresses?
Correct, I honestly don't know why it's not done yet, but I'm not the team member working on it. I may have to take over, we'll see.
@erk said:
Are the new IP's for LAX that got made available today not routing yet?
45.15.143.1 seems to hit 10.254.0.7 IP when tracerouting (which is strange, I didn't think you could see a 10.x.x.x on the public Internet)
Seattle --> LAX pushed further, likely going in a few hours. I had to spend a good amount of time on the packet loss issue and SolusVM API going slowly on the changes, and still need to wrap up other locations. LAX is almost out of the way though and that's the bulk of the changes. IP changes may have an intermission where I finally take the equipment down, we'll see how it goes.
Good news:
Flexential has finally responded and sent the form to begin getting equipment back out of Dallas and Denver, but it seems like they might still drag out the rest of the process.
Sabey is sending equipment to San Jose from Seattle.
Additional IPv4 leased, since we used Seattle's reserved IPv4 for the new cabinet in LAX and are doing both transfers still.
i have a vps at DENZ001 before, yesterday i found that it changed to LAXA007 now, But the ip is not be changed, still the same as the DENZ001 before, and i tested that this ip is empty route, is this normal?
@gritty I'm in the same boat with you. I expect that they are waiting for IP blocks to start routing before assigning them to our VMs on LAXA007. Should happen soon™ Of course maybe Qi forgot all about us, maybe.
It's 23rd today, I thought we could use new blocks from IPXO instead of the 47.87 one starting yesterday.
Found 2 IPs, 47.87.x.x and 89.33.x.x
Changed Main IP from 47 to 89 from Solus, and clicked reconfigure network. Boom, lost all connectivity.
Am I doing something wrong?
@tuc said:
Yet not seeing any new IPs as of now. Am I out of the boat?
IIRC you are on LAXA031, I also am on the node. Have a new IPv4, switched over early, does not route. So you are not behind at all.
I don't think they are up to LAXA031 yet. Status page shows LAXA004, LAXA005, LAXA006, LAXA007 are confirmed ready.
Still here, not exactly going well in terms of how long it's taking for various reasons, mainly more weird SolusVM changes they did in the background on whatever version. Same update that caused problems with migrations seem to cause something similar for IP changes, but only when it gets switched to the main IP address too fast.
Getting close to done for LAX1, as in it's been "done" in terms of the actual routing for a while, just working on the part where it switches main IP and removes the old one.
In any case I'm going to shift focus to other locations now so people can at least do the change, and there are definitely still some routing issues we have to wait on the DC for in some locations for some blocks because RPKI wasn't 100% ready for some when we requested them and I don't think they were revisited.
Taking servers down to LAX unfortunately getting pushed back again, but I'm simultaneously working on LAX2Z019 right now and trying to take it down with the batch, as well as NYCB027 so we can close out those other issues together, so hopefully it's worth the extra wait. Since it's the weekend now everything else logistics-wise should be easier, we don't have a ton of people using the elevators here, my car won't be blocked in, and less traffic the way I'm going.
Oh unfortunate side note, I did snap my glasses in half and I can't find the superglue, lots of squinting but I do need to figure that one out.
I already migrated one of my servers to new IP address because my uptime notification notified me so. I have one more server that it isn't migrated yet cuz' I need to point the DNS of the server to the new IP first. Can I temporarily "run" two IP addresses to the single machine for now until the DNS changes fully propagates?
@ricANNArdo said:
I already migrated one of my servers to new IP address because my uptime notification notified me so. I have one more server that it isn't migrated yet cuz' I need to point the DNS of the server to the new IP first. Can I temporarily "run" two IP addresses to the single machine for now until the DNS changes fully propagates?
Both the old and the new will be available to use for the rest of the week, but this is completely based on whether or not SolusVM made any other changes to prevent that from happening, which is possible.
You'd have to go in manually though and add in your old IP address., since once it's technically done it would otherwise be removed from SolusVM. If you do have issues and you need the old one added back in an emergency situations, we could force it back in but I don't know how busy we'll be with everything else to be able to process the request.
Comments
LAXA017 had a slightly different configuration than others, so it's not playing well with the network changes and causing heavy packet loss to nearby servers LAXA018 and LAXA019 as well. Having them attach KVM and should be corrected relatively painlessly after that.
It is possible for it to be complicated further if the reason for it being different than the others is something I recall correctly, in which case this node may need a motherboard swap or at the very least further attempts at BMC re-flash. In my brain though that was either LAXA027 or LAXA017, so we'll know in about half an hour.
Is there any ETC for DALZ004/DALZ007 (Network status says - waiting for equipment release.)
It's all waiting on one guy at Flexential that's apparently the only person that can deal with the problem, and he's so far decided that he doesn't very much like doing anything about it.
MY MISTAKE! It was originally at DENZ001 not SJC.
Still curious if there will be a backup restore coming.
Power down, and power back up. I'd need to know how exactly it shows with no OS installed to answer you further after that.
Done and no change. Ticket submitted.
Thanks
You had an old ISO mounted. We've been having lot of people not checking this, a lot and we've already added it to the reset button. I was going to tell you to use the reset button initially but told you the power down and power on specifically to make sure I could ask you this next series of questions.
How can we improve this further to where you do not miss that? When you initially saw it was offline, you just used the power on button instead, right? Or rather, when you were told to power it down, and then power it on, which is intentionally the wrong way of doing it, did you notice the message:
Or were you just doing all this through the full control panel?
I didn't pay attention to that message because there shouldn't be an ISO mounted. I have booted and rebooted this VM in the past (since I first set it up) and there has never been any issue with an ISO (either because one wasn't mounted or it wasn't first in the boot order). Maybe change the message to indicate that the boot order and/or presence of an ISO may have changed without the customer's intervention.
ANYWAY: There is still no OS present...???
None detected means none detected, not none.
It means the system doesn't know what's on there. This is most common when an ISO is mounted to install the OS, or an ISO was mounted but never used and SolusVM lost track of the original operating system name before the ISO mount. I've added more tooltips everywhere to improve explanations.
You had an ISO mounted from a long long time ago, and it doesn't exist anymore so it was trying to mount it still on reboot, you probably would not have noticed that previously if the service has been online since then or if the old ISO still existed on DENZ001 but not on LAXA007.
I do see this is on "hard disk only" for boot order, but I don't think that matters, this is based on SolusVM's code, not logic.
Are the new IP's for LAX that got made available today not routing yet?
45.15.143.1 seems to hit 10.254.0.7 IP when tracerouting (which is strange, I didn't think you could see a 10.x.x.x on the public Internet)
Got it! I made it to a place where I could VNC in and it appears the IP (which is still the IP from DENZ001) is not being routed.
So is this expected and we are now just waiting for new IP addresses?
Correct, I honestly don't know why it's not done yet, but I'm not the team member working on it. I may have to take over, we'll see.
Check network status page for updates.
When doing that make sure you remember you're on LAX2, not LAX.
Seattle --> LAX pushed further, likely going in a few hours. I had to spend a good amount of time on the packet loss issue and SolusVM API going slowly on the changes, and still need to wrap up other locations. LAX is almost out of the way though and that's the bulk of the changes. IP changes may have an intermission where I finally take the equipment down, we'll see how it goes.
Good news:
Is it possible to request 1 Dallas VM to be moved to LAX or upcoming OKC instead of NYC
i have a vps at DENZ001 before, yesterday i found that it changed to LAXA007 now, But the ip is not be changed, still the same as the DENZ001 before, and i tested that this ip is empty route, is this normal?
@gritty I'm in the same boat with you. I expect that they are waiting for IP blocks to start routing before assigning them to our VMs on LAXA007. Should happen soon™ Of course maybe Qi forgot all about us, maybe.
For staff assistance or support issues please use the helpdesk ticket system at https://support.lowendspirit.com/index.php?a=add
Make the same wish.
Have the honor of being the crybaby who pays $20 for a 128MB VPS at VirMach in 2023.
@FrankZ thanks for letting me know that I am not alone😂
maybe it must be patient enough to use Virmach service
It's 23rd today, I thought we could use new blocks from IPXO instead of the 47.87 one starting yesterday.
Found 2 IPs, 47.87.x.x and 89.33.x.x
Changed Main IP from 47 to 89 from Solus, and clicked reconfigure network. Boom, lost all connectivity.
Am I doing something wrong?
Had to revert back to 47 for it to work again.
Edit: Looks like it's not announced yet? 🤔
https://bgp.tools/prefix/89.33.192.0/24
Edit2: Node is RYZE.CHI-Z027.VMS
The Ultimate Speedtest Script | Get Instant Alerts on new LES/LET deals | Cheap VPS Deals
FREE KVM VPS - FreeVPS.org | FREE LXC VPS - MicroLXC
Yet not seeing any new IPs as of now. Am I out of the boat?
@VirMach Don't code when you are asleep
IIRC you are on LAXA031, I also am on the node. Have a new IPv4, switched over early, does not route. So you are not behind at all.
I don't think they are up to LAXA031 yet. Status page shows LAXA004, LAXA005, LAXA006, LAXA007 are confirmed ready.
For staff assistance or support issues please use the helpdesk ticket system at https://support.lowendspirit.com/index.php?a=add
Just reporting what I found:
Everything else working fine, thanks VirMach for keeping them running fine
the new IPs are from IPXO?
I bench YABS 24/7/365 unless it's a leap year.
Still here, not exactly going well in terms of how long it's taking for various reasons, mainly more weird SolusVM changes they did in the background on whatever version. Same update that caused problems with migrations seem to cause something similar for IP changes, but only when it gets switched to the main IP address too fast.
Getting close to done for LAX1, as in it's been "done" in terms of the actual routing for a while, just working on the part where it switches main IP and removes the old one.
In any case I'm going to shift focus to other locations now so people can at least do the change, and there are definitely still some routing issues we have to wait on the DC for in some locations for some blocks because RPKI wasn't 100% ready for some when we requested them and I don't think they were revisited.
Taking servers down to LAX unfortunately getting pushed back again, but I'm simultaneously working on LAX2Z019 right now and trying to take it down with the batch, as well as NYCB027 so we can close out those other issues together, so hopefully it's worth the extra wait. Since it's the weekend now everything else logistics-wise should be easier, we don't have a ton of people using the elevators here, my car won't be blocked in, and less traffic the way I'm going.
Oh unfortunate side note, I did snap my glasses in half and I can't find the superglue, lots of squinting but I do need to figure that one out.
^ Bound to be some Gaffer tape someplace! "There I fixed it!"
(ref. https://failblog.cheezburger.com/thereifixedit )
It wisnae me! A big boy done it and ran away.
NVMe2G for life! until death (the end is nigh)
now
09:54
Saturday, September 23, 2023
Eastern Time (ET)
I already migrated one of my servers to new IP address because my uptime notification notified me so. I have one more server that it isn't migrated yet cuz' I need to point the DNS of the server to the new IP first. Can I temporarily "run" two IP addresses to the single machine for now until the DNS changes fully propagates?
You probably saw me somewhere...
Both the old and the new will be available to use for the rest of the week, but this is completely based on whether or not SolusVM made any other changes to prevent that from happening, which is possible.
You'd have to go in manually though and add in your old IP address., since once it's technically done it would otherwise be removed from SolusVM. If you do have issues and you need the old one added back in an emergency situations, we could force it back in but I don't know how busy we'll be with everything else to be able to process the request.
LAX2 should be done now, keep in mind there's still a looooooooooong queue for the auto change to main IP but you can do this yourself.
Note -- this hasn't yet been fully tested though, there might be routing issues.