Hosthatch Stockholm IPv6 down

bjobjo OG
edited August 2023 in Outages

Since one hour, Smokeping was complaing my Storage VPS is down. Rebooted it, IPv4 is up again, but IPv6 still down. Interestingly the controlpanel shows 2 /64 prefixes for the server now, bot none of them work. Even the default gw fe80::1 is not pingable.

Tagged:
«13

Comments

  • Same here.

  • edited August 2023

    Same here.

    @bjo said: Interestingly the controlpanel shows 2 /64 prefixes for the server now

    That too.
    Old GleSYS AB /64 and new Prinode AB /64 subnet.

  • confirmed. 25min downtime according to hetrixtools. ping google from putty no response, but nextcloud is normal

  • bjobjo OG
    edited August 2023

    Old /64 seems up again OBE.

  • Not here. Still down.

    What does it mean "OBE"?

  • TheDPTheDP OG
    edited August 2023

    @Mumbly said: What does it mean "OBE"?

    The upstream maybe - Obenetwork AB.

    EDIT: Or maybe not, since that would be Portlane/GleSYS :smiley:

  • bjobjo OG
    edited August 2023

    The upstream maybe - Obenetwork AB.

    EDIT: Or maybe not, since that would be Portlane/GleSYS :smiley:

    It's both?

    From Netcup:

      3.|-- ae13-1386.RT.EQX.FKT.DE.r  0.0%    10    3.5   3.6   3.4   4.3   0.2
      4.|-- RT.TC1.STO.SE.retn.net     0.0%    10   22.8  23.6  22.7  29.3   2.0
      5.|-- GW-ObeNetwork.retn.net     0.0%    10   33.2  33.1  33.0  33.5   0.2
      6.|-- ae1-19.jhv-bgc.obe.net     0.0%    10   34.3  34.4  34.2  34.7   0.2
      7.|-- eth1-49-11.hgr-vbdc.obe.n  0.0%    10   35.1  35.2  35.1  35.5   0.1
      8.|-- eth1-52-demarc.hgr-vbdc.o  0.0%    10   37.2  59.2  35.0 127.4  35.6
      9.|-- 2a00:1a28:1157:xyz     0.0%    10   34.6  34.8  34.6  35.2   0.2
    
    

    And from my FTTH:

      8.|-- ddf-b3-link.ip.twelve99.n  0.0%    10    7.0   7.3   6.7   8.6   0.5
      9.|-- ddf-b4-v6.ip.twelve99.net  0.0%    10   12.8   8.6   7.8  12.8   1.5
     10.|-- adm-b10-v6.ip.twelve99.ne 10.0%    10   10.7  10.8  10.5  11.4   0.3
     11.|-- portlane-ic316381-adm-b2. 30.0%    10   10.7  11.1  10.7  12.1   0.5
     12.|-- be-4.cr1.mal4.se.portlane 30.0%    10   20.0  20.2  20.0  20.8   0.3
     13.|-- be-5.cr2.sto1.se.portlane 50.0%    10   26.6  26.7  26.6  26.9   0.1
     14.|-- be-1.pe3.sto1.se.portlane 40.0%    10   26.6  26.9  26.6  27.2   0.2
     15.|-- 2a00:1a28:1:3091::2        0.0%    10  434.7 161.3  32.8 566.7 193.0
     16.|-- 2a00:1a28:1157:xyz     0.0%    10   32.4  32.7  32.4  33.0   0.2
    

    And the "new" address from Netcup:

      5.|-- 2a00:11c0:47:1:47::212     0.0%    10   22.3  22.4  22.3  22.4   0.1
      6.|-- xe-0-1-1-amsix.ams-evo.ob  0.0%    10   23.2  23.6  23.1  25.8   0.8
      7.|-- ae2-10.mma-vg4.obe.net     0.0%    10   35.5  35.7  35.5  35.9   0.1
      8.|-- ae4-5.jhv-bgc.obe.net      0.0%    10   45.6  45.8  45.6  45.9   0.1
      9.|-- eth1-49-11.hgr-vbdc.obe.n  0.0%    10   46.6  46.6  46.4  47.0   0.2
     10.|-- eth1-52-demarc.hgr-vbdc.o  0.0%    10  453.9 183.0  50.3 453.9 123.5
     11.|-- 2a0e:dc0:2:b81:xyz         0.0%    10   46.1  46.1  45.7  46.5   0.2
    

    And from FTTH (really strange as my ISP (AS50629) also peers at Amsix:

      8.|-- ddf-b3-link.ip.twelve99.n  0.0%    10    7.2   7.4   7.0   8.0   0.3
      9.|-- ffm-bb1-v6.ip.twelve99.ne 20.0%    10   11.1  11.8  10.8  17.2   2.2
     10.|-- ???                       100.0    10    0.0   0.0   0.0   0.0   0.0
     11.|-- level3-ic319606-ffm-b11.i 80.0%    10   10.6  11.0  10.6  11.3   0.4
     12.|-- 2001:1900:2::3:102         0.0%    10   58.6  38.2  32.5  58.6   8.7
     13.|-- as3356-1.arn.as54104.net   0.0%    10   33.0  32.9  32.4  33.5   0.3
     14.|-- ae2-19.kis-kg.obe.net      0.0%    10   38.6  38.8  38.3  39.1   0.2
     15.|-- ae1-17.bmm-sk1.obe.net     0.0%    10   38.6  38.9  38.5  39.8   0.4
     16.|-- ae1-19.jhv-bgc.obe.net     0.0%    10   39.2  39.4  38.8  42.7   1.2
     17.|-- eth1-49-11.hgr-vbdc.obe.n  0.0%    10   40.2  39.9  39.4  40.8   0.4
     18.|-- eth1-52-demarc.hgr-vbdc.o  0.0%    10   57.4  66.8  41.0 125.6  32.0
     19.|-- 2a0e:dc0:2:b81b::xyz         0.0%    10   39.7  39.6  39.0  41.3   0.7
    
    Thanked by (1)TheDP
  • Not sure if this is related, but sharing since it's Stockholm.

    Dear Customer,

    We want to inform you of a scheduled maintenance that affects your active service(s) listed below.

    Maintenance start: 2023-08-22 21:30 CEST
    Maintenance end: 2023-08-22 23:30 CEST
    Expected impact: 45-90 minutes of downtime and reboot

    Services affected:
    panopticon (176.126.x.x) - Stockholm

    During this maintenance, we will safely shut down your VM, perform the maintenance, and boot it back online. You should see improved network performance after this has been completed.

    Sincerely,
    HostHatch Support Team

    Thanked by (1)FrankZ
  • edited August 2023

    It is related. When network came back after the maintenance, IPv6 didn't work anymore.

    btw. nice vps name.

    Thanked by (1)TheDP
  • @Mumbly said:
    It is related. When network came back after the maintenance, IPv6 didn't work anymore.

    Well, it's still 21.08.23 CEST, not 22.08. The maintenance according to the announcement should be tomorrow.

  • edited August 2023

    ah, crap... you're correct

  • And the outage began around 15:30 CEST or so, maybe @hosthatch srambled the dates and timezones?

  • @bjo said:
    And the outage began around 15:30 CEST or so, maybe @hosthatch srambled the dates and timezones?

    Or this upcoming maintenance is to address the issue(s).

  • I did not have any issues before.

  • Ipv6 working for me now.

  • @bjo said: Old /64 seems up again OBE.

    @dosai said: Ipv6 working for me now.

    Anyone tryed new one too?
    My old is still down and I am wondering if I should switch to the new one.

  • For me both are working, but the gateway is from the old prefix.

    Thanked by (1)Mumbly
  • Now down again since 22.12h CEST, within the announced timeslot.

  • bjobjo OG
    edited August 2023

    "boot it back online" did not happen. Started it via control panel, IPv4 works, IPv6 are broken. IPv6-gateway is still 2a00:1a28:1157::1.

    Edit: Both up again.

  • edited August 2023

    In my case IPv6 never came back since first outage from 2 days ago.
    Before I opened ticket I've redone everything from the beginning. No luck.

    Can you post ip -6 route output please (blank out part of your subnet).

    ::1 dev lo proto kernel metric 256 pref medium
    2a00:1a28:1157:xxxx::/64 dev eth0 proto kernel metric 256 pref medium
    fe80::/64 dev eth0 proto kernel metric 256 pref medium
    default via fe80::1 dev eth0 proto static metric 1024 pref medium
    
  • @Mumbly said:

    @bjo said: Old /64 seems up again OBE.

    @dosai said: Ipv6 working for me now.

    Anyone tryed new one too?
    My old is still down and I am wondering if I should switch to the new one.

    I didn't configure network, it auto switched to the new ipv6.

    Thanked by (1)Mumbly
  • edited August 2023

    @dosai said:
    I didn't configure network, it auto switched to the new ipv6.

    Can you post "ip -6 route" output please (blank out part of your subnet).

  • @Mumbly said:

    ::1 dev lo proto kernel metric 256 pref medium
    2a00:1a28:1157:xxxx::/64 dev eth0 proto kernel metric 256 pref medium
    fe80::/64 dev eth0 proto kernel metric 256 pref medium
    default via fe80::1 dev eth0 proto static metric 1024 pref medium
    
    2a00:1a28:1157:yxz::/64 dev eth0  metric 256 
    fe80::/64 dev eth0  metric 256 
    default via fe80::1 dev eth0  metric 1024 
    multicast ff00::/8 dev eth0  metric 256 
    

    Got a reply to my ticket from the first outage which asked me to reconfigure according to https://docs.hosthatch.com/networking/#ipv6-configuration

    Does fe80::1 reply on your box?

    Thanked by (1)Mumbly
  • @dosai said:
    I didn't configure network, it auto switched to the new ipv6.

    How? The docs says

    We do not support automatic (SLAAC) or dynamic (DHCPv6) IPv6 assignment
    
  • edited August 2023

    @bjo said: Got a reply to my ticket from the first outage which asked me to reconfigure according to https://docs.hosthatch.com/networking/#ipv6-configuration

    Does fe80::1 reply on your box?

    Yes, that's my IPv6 setup on every HH vps (and it worked also on .se until this outage from 2days ago)
    Reply?

    "If you choose to enable IPv6 when you first create your server, it will automatically be configured via cloud-init. However, manual configuration is needed if you choose to enable it after your server has been created, or if you install your OS via ISO."

    I tryed it also with clean reinstall from the template. It show initial ::1 IP in interface set by template, but still no luck.

  • I've deleted the old one now and activated the new one from 2a0e:dc0:2::/48

      1.|-- 2a00:1a28:1157::1          0.0%    10   18.1  19.8   4.4  72.7  19.6
      2.|-- fdca:f5d5:c3aa:cb2d::1     0.0%    10    8.0  14.5   3.4  58.5  16.4
      3.|-- be-4-706.pe3.sto1.se.port 10.0%    10    1.6   1.6   1.5   1.7   0.1
      4.|-- be-6.cr2.sto2.se.portlane  0.0%    10    2.4   2.0   1.9   2.4   0.1
      5.|-- 2001:4860:1:1::85e         0.0%    10    2.1   2.1   2.0   2.3   0.1
      6.|-- 2a00:1450:8110::1          0.0%    10    1.6   1.5   1.4   1.6   0.1
      7.|-- 2001:4860:0:1::5924        0.0%    10    1.5   1.5   1.4   1.7   0.1
      8.|-- 2001:4860:0:1344::5        0.0%    10    1.3   1.8   1.3   4.4   1.0
      9.|-- 2001:4860:0:134b::1        0.0%    10    2.7   2.7   2.4   3.4   0.3
     10.|-- 2001:4860:0:1::5123        0.0%    10    1.8   1.9   1.8   2.1   0.1
     11.|-- arn11s10-in-x03.1e100.net  0.0%    10    1.5   1.6   1.5   1.7   0.1
    
  • I tryed both, no luck and I am out of ideas as everything seems in order at my end.
    I will reinstall it once again via ISO (that's template now) once I come home and this is it ...at the mercy of their support.

    root@box:~# ifconfig
    eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
            inet 31.220.xx.xxx  netmask 255.255.255.0  broadcast 31.220.15.255
            inet6 2a00:1a28:1157:xxxx::1  prefixlen 64  scopeid 0x0<global>
            inet6 2a00:1a28:1157:xxxx:3:3:3:3  prefixlen 64  scopeid 0x0<global>
            inet6 2a00:1a28:1157:xxxx:2:2:2:2  prefixlen 64  scopeid 0x0<global>
            inet6 2a00:1a28:1157:xxxx:1:1:1:1  prefixlen 64  scopeid 0x0<global>
            inet6 fe80::220:56ff:fe09:c87f  prefixlen 64  scopeid 0x20<link>
            ether 00:20:56:09:c8:7f  txqueuelen 1000  (Ethernet)
            RX packets 218  bytes 18916 (18.4 KiB)
            RX errors 0  dropped 0  overruns 0  frame 0
            TX packets 175  bytes 19774 (19.3 KiB)
            TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
    
    lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
            inet 127.0.0.1  netmask 255.0.0.0
            inet6 ::1  prefixlen 128  scopeid 0x10<host>
            loop  txqueuelen 1000  (Local Loopback)
            RX packets 15  bytes 1452 (1.4 KiB)
            RX errors 0  dropped 0  overruns 0  frame 0
            TX packets 15  bytes 1452 (1.4 KiB)
            TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
    
    root@box:~# curl ipv6.google.com
    curl: (7) Failed to connect to ipv6.google.com port 80: No route to host
    
    root@box:~# ip -6 route
    ::1 dev lo proto kernel metric 256 pref medium
    2a00:1a28:1157:xxxx::/64 dev eth0 proto kernel metric 256 pref medium
    fe80::/64 dev eth0 proto kernel metric 256 pref medium
    default via fe80::1 dev eth0 metric 1024 onlink pref medium
    
  • I don't see the new prefix in your output.

  • Because it makes no difference. I tryed all combinations.

    root@box:~# ifconfig
    eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
            inet 31.220.xxx.xxx  netmask 255.255.255.0  broadcast 31.220.15.255
            inet6 2a0e:dc0:2:xxxx::2  prefixlen 64  scopeid 0x0<global>
            inet6 2a00:1a28:1157:xxxx::1  prefixlen 64  scopeid 0x0<global>
            inet6 2a0e:dc0:2:xxxx::1  prefixlen 64  scopeid 0x0<global>
            inet6 fe80::220:56ff:fe09:c87f  prefixlen 64  scopeid 0x20<link>
            inet6 2a00:1a28:1157:xxxx:2:2:2:2  prefixlen 64  scopeid 0x0<global>
            inet6 2a00:1a28:1157:xxxx:1:1:1:1  prefixlen 64  scopeid 0x0<global>
            ether 00:20:56:09:c8:7f  txqueuelen 1000  (Ethernet)
            RX packets 286  bytes 24726 (24.1 KiB)
            RX errors 0  dropped 0  overruns 0  frame 0
            TX packets 201  bytes 26519 (25.8 KiB)
            TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
    
    lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
            inet 127.0.0.1  netmask 255.0.0.0
            inet6 ::1  prefixlen 128  scopeid 0x10<host>
            loop  txqueuelen 1000  (Local Loopback)
            RX packets 15  bytes 1436 (1.4 KiB)
            RX errors 0  dropped 0  overruns 0  frame 0
            TX packets 15  bytes 1436 (1.4 KiB)
            TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
    
    root@box:~# curl ipv6.google.com
    curl: (7) Failed to connect to ipv6.google.com port 80: No route to host
    
  • I just tried and both, old and new subnet works.

    Thanked by (1)bjo
Sign In or Register to comment.