@Ympker said:
Just regarding streaming from home:
I fiddled forever with Plex since Vodafone doesn't give you a public IP. I am now using our FritzNAS (Fritzbox + any external hard drive) and while it doesn't have the looks from Plex, watching movies/series from remote works like a charm. Just your remote device probably will need IPv6 (either from your mobile carrier or the network/wifi you are in when watching). Only really tried watching on mobile devices, but you can also access the NAS in the browser and watch
We only got 25 Mbits, so you should be good.
I don't know if you are from the UK or not, but I'm with Vodafone for VDSL and have the 80mbps pro plan for £23 a month, they will give you a free static IP if you request one (I have one).
Thanks for bringing that up! Unfortunately, I am from Germany When I last called them and asked, they said that they won't do this anymore.
not true. at least if you are on cable. don't ask for a static IPv4 but for 'switching from DS lite to full dual stack' ;-)
afaik it incur some small fees, unless you are in the middle of a bigger complaint, then they're quite happy to give to you for free... or so I heard
@Ympker said:
Just regarding streaming from home:
I fiddled forever with Plex since Vodafone doesn't give you a public IP. I am now using our FritzNAS (Fritzbox + any external hard drive) and while it doesn't have the looks from Plex, watching movies/series from remote works like a charm. Just your remote device probably will need IPv6 (either from your mobile carrier or the network/wifi you are in when watching). Only really tried watching on mobile devices, but you can also access the NAS in the browser and watch
We only got 25 Mbits, so you should be good.
I don't know if you are from the UK or not, but I'm with Vodafone for VDSL and have the 80mbps pro plan for £23 a month, they will give you a free static IP if you request one (I have one).
Thanks for bringing that up! Unfortunately, I am from Germany When I last called them and asked, they said that they won't do this anymore.
not true. at least if you are on cable. don't ask for a static IPv4 but for 'switching from DS lite to full dual stack' ;-)
afaik it incur some small fees, unless you are in the middle of a bigger complaint, then they're quite happy to give to you for free... or so I heard
But dual stack doesn't mean static IP rather dedicated IP so basically what you've got with an old DSL16.000 service.
@Ympker said:
Just regarding streaming from home:
I fiddled forever with Plex since Vodafone doesn't give you a public IP. I am now using our FritzNAS (Fritzbox + any external hard drive) and while it doesn't have the looks from Plex, watching movies/series from remote works like a charm. Just your remote device probably will need IPv6 (either from your mobile carrier or the network/wifi you are in when watching). Only really tried watching on mobile devices, but you can also access the NAS in the browser and watch
We only got 25 Mbits, so you should be good.
I don't know if you are from the UK or not, but I'm with Vodafone for VDSL and have the 80mbps pro plan for £23 a month, they will give you a free static IP if you request one (I have one).
Thanks for bringing that up! Unfortunately, I am from Germany When I last called them and asked, they said that they won't do this anymore.
not true. at least if you are on cable. don't ask for a static IPv4 but for 'switching from DS lite to full dual stack' ;-)
afaik it incur some small fees, unless you are in the middle of a bigger complaint, then they're quite happy to give to you for free... or so I heard
But dual stack doesn't mean static IP rather dedicated IP so basically what you've got with an old DSL16.000 service.
that's true. but I have yet to see the IP change, even on a reconnect it stays. and with cable there is forced reconnect after a day anymore, so essentially you stay online anyway and the IP doesn't change. static enough for me... ;-)
@Amitz said: @foxone - Great, thank you! I was wondering whether simply attaching 4 drives via USB to my existing Linux PC at home would be a viable solution, too! Is there really no issue with creating a RAID 1 array with external USB drives?
No. Just remember to never use /dev/sdX devices, and use /dev/disk/by-id/xxx so that even if they get detected at different times they don't switch names.
Also, with lsusb -t you can see the internal hub tree of your server. Try to connect the drives at 10G (or dedicated 5G) hubs, so they don't get bottlenecked. Basically, never connect more devices than the speed your port can handle.
Having just done this I'd tread carefully - take a careful look at the USB power situation. Multiple nvme drives can overload the USB's aggregate ability to feed it. Worse it only shows up under load since nvme draw depends on usage. Chances are very good that it "works" but is outside of spec
All drives are powered externally with a beefy industrial 12v /5v power supply.
Sooo... I went ahead and bought one WD Elements 14 TB unit to do some testing and to start downloading all stuff from the server. I do not even want to think about how long this will take (I am innerly prepared for days if not weeks). When I am finished and happy with the result and performance, then I will buy a second one to do the backup.
Does anyone have an idea how long 10 TB might take with a 500 Mbps connection that is nearly maxed out? Are there calculators for something like this out there? I must admit being to lazy to calculate it myself, even though I have the general tool (brain) and apps (math knowledge) at hand... ;-)
//Edit:
Scratch that: I found a calculator. It says around 44 hours. Now that would be absolutely bearable...
Stupid question, but I am out of the Windows world since WindowsXP - The drive contained a folder named "System Volume Information" and was pre-formatted in NTFS. NTFS is fine for me, but I assume that I can safely delete this folder, right? I am using the drive exclusively under Linux and maybe MacOS. That "System Volume Information" folder is a Windows thing for snapshots & stuff, isn't it?
Amitz, a very stable genius (it's true!) and Grand Rectumfier of the official LESLOS® (LES League of Shitposters).
Certified braindead since 1974 and still perfectly happy.
I have also shucked Western Digital drives. Cost today is about $17 per TB, and was as low as $14.50/TB during the November/December shopping time. Voids the warranty, but cheaper than internal drives. Used five shucked 8 TBs for two years, and recently sold them after upgrading to three 14 TBs.
@natvps_uk said: You can also reduce the noise and power consumption to an minimum by using the least amount of disks possible
☝ This. Glad I went from 5 to 3 online hard drives. Three drives are as much noise I could tolerate in my home office.
You can also use a hot-swappable bay for plugging in/out drives when needed. For things like old downloads that are replaceable by downloading again, no need to be RAIDed and powered on all the time. I actually don't use RAID at all, just cron jobs that sync important directories across all 3 drives, which saves the disk space needed to mirror everything and saves the hassle of long-term RAID maintenance. Also, about 2x/year I manually backup stuff to a hot-swap drive that I keep elsewhere in the house, in case fire or theft in the office.
Maybe it's what you found, but this is the one I use: http://www.dslreports.com/calculator
mind the bits (bps) vs bytes (Bps) per second in the speed box
@Amitz said: The drive contained a folder named "System Volume Information"
Format the drive, use in Linux, Mac, whatever. Don't leave as NTFS unless using on Windows.
@jon617 said:
Don't leave as NTFS unless using on Windows.
Well, I will let it as NTFS. I can read & write it fine under Linux and macOS and I could still take it with me to Windows friends (the few I have, strange guys) and they can read the disk without issues. Same with my Samsung TV. I am not sure whether it accepts something at EXT4 or APFS. It might, but I never tried.
Amitz, a very stable genius (it's true!) and Grand Rectumfier of the official LESLOS® (LES League of Shitposters).
Certified braindead since 1974 and still perfectly happy.
I have 2 * 14TB WD elements and 2 * 8TB in my Synology ds920+ using Synology's hybrid raid which gives me about 25TB of usable space.
Been running these drives for sometime including using them in a custom built home server that used mergerfs and snapraid, not experienced any issues with them.
Amitz, a very stable genius (it's true!) and Grand Rectumfier of the official LESLOS® (LES League of Shitposters).
Certified braindead since 1974 and still perfectly happy.
18TB WD Elements is $280 today (in the US). Search WDBWLG0180HBK-NESN on WesternDigital.com, price changes after added to the cart. Assuming this one's shuckability is the same as the smaller guys, $15.56 per TB is a good price.
The first 5TB are transferred and it seems as if Vodafone (my cable internet provider) or Hetzner is slowly starting to throttle my connection to half of its speed. It suddenly dropped from an average of 495 Mbps over days to constant 250 Mbps. Strange, strange. Well, I've got time...
Amitz, a very stable genius (it's true!) and Grand Rectumfier of the official LESLOS® (LES League of Shitposters).
Certified braindead since 1974 and still perfectly happy.
Amitz, a very stable genius (it's true!) and Grand Rectumfier of the official LESLOS® (LES League of Shitposters).
Certified braindead since 1974 and still perfectly happy.
Amitz, a very stable genius (it's true!) and Grand Rectumfier of the official LESLOS® (LES League of Shitposters).
Certified braindead since 1974 and still perfectly happy.
@Amitz said: @Falzo: Thank you, that worked indeed! Back to 495 Mbps! Let's see how long the fun lasts...
yeah I am not sure which part of the chain is the actual weakest. I can see the DOCSIS 3.1 channel for upload dropping randomly and if that starts to happen it usually goes downhill quickly, bad connection/packet loss and so on. somehow the renegotiation seems to fix it, or maybe it is some kind of overflowing buffer somewhere or whatever. glad it worked for you!
@Falzo: Thanks for the explanation! I was already wondering why this could have happened, as there is no "official" cap on transfer volume (not even a "fair usage" clause) on this business contract. I assumed that it might be some kind of automatism after a certain bandwidth has been used, but obviously it not the case.
I have monitored both upload and download rates during the process and: The download rate dropped (and stayed there) right after the upload capacity was saturated (50 Mbps) for around 15mins and did not return to normal afterwards. So your theory seems very plausible.
Amitz, a very stable genius (it's true!) and Grand Rectumfier of the official LESLOS® (LES League of Shitposters).
Certified braindead since 1974 and still perfectly happy.
I don't know if cable has the same problem as VDSL with Bufferbloat, but if I don't cap my upload/downloads on pfSense and let it upload/download at max speed the level of CRC error on my line triggers a reset on the ISP side and it lowers the line speed to reduce the error rate.
I had to setup traffic shaper (FQ_CODEL) to 90% of my actual line capacity to avoid resets.
In the US at least, shucking does not void warranty; folks have shipped the bare drive back to WD and gotten warranty replacements. My previous post included a link to a price-tracking site for shucks; I believe 12TB or 14TB is the sweet spot for $/TB nowadays.
FWIW, I have 4x12TB and 1x14TB using snapraid and drivepool.; windows reports 43.6TB usable. All are still in the enclosures and plugged in via USB3, using external power supplies to my Windows 11 PC which is plenty beefy enough to handle plex transcodes if needed. I don't share out plex, just me/family and 95% is direct play LAN traffic.
Comments
You won't catch bit rot with this though. Might be an acceptable risk in this case though
not true. at least if you are on cable. don't ask for a static IPv4 but for 'switching from DS lite to full dual stack' ;-)
afaik it incur some small fees, unless you are in the middle of a bigger complaint, then they're quite happy to give to you for free... or so I heard
But dual stack doesn't mean static IP rather dedicated IP so basically what you've got with an old DSL16.000 service.
that's true. but I have yet to see the IP change, even on a reconnect it stays. and with cable there is forced reconnect after a day anymore, so essentially you stay online anyway and the IP doesn't change. static enough for me... ;-)
No. Just remember to never use /dev/sdX devices, and use /dev/disk/by-id/xxx so that even if they get detected at different times they don't switch names.
Also, with
lsusb -tyou can see the internal hub tree of your server. Try to connect the drives at 10G (or dedicated 5G) hubs, so they don't get bottlenecked. Basically, never connect more devices than the speed your port can handle.In my case:
All drives are powered externally with a beefy industrial 12v /5v power supply.
That's ideal. I went with USB powered and yeah not the most inspired of plans..4th one added started to cause issues
Sooo... I went ahead and bought one WD Elements 14 TB unit to do some testing and to start downloading all stuff from the server. I do not even want to think about how long this will take (I am innerly prepared for days if not weeks). When I am finished and happy with the result and performance, then I will buy a second one to do the backup.
Does anyone have an idea how long 10 TB might take with a 500 Mbps connection that is nearly maxed out? Are there calculators for something like this out there? I must admit being to lazy to calculate it myself, even though I have the general tool (brain) and apps (math knowledge) at hand... ;-)
//Edit:
Scratch that: I found a calculator. It says around 44 hours. Now that would be absolutely bearable...
Stupid question, but I am out of the Windows world since WindowsXP - The drive contained a folder named "System Volume Information" and was pre-formatted in NTFS. NTFS is fine for me, but I assume that I can safely delete this folder, right? I am using the drive exclusively under Linux and maybe MacOS. That "System Volume Information" folder is a Windows thing for snapshots & stuff, isn't it?
Amitz, a very stable genius (it's true!) and Grand Rectumfier of the official LESLOS® (LES League of Shitposters).
Certified braindead since 1974 and still perfectly happy.
I have also shucked Western Digital drives. Cost today is about $17 per TB, and was as low as $14.50/TB during the November/December shopping time. Voids the warranty, but cheaper than internal drives. Used five shucked 8 TBs for two years, and recently sold them after upgrading to three 14 TBs.
☝ This. Glad I went from 5 to 3 online hard drives. Three drives are as much noise I could tolerate in my home office.
You can also use a hot-swappable bay for plugging in/out drives when needed. For things like old downloads that are replaceable by downloading again, no need to be RAIDed and powered on all the time. I actually don't use RAID at all, just cron jobs that sync important directories across all 3 drives, which saves the disk space needed to mirror everything and saves the hassle of long-term RAID maintenance. Also, about 2x/year I manually backup stuff to a hot-swap drive that I keep elsewhere in the house, in case fire or theft in the office.
level of laziness: premium
it's 10 * 1024 * 1024 * 8 / 500 / 60 / 60 = 46.6 hours - consider overhead and shit and add 20% or so but still, 2-3 days only ;-)
Maybe it's what you found, but this is the one I use: http://www.dslreports.com/calculator
mind the bits (bps) vs bytes (Bps) per second in the speed box
Format the drive, use in Linux, Mac, whatever. Don't leave as NTFS unless using on Windows.
This will depend on how important your data is. a Simple NAS would be enough especially if you don't need a noisy server at home
Web Hosting - VPS Cloud - EU Dedicated Offers Managed Hosting Made Easy Dedicated Servers, Private Cloud, Colocation, VPS, Shared & Reseller Hosting
Well, I will let it as NTFS. I can read & write it fine under Linux and macOS and I could still take it with me to Windows friends (the few I have, strange guys) and they can read the disk without issues. Same with my Samsung TV. I am not sure whether it accepts something at EXT4 or APFS. It might, but I never tried.
Amitz, a very stable genius (it's true!) and Grand Rectumfier of the official LESLOS® (LES League of Shitposters).
Certified braindead since 1974 and still perfectly happy.
/me suffers in GNU
I have 2 * 14TB WD elements and 2 * 8TB in my Synology ds920+ using Synology's hybrid raid which gives me about 25TB of usable space.
Been running these drives for sometime including using them in a custom built home server that used mergerfs and snapraid, not experienced any issues with them.
If I have learned something during my years using linux, then that there is always someone suffering in GNU.
Amitz, a very stable genius (it's true!) and Grand Rectumfier of the official LESLOS® (LES League of Shitposters).
Certified braindead since 1974 and still perfectly happy.
GNU is Not Unsuffering
blog | exploring visually |
The easy solution is to get a NAS with enough (is it ever enough?) disk space and use that.
Silent and accessible from almost everywhere.
“Technology is best when it brings people together.” – Matt Mullenweg
18TB WD Elements is $280 today (in the US). Search WDBWLG0180HBK-NESN on WesternDigital.com, price changes after added to the cart. Assuming this one's shuckability is the same as the smaller guys, $15.56 per TB is a good price.
Checked my prices today for a
Total Price, converted to USD = $3,256
But that would be enough for some time ....... 54TB with Synology Raid (which is software Raid5)
“Technology is best when it brings people together.” – Matt Mullenweg
Seagate is now shipping 22TB SMR HDDs.
Source.
The first 5TB are transferred and it seems as if Vodafone (my cable internet provider) or Hetzner is slowly starting to throttle my connection to half of its speed. It suddenly dropped from an average of 495 Mbps over days to constant 250 Mbps. Strange, strange. Well, I've got time...
Amitz, a very stable genius (it's true!) and Grand Rectumfier of the official LESLOS® (LES League of Shitposters).
Certified braindead since 1974 and still perfectly happy.
feel you bro... try restarting the router in between so the cable connection fully resyncs... seems to help some times for me.
Thank you so much for your empathy!
I will try that!
Amitz, a very stable genius (it's true!) and Grand Rectumfier of the official LESLOS® (LES League of Shitposters).
Certified braindead since 1974 and still perfectly happy.
Imagine resilvering those
@Falzo: Thank you, that worked indeed!
Back to 495 Mbps! Let's see how long the fun lasts...
Amitz, a very stable genius (it's true!) and Grand Rectumfier of the official LESLOS® (LES League of Shitposters).
Certified braindead since 1974 and still perfectly happy.
yeah I am not sure which part of the chain is the actual weakest. I can see the DOCSIS 3.1 channel for upload dropping randomly and if that starts to happen it usually goes downhill quickly, bad connection/packet loss and so on. somehow the renegotiation seems to fix it, or maybe it is some kind of overflowing buffer somewhere or whatever. glad it worked for you!
@Falzo: Thanks for the explanation! I was already wondering why this could have happened, as there is no "official" cap on transfer volume (not even a "fair usage" clause) on this business contract. I assumed that it might be some kind of automatism after a certain bandwidth has been used, but obviously it not the case.
I have monitored both upload and download rates during the process and: The download rate dropped (and stayed there) right after the upload capacity was saturated (50 Mbps) for around 15mins and did not return to normal afterwards. So your theory seems very plausible.
Amitz, a very stable genius (it's true!) and Grand Rectumfier of the official LESLOS® (LES League of Shitposters).
Certified braindead since 1974 and still perfectly happy.
I don't know if cable has the same problem as VDSL with Bufferbloat, but if I don't cap my upload/downloads on pfSense and let it upload/download at max speed the level of CRC error on my line triggers a reset on the ISP side and it lowers the line speed to reduce the error rate.
I had to setup traffic shaper (FQ_CODEL) to 90% of my actual line capacity to avoid resets.
In the US at least, shucking does not void warranty; folks have shipped the bare drive back to WD and gotten warranty replacements. My previous post included a link to a price-tracking site for shucks; I believe 12TB or 14TB is the sweet spot for $/TB nowadays.
FWIW, I have 4x12TB and 1x14TB using snapraid and drivepool.; windows reports 43.6TB usable. All are still in the enclosures and plugged in via USB3, using external power supplies to my Windows 11 PC which is plenty beefy enough to handle plex transcodes if needed. I don't share out plex, just me/family and 95% is direct play LAN traffic.