BTW, has anyone had any more success with using a NetBSD KASLR kernel? Using the instructions from https://wiki.netbsd.org/security/kaslr/ only results in a page fault:
Thanks for cloudbsd.xyz and also for suggesting unitedbsd.com on cloudbsd.com.
I read cloudbsd.xyz. I thought it would be a lot of fun to try it!
I plan to post here my quick recipe listing the steps I used to install from @cmeerw's NetBSD 10 image and update to what seems like self-compiled NetBSD-current. The post is done -- I just want to run through the steps once more to double check.
Then I hope to try your cloudbsd.xyz install method. Haha, I bet fifty cents I can generate a question or two or three!
Bonus points for non-Americanized (sic) spelling.
An interesting quick read (excluding the code), that (IMHO) is well written/formatted. A few corrections..
pivot_chroot on the other end will actually swap mounting points.
-->
pivot_root on the other hand will actually swap mounting points.
..understanding quite straightforward.
to be swapped, once everything is ready.
(might read a bit better)
..go further: use an emulator
..tools to be developped
..the processus processes
(said phonetically? )
Well done though!
(I wisnae being a totally pedant )
It wisnae me! A big boy done it and ran away.
NVMe2G for life! until death (the end is nigh)
An interesting quick read (excluding the code), that (IMHO) is well written/formatted.
Well done though!
First, thanks for the compliment on the article
A few corrections..
No
I don't want be brutal, but did not come here to seek for grammar feedbacks.
I already got feedbacks on that early on and only came because people looked interested and I like to share.
English is not my first language and was never good at writing/formulations even in my mother-tongue.
In my experience everyone have (and often colliding) feedbacks on potential corrections.
There is a time where it is relentless not to "cut the losses" and where it has to stop.
Otherwise in 4 years from now, someone's gonna come back again telling me to change something.
I was open to grammar feedbacks while it was fresh, not any-more.
(I wisnae being a totally pedant )
I know this comes out of a good intent.
However 4 months after publication, this becomes irrelevant and annoying.
I'm happy to share interesting things and that is what mattered/matters.
I'll take the feedback anyway, but no guaranty in terms of publishing the changes on the short term.
Bonus points for non-Americanized (sic) spelling.
I try my best, currently living in Scotland.
wisnae
Always happy talking with a Scottish fellow.
Nb: The pivot_root part was hard to write to keep some clarity without writing an essay just on that.
Nb2: Yeah maybe I'm a bit susceptible regarding my writing abilities.
I don't want be brutal, but did not come here to seek for grammar feedbacks.
LES uses an advanced biomechanical AI for spelling and grammar corrections.
The feature is now fully automated so users needn't make a request or tick any options for it to be activated.
It is also hard-coded, so can't be disabled.
I think we see fewer providers offering NetBSD images because NetBSD isn't requested as often as other BSDs.
I used @cmeerw's NetBSD 10 image because it was readily available thanks to the courtesy of Linveo. Some providers won't install custom OS images, so their customers are stuck with what's available. That's why @naguam developed CloudBSD.xyz, so people could install NetBSD without needing any customization on the provider side.
Even if I had installed using a NetBDSD-current image, I still would have to go through pretty much the same steps to keep the image up-to-date. All I have to do now to stay up-to-date is run cvs update, rebuild, reinstall, and reboot. I'm sure there are times when the build breaks or the reboot fails, but in my limited experience of a few years, I don't recall a build or a reboot failure.
If I misunderstood your question, please ask again.
@Crab said: May I ask what was your experience with them? What made you go back to Slack?
With @Crab, may I please join in asking what was your experience with the BSDs and what made you go back to Slack?
FWIW, I like Slackware too! Darkstar, my old server in Texas, ran Slackware-current. Everything worked great! Maybe LES needs a Slackware thread? @AuroraZero also likes Slack, or so I hear. I vote for making a Slackware thread!
In case anybody has been trying to set up FreeBSD as a VPN server and has been struggling with somewhat lackluster network performance with virtio drivers (e1000 actually work very well in KVM), keep on reading.
Despite @linveo having excellent network performance on their nodes, the VPN download has been very slow (~1Mbps), and I wanted to get to the bottom of it.
First, I thought that there's something to do with my firewall/NAT settings and I tried regular ipfw, ipfw in-kernel and pf, but they all shared the same symptoms: download is crawling, but upload is good.
Second, I was wondering whether the network driver has something to do with it and it absolutely does. I tried virtio with other providers and the same download performance issue kept occurring. Some control panels allow you to use e1000, which is not necessarily provide you the greatest performance, but is a reliable driver and with that VPN connections sped up significantly.
@linveo has excellent network performance on his nodes, but VirtFusion they use doesn't allow to change the network driver, so I kept looking for some clues and I stumbled upon this thread:
After a reboot, I was very happy and relieved to see that the download speeds increased drastically, over 100x! YMMW, but give it a try if you have had similar issues.
Also please note that I did try OpenVPN and Wireguard just to see whether there could be something to do with the actual VPN implementation, but both were actually behaving the same way.
TL;DR: If you have download performance problems with FreeBSD and either OpenVPN or Wireguard VPN server, try the /boot/loader.conf settings provided above.
It was the VPN problems i had.. 1st i had firewalls problems could not get any internet, and after makin that work; slow speeds. I know that network is good and should receive better speeds but that was slow. So now is rock solid
I ended up reinstalling and rebuilding my nice Linveo VPS -- not as expected in Debian so as to try @naguam's install -- but, instead of Debian, again in NetBSD-current upgraded from @cmeerw's NetBSD 10 Minimum.
This time I additionally installed pkgsrc and compiled a few packages also from src. Everything seems to work really fast and really smoothly.
Maybe I should continue doing more NetBSD-current stuff on the Linveo VPS and try the CloudBSD.xyz Debian to NetBSD install elsewhere.
Back in the old days I had stuff I wanted to accomplish with NetBSD. Now I have free time to look around and to try stuff. It's peaceful!
Might anyone here want a shell account on my NetBSD-current VPS?
@Not_Oles said: What are all you other guys doing on BSD nowadays?
Not much really, just noticed a small glitch in the network setup in my NetBSD 10 image. But it's probably really just cosmetic - it adds "alias" flags to the addresses when it shouldn't. Just something to keep in mind for the 10.1 update, once that gets released.
Other than that (not BSD related), need to start replacing my InceptionHosting VPSes, and then watch the C++ safety battles next week.
Today I had fun doing three things on my Linveo NetBSD-current VPS.
First, I tested the backup and restore function of the Virtfusion hypervisor. It seemed to work okay.
My test procedure was:
Initially I shutdown the VM within NetBSD (shutdown -h now). Then I powered off the VM in the Linveo Control Panel.
Next I made the backup, then restarted the VM, made a test file (touch tomwashere), and shut the VM down a second time, again both inside NetBSD and in the Control Panel.
Then I restored from the backup, which seemed to restart the VM automagically.
The restored VM seems to work fine, even without tomwashere (at least, it appears that the backup and restore function did not simply restart the existing image which had contained tomwashere).
Making the backup took about 7 minutes. Restoring from the backup took about 3-1/2 minutes.
I didn't see a way to export or download the backup image to another location. Did I miss something?
Here's the command I used to add a little left margin when reading the NetBSD code style guide on my Linveo VPS via ssh from my Chromebook:
linveo# pwd
/usr/src/share/misc
linveo# sed 's/^/ /g' style | less
Third, I wondered about NetBSD's ability to run Linux binaries. I looked at Chapter 31 of the NetBSD Guide, which covers NetBSD's Linux emulation. I wondered whether NetBSD might run Yabs. I'm still using the default sh shell, but pkgsrc has bash as well as fio and iperf3. Maybe even Geekbench 6 might run?
@Not_Oles said:
What are all you other guys doing on BSD nowadays?
I used to host the website on a NetBSD VPS running Caddy in a chroot as a user (and using npf to redirect the standard ports to ports above 1024).
I also used it for OpenVPN. (I had problems with IKEv2 and wireguard (both their implementation and the userspace implementation so I decided OpenVPN was good enough).
Now I always got interests in various OS and NetBSD looked more approachable to me to contribute at my level.
Overall you can host most common webservices on NetBSD despites some of its limitations.
Here is a non-exhaustive list of limitations I have with NetBSD for server use.
Lack of integration with many automation tools.
virtio, necessary for performance in most cloud providers is not always perfectly supported. (there are drivers but I've got many problems, including regarding storage and network where it is the most crucial.
Lack of hardware support in case of bare metal.
NPF is not too bad, but lacks features (it is L3 only, but sometimes L2 can be useful)
Same with other BSD but to be fair there is a need for a container-like solution and resource isolation among processes. (could be proper jails, only FreeBSD has some).
The Linux CAPs are very useful to let an app use reserved port ranges ( < 1024), on NetBSD you rely on manual port-forwarding firewall config.
Because NPF is custom, it is clearly absent of all automated configuration tools or api in programming langages.
NPF (not exhaustive) lacks proper documentation. I read examples and managed to build something, but to get what's allowed or what's not, is very hard.
NetBSD tun only supports ipv4, it's probably gonna change soon (maybe 10.1), but it is a problem.
A lot of Integrated services lack updates in terms of modern needs (Ipsec, TLS versions) and a lot of alternatives in repos lacks a default service script available.
Other things I forget.
And overall I'm sold with the Linux's Plan9 inspired Namespaces, necessary for containers, as well as cgroups.
One can do very interesting stuff with these.
I really thing BSD's would benefit from such a thing.
Here is a non-exhaustive list of limitations I have with NetBSD for server use.
[ . . . ]
virtio, necessary for performance in most cloud providers is not always perfectly supported. (there are drivers but I've got many problems, including regarding storage and network where it is the most crucial.
@Not_Oles said: I tested the backup and restore function of the Virtfusion hypervisor. It seemed to work okay.
This means that I can install Debian on my Linveo VPS, try @naguam's NetBSD install at CloudBSD.xyz all the while feeling somewhat confident that I probably can quickly and easily restore to the point where I left NetBSD.
Does anybody have a Kimsufi system with FreeBSD 14.1 and working IPv6 setup? I had a weak moment and got a box from their Black Friday sale, but IPv6 simply refuses to work no matter what I do. I have verified it working fine on Linux with their rescue system, so it has to be the OS setup, but as you know there's not much to it.
It really shouldn't be much more than this in /etc/rc.conf which works great elsewhere:
I can ping the gateway address just fine, but nothing beyond that. SLAAC doesn't seem to be supported, so it has to be set statically. I have tried different prefix lengths as well.
Trying to add the default router manually gives this error:
add net default: gateway 2607:5300:xx:xx:xx fib 0: Invalid argument
I have googled this up quite a bit without real success and tried various different ways described at FreeBSD forums and Reddit, but nothing conclusive. I even reached out to their support and all they told me is to follow their guide which doesn't say much.
Comments
I've been through this pain. Ended up with IPFW more down to the (potential) integration with Webmin, than any other reason.
It wisnae me! A big boy done it and ran away.
NVMe2G for life! until death (the end is nigh)
BTW, has anyone had any more success with using a NetBSD KASLR kernel? Using the instructions from https://wiki.netbsd.org/security/kaslr/ only results in a page fault:
@cmeerw Maybe I could build KASLR and try it if my building KASLR seems like it somehow might be helpful?
No, the string "KASLR" probably isn't random! /s
K = kernel
A = Address
S = Space
L = Layout
R = Randomization
Address space layout randomization
https://en.wikipedia.org/wiki/Address_space_layout_randomization
Kernel address space layout randomization
https://lwn.net/Articles/569635/
I hope everyone gets the servers they want!
Opinions will vary but I’ve used PF for many years with success.
Hi,
I am the author of https://cloudbsd.xyz.
I saw people got interested in here so I created an account.
If you got questions do not hesitate.
For you information as I saw the question being asked, systemctl isolate is equivalent to telinit with sysv.
Systemd targets are equivalent to sysv runlevels.
In my case it is because it is an easy way to shutdown everything that's not a rescue ssh shell.
Cheers,
Hello @naguam!
Wow! Nice to meet you!
Thanks for cloudbsd.xyz and also for suggesting unitedbsd.com on cloudbsd.com.
I read cloudbsd.xyz. I thought it would be a lot of fun to try it!
I plan to post here my quick recipe listing the steps I used to install from @cmeerw's NetBSD 10 image and update to what seems like self-compiled NetBSD-current. The post is done -- I just want to run through the steps once more to double check.
Then I hope to try your cloudbsd.xyz install method. Haha, I bet fifty cents I can generate a question or two or three!
Thanks again for joining us here at LES!
Best!
Tom
I hope everyone gets the servers they want!
Bonus points for non-Americanized (sic) spelling.
An interesting quick read (excluding the code), that (IMHO) is well written/formatted. A few corrections..
-->
(might read a bit better)
(said phonetically?
)
Well done though!
)
(I wisnae being a totally pedant
It wisnae me! A big boy done it and ran away.
NVMe2G for life! until death (the end is nigh)
First, thanks for the compliment on the article
No
I don't want be brutal, but did not come here to seek for grammar feedbacks.
I already got feedbacks on that early on and only came because people looked interested and I like to share.
English is not my first language and was never good at writing/formulations even in my mother-tongue.
In my experience everyone have (and often colliding) feedbacks on potential corrections.
There is a time where it is relentless not to "cut the losses" and where it has to stop.
Otherwise in 4 years from now, someone's gonna come back again telling me to change something.
I was open to grammar feedbacks while it was fresh, not any-more.
I know this comes out of a good intent.
However 4 months after publication, this becomes irrelevant and annoying.
I'm happy to share interesting things and that is what mattered/matters.
I'll take the feedback anyway, but no guaranty in terms of publishing the changes on the short term.
I try my best, currently living in Scotland.
Always happy talking with a Scottish fellow.
Nb: The pivot_root part was hard to write to keep some clarity without writing an essay just on that.
Nb2: Yeah maybe I'm a bit susceptible regarding my writing abilities.
I'm always in awe of guys like you and @bikegremlin when it comes to your comms. skills.
The quoting of ..
.. was primarily to save confusion, due to the typo.
It wisnae me! A big boy done it and ran away.
NVMe2G for life! until death (the end is nigh)
LES uses an advanced biomechanical AI for spelling and grammar corrections.
The feature is now fully automated so users needn't make a request or tick any options for it to be activated.
It is also hard-coded, so can't be disabled.
Relja of House Novović, the First of His Name, King of the Plains, the Breaker of Chains, WirMach Wolves pack member
BikeGremlin's web-hosting reviews
Why people do not create some templates ready made for it?
I believe in good luck. Harder that I work ,luckier i get.
Hi @Chievo!
Sorry for the late reply. I think you are asking why people do not create templates ready made for NetBSD-current.
At https://nycdn.netbsd.org/pub/NetBSD-daily/HEAD/202410301200Z/images/ you can see that there are NetBSD-current images prepared every few days.
I think we see fewer providers offering NetBSD images because NetBSD isn't requested as often as other BSDs.
I used @cmeerw's NetBSD 10 image because it was readily available thanks to the courtesy of Linveo. Some providers won't install custom OS images, so their customers are stuck with what's available. That's why @naguam developed CloudBSD.xyz, so people could install NetBSD without needing any customization on the provider side.
Even if I had installed using a NetBDSD-current image, I still would have to go through pretty much the same steps to keep the image up-to-date. All I have to do now to stay up-to-date is run
cvs update, rebuild, reinstall, and reboot. I'm sure there are times when the build breaks or the reboot fails, but in my limited experience of a few years, I don't recall a build or a reboot failure.If I misunderstood your question, please ask again.
Best wishes!
Tom
I hope everyone gets the servers they want!
Sadly after 2 months tinkering with BSD's i just could not resist and went back to this beauty on Linveo
ServerStatus , slackvpn <-- openVPN auto install script for Slackware 15
May I ask what was your experience with them? What made you go back to Slack?
@emperor
With @Crab, may I please join in asking what was your experience with the BSDs and what made you go back to Slack?
FWIW, I like Slackware too! Darkstar, my old server in Texas, ran Slackware-current. Everything worked great! Maybe LES needs a Slackware thread? @AuroraZero also likes Slack, or so I hear. I vote for making a Slackware thread!
I hope everyone gets the servers they want!
In case anybody has been trying to set up FreeBSD as a VPN server and has been struggling with somewhat lackluster network performance with virtio drivers (e1000 actually work very well in KVM), keep on reading.
Despite @linveo having excellent network performance on their nodes, the VPN download has been very slow (~1Mbps), and I wanted to get to the bottom of it.
First, I thought that there's something to do with my firewall/NAT settings and I tried regular ipfw, ipfw in-kernel and pf, but they all shared the same symptoms: download is crawling, but upload is good.
Second, I was wondering whether the network driver has something to do with it and it absolutely does. I tried virtio with other providers and the same download performance issue kept occurring. Some control panels allow you to use e1000, which is not necessarily provide you the greatest performance, but is a reliable driver and with that VPN connections sped up significantly.
@linveo has excellent network performance on his nodes, but VirtFusion they use doesn't allow to change the network driver, so I kept looking for some clues and I stumbled upon this thread:
https://www.reddit.com/r/freebsd/comments/jqa4vw/wireguard_vpn_very_slow_on_freebsd/
They advised to put these lines into /boot/loader.conf:
hw.vtnet.0.tso_disable="1"hw.vtnet.tso_disable="1"hw.vtnet.lro_disable="1"hw.vtnet.0.lro_disable="1"hw.vtnet.csum_disable="1"hw.vtnet.0.csum_disable="1"After a reboot, I was very happy and relieved to see that the download speeds increased drastically, over 100x! YMMW, but give it a try if you have had similar issues.
Also please note that I did try OpenVPN and Wireguard just to see whether there could be something to do with the actual VPN implementation, but both were actually behaving the same way.
TL;DR: If you have download performance problems with FreeBSD and either OpenVPN or Wireguard VPN server, try the /boot/loader.conf settings provided above.
You sexy biatch!!!!
Free Hosting at YetiNode | Cryptid Security | URL Shortener | LaunchVPS | ExtraVM | Host-C | In the Node, or Out of the Loop?
It was the VPN problems i had.. 1st i had firewalls problems could not get any internet, and after makin that work; slow speeds. I know that network is good and should receive better speeds but that was slow. So now is rock solid
ServerStatus , slackvpn <-- openVPN auto install script for Slackware 15
I ended up reinstalling and rebuilding my nice Linveo VPS
-- not as expected in Debian so as to try @naguam's install -- but, instead of Debian, again in NetBSD-current upgraded from @cmeerw's NetBSD 10 Minimum.
This time I additionally installed pkgsrc and compiled a few packages also from src. Everything seems to work really fast and really smoothly.
Maybe I should continue doing more NetBSD-current stuff on the Linveo VPS and try the CloudBSD.xyz Debian to NetBSD install elsewhere.
Back in the old days I had stuff I wanted to accomplish with NetBSD. Now I have free time to look around and to try stuff. It's peaceful!
Might anyone here want a shell account on my NetBSD-current VPS?
What are all you other guys doing on BSD nowadays? @Crab @FrankCastle @cmeerw @AlwaysSkint @naguam
I hope everyone gets the servers they want!
Other pressures preventing me from playing.
It wisnae me! A big boy done it and ran away.
NVMe2G for life! until death (the end is nigh)
Not much really, just noticed a small glitch in the network setup in my NetBSD 10 image. But it's probably really just cosmetic - it adds "alias" flags to the addresses when it shouldn't. Just something to keep in mind for the 10.1 update, once that gets released.
Other than that (not BSD related), need to start replacing my InceptionHosting VPSes, and then watch the C++ safety battles next week.
Today I had fun doing three things on my Linveo NetBSD-current VPS.
First, I tested the backup and restore function of the Virtfusion hypervisor. It seemed to work okay.
My test procedure was:
Initially I shutdown the VM within NetBSD (
shutdown -h now). Then I powered off the VM in the Linveo Control Panel.Next I made the backup, then restarted the VM, made a test file (
touch tomwashere), and shut the VM down a second time, again both inside NetBSD and in the Control Panel.Then I restored from the backup, which seemed to restart the VM automagically.
The restored VM seems to work fine, even without
tomwashere(at least, it appears that the backup and restore function did not simply restart the existing image which had containedtomwashere).Making the backup took about 7 minutes. Restoring from the backup took about 3-1/2 minutes.
I didn't see a way to export or download the backup image to another location. Did I miss something?
Second, I skimmed the NetBSD Developer Documentation: Notes on development and code style and the referenced NetBSD source code style guide. I noticed the line which says "from: @(#)style 1.12 (Berkeley) 3/18/94." Just for fun, if I look, maybe I will be able to find the 1994 antecedent Berkeley source. Maybe it's in the Unix Tree?
Here's the command I used to add a little left margin when reading the NetBSD code style guide on my Linveo VPS via ssh from my Chromebook:
Third, I wondered about NetBSD's ability to run Linux binaries. I looked at Chapter 31 of the NetBSD Guide, which covers NetBSD's Linux emulation. I wondered whether NetBSD might run Yabs. I'm still using the default
shshell, but pkgsrc hasbashas well asfioandiperf3. Maybe even Geekbench 6 might run?Yeah, another fun day here!
I hope everyone gets the servers they want!
Howdy! Welcome to the site, and thank you for the software!
Right now just chillin'. FreeBSD 14.2 is out soon, so plenty of fun coming up.
Hopefully the NetBSD fix that @cmeerw introduced will be mainlined soon, so I could kick more of its tires.
That and saving some energy for upcoming Black Friday. Hopefully we'll see some killer deals, so that "everyone gets the servers they want!"
I used to host the website on a NetBSD VPS running Caddy in a chroot as a user (and using npf to redirect the standard ports to ports above 1024).
I also used it for OpenVPN. (I had problems with IKEv2 and wireguard (both their implementation and the userspace implementation so I decided OpenVPN was good enough).
Now I always got interests in various OS and NetBSD looked more approachable to me to contribute at my level.
Overall you can host most common webservices on NetBSD despites some of its limitations.
On my day to day, I run Linux though.
What are its main limitations in your opinion?
Here is a non-exhaustive list of limitations I have with NetBSD for server use.
And overall I'm sold with the Linux's Plan9 inspired Namespaces, necessary for containers, as well as cgroups.
One can do very interesting stuff with these.
I really thing BSD's would benefit from such a thing.
Hi @naguam!
What would you do to test NetBSD compatibility on a new Linux based cloud provider?
Can you share NetBSD compatibility information respecting Hetzner Cloud and Linveo? Linveo is Virtfusion based, if that helps.
Thanks!
Tom
I hope everyone gets the servers they want!
This means that I can install Debian on my Linveo VPS, try @naguam's NetBSD install at CloudBSD.xyz all the while feeling somewhat confident that I probably can quickly and easily restore to the point where I left NetBSD.
I hope everyone gets the servers they want!
Does anybody have a Kimsufi system with FreeBSD 14.1 and working IPv6 setup? I had a weak moment and got a box from their Black Friday sale, but IPv6 simply refuses to work no matter what I do. I have verified it working fine on Linux with their rescue system, so it has to be the OS setup, but as you know there's not much to it.
It really shouldn't be much more than this in /etc/rc.conf which works great elsewhere:
ifconfig_igb0_ipv6="inet6 2607:5300:xx:xx:xx prefixlen 128"
ipv6_defaultrouter="2607:5300:xx:xx:xx"
I can ping the gateway address just fine, but nothing beyond that. SLAAC doesn't seem to be supported, so it has to be set statically. I have tried different prefix lengths as well.
Trying to add the default router manually gives this error:
add net default: gateway 2607:5300:xx:xx:xx fib 0: Invalid argument
I have googled this up quite a bit without real success and tried various different ways described at FreeBSD forums and Reddit, but nothing conclusive. I even reached out to their support and all they told me is to follow their guide which doesn't say much.
Any tips or advice are greatly appreciated!