Pros and cons for OpenVZ 7 as compared to KVM

flips · December 2019

I've been curious about OpenVZ, as I never tried it before (only ran KVM in VPS'es, and on my local servers/workstations I've been running LXC/LXD, Xen, KVM, VirtualBox, VMware).

After testing a bit these are my initial thought. (I might be wrong.)

Pros:

Usually cheap(er than KVM)
Templates, disk usage lower (<400 MB for Debian 10)
Lower memory usage, more free memory (running the same processes)
Theoretically it could/should have lower overhead CPU wise? (I haven't experienced/been able to confirm this.)
For the provider: Easier to monitor/stop/prevent abuse

Cons:

Less options for OS selection (share Linux kernel, can't install BSD etc.)
Monitoring by provider easier
Server can more often be a bit heavier utilized/more crowded (?)
Can't load kernel specific stuff/modules (due to shared kernel)
Not everything works (as an example, stick with iptables, not nftables, for the time being)

There might also be other limitations I should have mentioned?

One thing I've noticed, is for disk I/O, I get something like this on my OpenVZ 7 instances:

 Disk Speed:
   1st run    : 232 MB/s
   2nd run    : 684 MB/s
   3rd run    : 774 MB/s

So, 1st run is much slower than the next. I didn't see that as clearly on a KVM node, but I'm not sure if it's really related to KVM vs. OVZ, maybe it has more to do with caching.

I've also noticed on an OpenVZ 7 instance with 2 vCPU that "multi core" CPU benchmark scores were lower than the single core score. But I have no idea of why, so I'll just mention it in case it's relevant.

TL;DR

All in all, if the price is right, OpenVZ 7 might still be a good alternative, if the limitations is something you can live with.

There much I don't know, so there might be vital points I've missed.

Neoon · December 2019

If the first run is slower then the second one, means its cached, most likely.
The first result shows the real speed.

InceptionHosting · December 2019

The BIG advantage and probably the only reason it still exists in the hosting world is that OpenVZ allows significantly greater density and as such significantly lower pricing while offering at LEAST 80% of the end use cases of full virtualization.

WSS · December 2019

@AnthonySmith said:
offering at LEAST 80% of the end use cases of full virtualization.

Like a really shitty firewall with limited iptables resources, and shared port queue across the entire hypervisor? Not very useful, overall, unless, ya know, you just want a playground for simple apps and/or a not-shared HTTP(x) instance. And, of course, proxies.

If the client isn't the kind to abuse resources, sure, OpenVZ is perfectly fine - but it's only easier to manage when your clients aren't shitheads. I know you prefer VZ, and it makes it easier to manage for the dregs of us LESplayers, but let's be honest- by "density", you mean overselling, plain and simple.

havoc · December 2019

I avoid it like the plague due to increased risk of overselling - which goes hand in hand with what Anthony said. Vcpu in that concept becomes a very fluid concept

Plus I don't like the container like nature of it.

Def has its place though. Eg storage or vpn vps

InceptionHosting · December 2019

@WSS said:

@AnthonySmith said:
offering at LEAST 80% of the end use cases of full virtualization.

Like a really shitty firewall with limited iptables resources, and shared port queue across the entire hypervisor? Not very useful, overall, unless, ya know, you just want a playground for simple apps and/or a not-shared HTTP(x) instance. And, of course, proxies.

If the client isn't the kind to abuse resources, sure, OpenVZ is perfectly fine - but it's only easier to manage when your clients aren't shitheads. I know you prefer VZ, and it makes it easier to manage for the dregs of us LESplayers, but let's be honest- by "density", you mean overselling, plain and simple.

Yes I mean overselling, that is why it is cheaper for end users who don’t give a shit about 99% of what you just said

But I also mean density as a general advantage.

WSS · December 2019

@AnthonySmith said:
Yes I mean overselling, that is why it is cheaper for end users who don’t give a shit about 99% of what you just said

I question these numbers. I've found that after one runs past "Hey my very own VPS! Cool!" and into "Whaddyamean I can't run my own kernel?", that the OpenVZ numeric is a steady decline. Sure, there are people who don't know, and those who don't care - but even for the worst-knowledgeable customers I've ever dealt with- they somehow managed to grasp that upgrading to an unsupported distribution was their fault, but it was still mine to fix it.

Other than simple users as mentioned above (proxy, small website, tiny minimal tasks (DNS/MX backup, etc) - who have you found that just don't care how limited it is, because they're saving a couple bucks?

skorous · December 2019

@WSS said: Other than simple users as mentioned above (proxy, small website, tiny minimal tasks (DNS/MX backup, etc) - who have you found that just don't care how limited it is, because they're saving a couple bucks?

I have a different question for you, what percentage of people do you think go outside of the stock OS packages to run their own kernel, etc...? Not being a dick, an honest question.

Other than my NAT boxes I only have one OVZ box and you're right, it's a simple webserver. I know I'm probably not your average user though.

InceptionHosting · December 2019

Ok fair of you to question, I only have my own experience from the last 6 years to fall back on to make my conclusions and estimates.

I have never hid my numbers, VZ oversells without end user impact 4:1 kvm 1.5 ish :1 and outside of hosting if containers work for the use case 8:1 density is not out of the question, kvm can’t even touch that.

Daniel · December 2019

KVM is part of the mainline Linux kernel, whereas OpenVZ is a third party kernel modification. This means OpenVZ is far more likely to disappear in the future. KVM is more future-proof.

The only real use case for OpenVZ that I can think of is cheap VPS hosting where the provider can oversell more easily. For most use cases, KVM is better, and for containerization on your own server, LXC is a better choice (it's also part of the mainline Linux kernel).

@flips said: Templates, disk usage lower (<400 MB for Debian 10)

I don't think there's actually a significant difference in disk space usage... There's not really a reason that an OpenVZ template would use less disk space, I guess other than the kernel not being part of the template?

@flips said: One thing I've noticed, is for disk I/O, I get something like this on my OpenVZ 7 instances

What disk I/O test is that? Don't trust dd, it's not reliable for benchmarking, and sequential writes are something you very rarely encounter in real life (random write and read speeds are more important). The best is probably fio but I'm not sure if it works on OpenVZ.

WSS · December 2019

@skorous said:

@WSS said: Other than simple users as mentioned above (proxy, small website, tiny minimal tasks (DNS/MX backup, etc) - who have you found that just don't care how limited it is, because they're saving a couple bucks?

I have a different question for you, what percentage of people do you think go outside of the stock OS packages to run their own kernel, etc...? Not being a dick, an honest question.

In my experience within the last three years? Probably at least 30% of the clients I've dealt with. Most of them try to blindly follow "Upgrade 101" pathways and completely break the userland, and get pissed when the OVZ can't just be restored because of it's shit init system.

@AnthonySmith said:
Ok fair of you to question, I only have my own experience from the last 6 years to fall back on to make my conclusions and estimates.

I have never hid my numbers, VZ oversells without end user impact 4:1 kvm 1.5 ish :1 and outside of hosting if containers work for the use case 8:1 density is not out of the question, kvm can’t even touch that.

Granted. It's also emulating a complete system verses an abstracted layer with shared freaking ports. OVZ does really poorly for the whole node when something gets attacked. That, the limited use (for my own personal needs), and the shitty bridged networking is why I'd rather pay a couple bucks more for KVM.

skorous · December 2019

@WSS said: In my experience within the last three years? Probably at least 30% of the clients I've dealt with. Most of them try to blindly follow "Upgrade 101" pathways and completely break the userland, and get pissed when the OVZ can't just be restored because of it's shit init system.

lol. I knew that group was there I just didn't expect it to be such a high percentage. I guess I assumed they'd know better.

vimalware · December 2019

Can wireguard kernel module be dkms/load-ed on a 4.15 openvz7 kernel?

APenguinWeb · December 2019

@vimalware said:
Can wireguard kernel module be dkms/load-ed on a 4.15 openvz7 kernel?

Kernel modules cannot be loaded from inside of a container, You would have to use the wireguard-go user space implementation which isn't as tested or as fast as the kernel one.

masedi · December 2019

openvz allows provider to oversold its resources, so the price is cheaper than KVM, isn't?
as a customer, I really don't like this

Daniel · December 2019

@APenguinWeb said: You would have to use the wireguard-go user space implementation which isn't as tested or as fast as the kernel one.

Disagree with "isn't as tested"... The Wireguard-go userspace implementation powers all the non-Linux OSes supported by Wireguard (including the Android app, iOS app, Windows version and MacOS version).

I think with OpenVZ you can install the Wireguard kernel module on the host then add a Wireguard interface to a VPS, but in that case the VPN needs to be configured on the host (which isn't possible on a regular VPS host). I definitely know this works with LXC.

havoc · December 2019

@AnthonySmith said:
Ok fair of you to question, I only have my own experience from the last 6 years to fall back on to make my conclusions and estimates.

I have never hid my numbers, VZ oversells without end user impact 4:1 kvm 1.5 ish :1 and outside of hosting if containers work for the use case 8:1 density is not out of the question, kvm can’t even touch that.

Interesting. Never saw numbers for it.

The pricing diffs seem smaller than those suggest though

jvnadr · December 2019

@masedi said: as a customer, I really don't like this

You should like it. it always depends, though, on your needs. E.g., if you want to host a small to medium website you can use an OpenVZ vps with much more resources than a KVM, to have plenty of space when your server have spikes. If the provider is good and does a proper and active monitoring of his system, he can oversell heavily the node and balance it perfectly, because usually most of the clients either idle or use much lower resources than the named ones.
Having a huge KVM in case of some spikes of a server, when in reality you use 20-25% max of the resources, is a big waste. Choosing a tight KVM on the other hand, won;t give you room when you face some more traffic than the usual or need to run some heavy tasks (compile, backups etc.).
Of course, if the hosting provider is shady and oversells 10000% the server, then, the issue is with the shady provider, not OVZ itself. After all, a bad provider can also oversell KVM (although it is more tricky and risky). Just buy from a respected provider and you'll be fine.

@WSS said: Probably at least 30% of the clients I've dealt with.

I guess this is a fair number, accurate. OTOH, not everybody really needs custom kernel. A small vpn, a monitoring system, most of the website hosting won't need a custom kernel. So, it is always getting what you really need in the best value for money ratio.

InceptionHosting · December 2019

@havoc, that is an ‘upto’ Number not a standard.

Neoon · December 2019

I get that with OVZ, you can fit more users on a node, but, the afford you need to put into it like migration and all that ovz stuff.
How does this balance out to KVM? In terms of money/time.

Proxmox dropped OVZ, then you needed to migrate to LXC.
Now OVZ 6 is EOL, needs to be migrated to OVZ 7, has issues with unstable modules etc..
I mean not only the migrations, but the time you need to invest, compared to KVM overall?

Since I only use KVM, I did not had any of these issues, including upgrading Proxmox 4.x, 5.x and 6.x.

PHP_Backend · December 2019

In my experience, my website faced more downtime with OVZ than KVM due to some external issue (like, other user facing attack or something, burst resource got took away from my process aggressively, provider forgot to monitor the node... etc)
So, KVM is a must go for me now.

flips · December 2019

@Daniel said:

@flips said: Templates, disk usage lower (<400 MB for Debian 10)

I don't think there's actually a significant difference in disk space usage... There's not really a reason that an OpenVZ template would use less disk space, I guess other than the kernel not being part of the template?

Seems to me that there is a difference, I also wondered why. I assumed there must be more shared that just kernel (something in /var, /sys, /lib, maybe, I dunno). There was a discussion on this in some other thread here, IIRC.
I guess I should do a dpkg --get-selections on a minimal OVZ VPS, and do --set-selections on a KVM.
(But I have struggled to get a KVM Debian 10 down under 1 GB, and the OVZ minimal template is 399 MB, IIRC.)

@flips said: One thing I've noticed, is for disk I/O, I get something like this on my OpenVZ 7 instances

What disk I/O test is that? Don't trust dd, it's not reliable for benchmarking, and sequential writes are something you very rarely encounter in real life (random write and read speeds are more important). The best is probably fio but I'm not sure if it works on OpenVZ.

I think that was the I/O part of bench.monster/speedtest.sh, so I think it's just dd, yes.
I've used that to get geekbench CPU.
I'm not sure which benchmark tests I should use.
Seems to me that the geekbench CPU tests are a bit heavy -- if I run them too often, @AnthonySmith, @cam, @seriesn etc might become mad at me ...
bench.sh doesn't do much for CPU testing IIRC, and nench.sh has a very short one.
Would be nice with a CPU bench somewhere in between nench and geekbench4.

As for comparing CPU between OpenVZ 7 and KVM VPS'es, I'm not sure number of cores compares too well, but I guess it depends on the density on the host on both platforms, and I can just assume OVZ numbers will be generally a bit lower than KVM ... (Just talking from a user perspective here.)

masedi · December 2019

@jvnadr said:

@masedi said: as a customer, I really don't like this

You should like it.

as i mentioned that i really don't like provider who overselling their resource

i know there are several Vz providers who don't oversell

Neoon · December 2019

@masedi said:

@jvnadr said:

@masedi said: as a customer, I really don't like this

You should like it.

as i mentioned that i really don't like provider who overselling their resource

i know there are several Vz providers who don't oversell

There is no issue with such, even your local ISP oversold your DSL line.
The issue is, if that overselling gets out of hand aka stuff goes bad.

poisson · December 2019

@flips said:
I've used that to get geekbench CPU.
I'm not sure which benchmark tests I should use.
Seems to me that the geekbench CPU tests are a bit heavy -- if I run them too often, @AnthonySmith, @cam, @seriesn etc might become mad at me ...
bench.sh doesn't do much for CPU testing IIRC, and nench.sh has a very short one.
Would be nice with a CPU bench somewhere in between nench and geekbench4.

As for comparing CPU between OpenVZ 7 and KVM VPS'es, I'm not sure number of cores compares too well, but I guess it depends on the density on the host on both platforms, and I can just assume OVZ numbers will be generally a bit lower than KVM ... (Just talking from a user perspective here.)

I have been running daily Geekbenches as part of extended testing on @seriesn's servers. Don't worry, he won't get mad at you. You just spend a few hours wondering why Geekbench isn't completing.

flips · December 2019

@poisson said:
I have been running daily Geekbenches ...

... as in bench.monster/speedtest.sh ?

seriesn · December 2019

@poisson said:

@flips said:
I've used that to get geekbench CPU.
I'm not sure which benchmark tests I should use.
Seems to me that the geekbench CPU tests are a bit heavy -- if I run them too often, @AnthonySmith, @cam, @seriesn etc might become mad at me ...
bench.sh doesn't do much for CPU testing IIRC, and nench.sh has a very short one.
Would be nice with a CPU bench somewhere in between nench and geekbench4.

As for comparing CPU between OpenVZ 7 and KVM VPS'es, I'm not sure number of cores compares too well, but I guess it depends on the density on the host on both platforms, and I can just assume OVZ numbers will be generally a bit lower than KVM ... (Just talking from a user perspective here.)

I have been running daily Geekbenches as part of extended testing on @seriesn's servers. Don't worry, he won't get mad at you. You just spend a few hours wondering why Geekbench isn't completing.

@flips said:

@poisson said:
I have been running daily Geekbenches ...

... as in bench.monster/speedtest.sh ?

I mean,
I have a script that runs on the background to ensure no one is hogging up the resources for too long. But that also depends on multiple other variables including nodes usage.

You know the usual no dick policy. And then @poisson will send a message, super confused "Yo dude, you broke my VM again"

Daniel · December 2019

@jvnadr said: I guess this is a fair number, accurate. OTOH, not everybody really needs custom kernel. A small vpn, a monitoring system, most of the website hosting won't need a custom kernel.

It's not just getting a custom kernel, it's also getting a modern kernel. Even OpenVZ7 still uses a 3.10 series kernel, so you're missing out on a lot of newer features. I know some newer features get backported but it's not very common.

@flips said: But I have struggled to get a KVM Debian 10 down under 1 GB

It's possible the kernel is larger than I remember, which would definitely cause a difference vs OpenVZ!

Is that KVM installed via a netinst CD? If you install via netinst and don't select any extra options (like the default system utilities) at the end of the installation then it should be pretty minimal. You can also create very minimal installations using debootstrap. Maybe I'll play around with Debian again when I'm back from vacation and see what can be done to reduce the installation size.

@flips said: so I think it's just dd, yes.

If it's just dd then the numbers don't really mean anything...

havoc · December 2019

Neoon said: even your local ISP oversold your DSL line.

Yup. 20:1 is the norm locally

Fortunately people caught on and ISPs started declaring it in marketing info. i.e. Normal we guarantee sub 20, pay for pro and it's 10

flips · December 2019

@seriesn said:

@poisson said:
I have been running daily Geekbenches ...

I mean,
I have a script that runs on the background to ensure no one is hogging up the resources for too long. But that also depends on multiple other variables including nodes usage.

You know the usual no dick policy. And then @poisson will send a message, super confused "Yo dude, you broke my VM again"

And here I try not to run geekbench more than once or twice a week ...

For my usage, the full geekbench cpu test is overkill, but nench.sh's test feels a bit too simplified.

@Daniel said:
It's not just getting a custom kernel, it's also getting a modern kernel. Even OpenVZ7 still uses a 3.10 series kernel, so you're missing out on a lot of newer features. I know some newer features get backported but it's not very common.

Should have been Linux 3.11 for Workgroups...

@flips said: But I have struggled to get a KVM Debian 10 down under 1 GB

It's possible the kernel is larger than I remember, which would definitely cause a difference vs OpenVZ!

Is that KVM installed via a netinst CD? If you install via netinst and don't select any extra options (like the default system utilities) at the end of the installation then it should be pretty minimal. You can also create very minimal installations using debootstrap. Maybe I'll play around with Debian again when I'm back from vacation and see what can be done to reduce the installation size.

I just started out from a provided image (on some maybe upgraded from Debian 9, then used deborphan etc to shrink it), should probably try from netinst or just debootstrap it, yes.

@flips said: so I think it's just dd, yes.

If it's just dd then the numbers don't really mean anything...

Noted.

WSS · December 2019

@Daniel said:
It's not just getting a custom kernel, it's also getting a modern kernel. Even OpenVZ7 still uses a 3.10 series kernel, so you're missing out on a lot of newer features. I know some newer features get backported but it's not very common.

This just isn't completely true. OpenVZ 6 with the 2.3.32-stab13x(something) backported some ctls used by systemd so Ubuntu 18 would work. As well, if you use KernelCare/kpatch, you will often get security, and firmware updates.

@flips said:
(But I have struggled to get a KVM Debian 10 down under 1 GB, and the OVZ minimal template is 399 MB, IIRC.)

You're going to want to use the netinst and/or alt-installer. The default kernel is a pig. Not as bad as CentOS, but it's a pig.

Pros and cons for OpenVZ 7 as compared to KVM

TL;DR

Comments