VirMach suspended my VPS yesterday due to bandwidth over usage. I got 1.95TB of bandwidth in this package. This VPS just hosts one website which is behind Cloudflare and according to CF, I have only used 17GB bandwidth from 1st Jan till 24 Jan.
Do you guys know if there is some bandwidth report or logs in VirMach control panel to check? Very strange that this website which
has never crossed 100GB in a month in last 2 years, all of a sudden crossed 1.95TB in 24 days.
I suspect it could be due to one plugin which caches JS and images and was taking up 30GB space, but I am not sure. I have disabled that plugin.
Or are there some logs/utility available in Debian to check?
This would allow you to establish if it was short period of time (so like DDOS, hacked server sending shit) or was constant over time - so maybe this plugin.
If you don't have firewall set to discard everything except Cloudflare using Cloudflare for "17GB of bandwidth" is kinda meh - your machine is still on the internet and talking to other things outside of Cloudflare.
--
For Debian - there is nothing installed by default to monitor traffic - if you installed vnstat earlier then you could also have nice stats https://humdi.net/vnstat/ - but that need to be installed earlier as is collects data as it go.
HetrixTools agent deployed? They have Network graph too.
This graph is only showing today's data. Can't seem to find custom date range. I have now installed vnstat so let's see. I am using ufw and have always blocked all ports except 21, 22, 53, 80, 123, 443, 465, 587, 993 for both incoming and outgoing.
That's your main issue - Ubuntu on a server, pah! Replace ufw with csf (if possible on that OS). Install modsecurity; free rules being better than none.
It wisnae me! A big boy done it and ran away.
NVMe2G for life! until death (the end is nigh)
@lesuser said: I added a WAF in Cloudflare to block all these user agents so let's see.
You might want to add them on the VM directly, because as you stated above, the traffic did not appear to come thru cloudflare.
Although the suggestion above is a good practice, I do not expect this will solve your issue. The traffic shows outgoing, and not reflected in cloudflare, so I expect the traffic is originating from your VM for reasons other than requests to your website..
Peace on earth will come to stay, when we all live as LESbians every day.
For staff assistance or support issues please use the helpdesk ticket system at https://support.lowendspirit.com/index.php?a=add
@lesuser said: I added a WAF in Cloudflare to block all these user agents so let's see.
You might want to add them on the VM directly, because as you stated above, the traffic did not appear to come thru cloudflare.
I think they are coming from Cloudflare but I am not very well versed in Linux related things so I am not sure. Here is one of the entries from nginx access logs. Let me see how can I define these rules in nginx as .htaccess works in apache.
I also have a cron job running which takes database and file backup and rsync it to another server every 30 minutes. If this job becomes wonky then the other server from another provider should have been suspended also as it has much lower bandwidth that this one.
Peace on earth will come to stay, when we all live as LESbians every day.
For staff assistance or support issues please use the helpdesk ticket system at https://support.lowendspirit.com/index.php?a=add
I don't think you can get constant 25MB/s on HTML files - it would need to be a lot of images/videos, big images or like hundreds of requests per second.
Here's a friggin' surprise, not:
ec2-47-128-96-140.ap-southeast-1.compute.amazonaws.com
What really pisses me off are the number of 'legitimate' services that use Hetzner and Amazon compute. Otherwise, I'd block their complete ranges - tried it once and screwed up a lot of software updates. Both providers harbour real problematic nefarious assholes. (Not sitting on the fence. )
@lesuser said: takes database and file backup and rsync it to another server every 30 minutes.
Have you monitored the size of these, let's face it, fairly frequent bundles? Do you periodically run full backups (say twice per day) and then differential ones? You may be better served by direct (differential) rsync of the files and remote database replication.
@lesuser What about blocking non-Cloudflare IPs with ufw?
for url in "https://www.cloudflare.com/ips-v4" "https://www.cloudflare.com/ips-v6"; do
curl -s $url | while read ip; do
sudo ufw allow from $ip to any port 443 comment "Cloudflare IP"
done
done
Seems like QN LAX having issues again. I'm trying to get into the facility so if anyone's there to let me in I'll give you guys some live updates on whether everything's on fire, otherwise I'll perform some maintenance while it's down anyway (mostly applying paste.)
Based on how slow they are at responding right now and probably receiving an insane volume of tickets though I'm not sure if I'll be able to get in. I'll still (maybe) try. With this strategy:
@VirMach said:
Seems like QN LAX having issues again. I'm trying to get into the facility so if anyone's there to let me in I'll give you guys some live updates on whether everything's on fire, otherwise I'll perform some maintenance while it's down anyway (mostly applying paste.)
Based on how slow they are at responding right now and probably receiving an insane volume of tickets though I'm not sure if I'll be able to get in. I'll still (maybe) try. With this strategy:
Looks like there's been a lot of complaints with QN lately.
You can block both in and out traffic for all ports using ufw and whitelist one port for Cloudflare ZeroTrust so you can access both HTTP and HTTPS with it.
Comments
Good evening
Hi imok
dnscry.pt - Public DNSCrypt resolvers hosted by LowEnd providers • Need a free NAT LXC? -> https://microlxc.net/
any flashes
I bench YABS 24/7/365 unless it's a leap year.
VirMach suspended my VPS yesterday due to bandwidth over usage. I got 1.95TB of bandwidth in this package. This VPS just hosts one website which is behind Cloudflare and according to CF, I have only used 17GB bandwidth from 1st Jan till 24 Jan.
Do you guys know if there is some bandwidth report or logs in VirMach control panel to check? Very strange that this website which
has never crossed 100GB in a month in last 2 years, all of a sudden crossed 1.95TB in 24 days.
I suspect it could be due to one plugin which caches JS and images and was taking up 30GB space, but I am not sure. I have disabled that plugin.
Or are there some logs/utility available in Debian to check?
Fast as fuck Core i9 VPS (aff) | Powerful AMD Ryzen VPS (aff)
This would allow you to establish if it was short period of time (so like DDOS, hacked server sending shit) or was constant over time - so maybe this plugin.
If you don't have firewall set to discard everything except Cloudflare using Cloudflare for "17GB of bandwidth" is kinda meh - your machine is still on the internet and talking to other things outside of Cloudflare.
--
For Debian - there is nothing installed by default to monitor traffic - if you installed vnstat earlier then you could also have nice stats https://humdi.net/vnstat/ - but that need to be installed earlier as is collects data as it go.
HetrixTools agent deployed? They have Network graph too.
Haven't bought a single service in VirMach Great Ryzen 2022 - 2023 Flash Sale.
https://lowendspirit.com/uploads/editor/gi/ippw0lcmqowk.png
This graph is only showing today's data. Can't seem to find custom date range. I have now installed
vnstatso let's see. I am usingufwand have always blocked all ports except 21, 22, 53, 80, 123, 443, 465, 587, 993 for both incoming and outgoing.Fast as fuck Core i9 VPS (aff) | Powerful AMD Ryzen VPS (aff)
I have noticed a massive increase in bots & scrapers, on a few of the sites that I host. Could be that.
It wisnae me! A big boy done it and ran away.
NVMe2G for life! until death (the end is nigh)
Full VPS Control PanelHere's my last 30 days traffic. From 25 Dec 2024 to 25 Jan 2025.
Fast as fuck Core i9 VPS (aff) | Powerful AMD Ryzen VPS (aff)
^ maxed out for a whole week doesn't look good.
.htaccess snippet, as an example:
BrowserMatchNoCase "libwww-perl" bad_bot
BrowserMatchNoCase "wget" bad_bot
BrowserMatchNoCase "LieBaoFast" bad_bot
BrowserMatchNoCase "Mb2345Browser" bad_bot
BrowserMatchNoCase "zh-CN" bad_bot
BrowserMatchNoCase "MicroMessenger" bad_bot
BrowserMatchNoCase "zh_CN" bad_bot
BrowserMatchNoCase "Kinza" bad_bot
BrowserMatchNoCase "Bytespider" bad_bot
BrowserMatchNoCase "Baiduspider" bad_bot
BrowserMatchNoCase "Sogou" bad_bot
BrowserMatchNoCase "Datanyze" bad_bot
BrowserMatchNoCase "AspiegelBot" bad_bot
BrowserMatchNoCase "adscanner" bad_bot
BrowserMatchNoCase "serpstatbot" bad_bot
BrowserMatchNoCase "spaziodat" bad_bot
BrowserMatchNoCase "undefined" bad_bot
BrowserMatchNoCase "petalbot" bad_bot
BrowserMatchNoCase "PetalBot" bad_bot
BrowserMatchNoCase "PerplexityBot" bad_bot
BrowserMatchNoCase "openai" bad_bot
BrowserMatchNoCase "GPTBot" bad_bot
BrowserMatchNoCase "SemrushBot" bad_bot
BrowserMatchNoCase "BLEXBot" bad_bot
SetEnvIfNoCase User-Agent .ahrefsbot. bad_bot
SetEnvIfNoCase User-Agent .semrush. bad_bot
SetEnvIfNoCase User-Agent .bytespyder. bad_bot
SetEnvIfNoCase User-Agent .bytedance. bad_bot
SetEnvIfNoCase User-Agent .claudebot. bad_bot
SetEnvIfNoCase User-Agent .magesift. bad_bot
SetEnvIfNoCase User-Agent .otbot. bad_bot
SetEnvIfNoCase User-Agent .petalbot. bad_bot
SetEnvIfNoCase User-Agent .Owler. bad_bot
SetEnvIfNoCase User-Agent .spider. bad_bot
SetEnvIfNoCase User-Agent .perplexity. bad_bot
Order Deny,Allow
Deny from env=bad_bot
Note: forum screwed the format..
SetEnvIfNoCase User-Agent .*ahrefsbot.* bad_botBan the feckers, ban 'em all !!!
It wisnae me! A big boy done it and ran away.
NVMe2G for life! until death (the end is nigh)
That's your main issue - Ubuntu on a server, pah! Replace ufw with csf (if possible on that OS). Install modsecurity; free rules being better than none.
It wisnae me! A big boy done it and ran away.
NVMe2G for life! until death (the end is nigh)
25 MB/s, 24/7 for 4 days
Haven't bought a single service in VirMach Great Ryzen 2022 - 2023 Flash Sale.
https://lowendspirit.com/uploads/editor/gi/ippw0lcmqowk.png
But why this bandwidth is not reflected in Cloudflare?
Fast as fuck Core i9 VPS (aff) | Powerful AMD Ryzen VPS (aff)
I added a WAF in Cloudflare to block all these user agents so let's see.
Fast as fuck Core i9 VPS (aff) | Powerful AMD Ryzen VPS (aff)
You might want to add them on the VM directly, because as you stated above, the traffic did not appear to come thru cloudflare.
Although the suggestion above is a good practice, I do not expect this will solve your issue. The traffic shows outgoing, and not reflected in cloudflare, so I expect the traffic is originating from your VM for reasons other than requests to your website..
Peace on earth will come to stay, when we all live as LESbians every day.
For staff assistance or support issues please use the helpdesk ticket system at https://support.lowendspirit.com/index.php?a=add
Because you have multiple entry points, Cloudflare is only one of them
I think they are coming from Cloudflare but I am not very well versed in Linux related things so I am not sure. Here is one of the entries from nginx access logs. Let me see how can I define these rules in nginx as
.htaccessworks in apache.47.128.96.140 - - [23/Jan/2025:01:41:40 -0500] "GET /product/clothing/track-suit/ HTTP/1.1" 200 33439 "-" "Mozilla/5.0 (compatible; Bytespider; [email protected]) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.0.0 Safari/537.36"Fast as fuck Core i9 VPS (aff) | Powerful AMD Ryzen VPS (aff)
I also have a cron job running which takes database and file backup and rsync it to another server every 30 minutes. If this job becomes wonky then the other server from another provider should have been suspended also as it has much lower bandwidth that this one.
Fast as fuck Core i9 VPS (aff) | Powerful AMD Ryzen VPS (aff)
Modsecurity with standard and custom rules really does solve so many day to day issues. I don't leave home without it.
@lesuser here is a link to a how to do modsecurity for Nginx
Peace on earth will come to stay, when we all live as LESbians every day.
For staff assistance or support issues please use the helpdesk ticket system at https://support.lowendspirit.com/index.php?a=add
Based on your comments I’d guess the machine got owned and is spewing traffic for someone else.
Restore from unrelated backups on a new machine at your new provider I guess.
I don't think you can get constant 25MB/s on HTML files - it would need to be a lot of images/videos, big images or like hundreds of requests per second.
Haven't bought a single service in VirMach Great Ryzen 2022 - 2023 Flash Sale.
https://lowendspirit.com/uploads/editor/gi/ippw0lcmqowk.png
Here's a friggin' surprise, not:
ec2-47-128-96-140.ap-southeast-1.compute.amazonaws.com
What really pisses me off are the number of 'legitimate' services that use Hetzner and Amazon compute. Otherwise, I'd block their complete ranges - tried it once and screwed up a lot of software updates. Both providers harbour real problematic nefarious assholes. (Not sitting on the fence.
)
It wisnae me! A big boy done it and ran away.
NVMe2G for life! until death (the end is nigh)
Have you monitored the size of these, let's face it, fairly frequent bundles? Do you periodically run full backups (say twice per day) and then differential ones? You may be better served by direct (differential) rsync of the files and remote database replication.
It wisnae me! A big boy done it and ran away.
NVMe2G for life! until death (the end is nigh)
Am I misremembering or has VirMach not been providing a stats chart for cpu usage on the statistics page?
Have the honor of being the crybaby who pays $20 for a 128MB VPS at VirMach in 2023.
@lesuser What about blocking non-Cloudflare IPs with ufw?
Seems like QN LAX having issues again. I'm trying to get into the facility so if anyone's there to let me in I'll give you guys some live updates on whether everything's on fire, otherwise I'll perform some maintenance while it's down anyway (mostly applying paste.)
Based on how slow they are at responding right now and probably receiving an insane volume of tickets though I'm not sure if I'll be able to get in. I'll still (maybe) try. With this strategy:
Looks like there's been a lot of complaints with QN lately.
https://lowendtalk.com/discussion/comment/4279444/#Comment_4279444
This thread is a good read.
The Ultimate Speedtest Script | Get Instant Alerts on new LES/LET deals | Cheap VPS Deals | VirMach Flash Sales Notifier
FREE KVM VPS - FreeVPS.org | FREE LXC VPS - MicroLXC
@lesuser
You can block both in and out traffic for all ports using ufw and whitelist one port for Cloudflare ZeroTrust so you can access both HTTP and HTTPS with it.
This works for me.
https://microlxc.net/
Get yourself a luxury vehicle and you'll be the first one getting in.
No hostname left!