Monitor website uptime

somik · July 2023

So here's the situation.

I was using a custom script to monitor my server's uptime. It would monitor the ports listed and send me a message on Telegram (using bot API) if any of those goes down or comes back online.

Up until recently, it was mostly working fine until I updated the script, upon which the script stopped working partially. It will monitor uptime but no longer send me notifications if something goes offline or comes back online. It still sends notifications regarding the health status.

Ah, did I mention that I deleted the original backup (that I made before the modification) from the server during a previous cleanup? Anyway, I do not have enough time to fix the script and was looking for a simple opensource script/project that can do the same.

Requirements:

Monitor uptime for ports and websites
Send message via telegram if something goes down or comes back up
Send a daily telegram message with the uptime status + monitor duration
(optional) Monitor latency
(optional) Does not require docker or nodejs
(optional) PHP or python based script

If you guys know of such a project/script, do let me know.

PS. I am NOT looking for a public service that does the same, unless I can self host it. Many of the servers that require monitoring are on local network without publicly open ports and I would like to keep it the same.

For those who do not have time to read the full thread, here are the scripts being discussed here:

@msatt recommended: https://github.com/louislam/uptime-kuma/wiki

@Asim recommended: https://hub.docker.com/r/jasonrivers/nagios

@Mason recommended: https://www.librenms.org/ | https://github.com/librenms/librenms

@AuroraZero recommended: https://github.com/bderenzo/tinystatus

@iandk's custom solution: https://github.com/iandk/pymon

msatt · July 2023

Sounds like you want uptime kuma.
Works fine on my internal systems.

Asim · July 2023

@somik Use this docker container https://hub.docker.com/r/jasonrivers/nagios
with a custom hook that calls the telegram API to send out HOST DOWN and SERVICE DOWN alerts

That's what I am doing

Jab · July 2023

Send a daily telegram message with the uptime status + monitor duration

You guys sure that uptime kuma has that?

msatt · July 2023

uptime kuma does support telegram notifications.
However I can't see a way to get daily status messages.
For me, I don't need a message to tell me when things are ok as they just waste my time. I do need to know when something is down, so do monitor uptime kuma functionality via a different mechanism / alert (just incase).

skorupion · July 2023

seems like self-hosted grafana would be the best here

rcy026 · July 2023

Nagios or Icinga could certainly do it, but the learning curve is steep. Uptime Kuma would probably have you up and running in minutes but I think you will have to do some custom api calls to get the exact functionality you want.

I think things like sending a daily telegram with active statuses is something that you will have to develop yourself. It should not require many lines of code with a good api, but I doubt that there is a "out of the box" solution that fits you perfectly. Nagios and Icinga certainly have the capabilities to do it but setting it up correctly is probably no easier task then simply writing a shell script for it against Uptime Kuma's api.

What you want to monitor is pretty basic, so I think something like Uptime Kuma would handle that without problems. Nagios and Icinga is a lot more powerful but takes some effort to understand and use, and honestly it's just overkill in this case.
You are obviously no stranger to scripting so creating a simple script that gives you the functionality Kuma lacks should not be a problem for you, so I would say go with Kuma.

rcy026 · July 2023

@skorupion said:
seems like self-hosted grafana would be the best here

Please explain how? Grafana is a visualization tool, it cant even do the most basic monitoring without help.
You would need something like Telegraf or node-exporter, or some kind of custom solution that feeds a Prometheus, Influx or similar that Grafana can fetch data from. You could use the data from Icinga, Nagios or Kuma, but then Grafana would basically be totally redundant and just add unnecessary complexity.
All are good solutions when working together, no question about that, but Grafana alone will not make it.

somik · July 2023

@msatt said:
Sounds like you want uptime kuma.
Works fine on my internal systems.

Thanks, i'll take a look at it over this weekend.

@Asim said:
@somik Use this docker container https://hub.docker.com/r/jasonrivers/nagios
with a custom hook that calls the telegram API to send out HOST DOWN and SERVICE DOWN alerts

That's what I am doing

I am trying to stay away from dockers as they are usually more resource heavy, but if there isn't a better option, i'll take a look at it. Thanks.

@Jab said:

Send a daily telegram message with the uptime status + monitor duration

You guys sure that uptime kuma has that?

The reason I want the daily status is because my monitoring server went down and I did not have a backup server to tell me that my monitoring server is down. So i guess i can put the idlers to good use in this case to monitor my public servers and main monitoring server.

@msatt said:
uptime kuma does support telegram notifications.
However I can't see a way to get daily status messages.
For me, I don't need a message to tell me when things are ok as they just waste my time. I do need to know when something is down, so do monitor uptime kuma functionality via a different mechanism / alert (just incase).

All goes well until the monitoring server goes down But As i said, i'll setup 2 servers so the could monitor each other.

@skorupion said:
seems like self-hosted grafana would be the best here

grafana requires extensive setups to monitor stuff and needs more setup to enable telegram (if they even have it).

@rcy026 said:
Nagios or Icinga could certainly do it, but the learning curve is steep. Uptime Kuma would probably have you up and running in minutes but I think you will have to do some custom api calls to get the exact functionality you want.

Custom API calls should be ok. I mean that's why I was looking for opensource in the first place.

@rcy026 said:
I think things like sending a daily telegram with active statuses is something that you will have to develop yourself. It should not require many lines of code with a good api, but I doubt that there is a "out of the box" solution that fits you perfectly. Nagios and Icinga certainly have the capabilities to do it but setting it up correctly is probably no easier task then simply writing a shell script for it against Uptime Kuma's api.

Thanks, i'll try it.

@rcy026 said:
What you want to monitor is pretty basic, so I think something like Uptime Kuma would handle that without problems. Nagios and Icinga is a lot more powerful but takes some effort to understand and use, and honestly it's just overkill in this case.
You are obviously no stranger to scripting so creating a simple script that gives you the functionality Kuma lacks should not be a problem for you, so I would say go with Kuma.

Exactly. No need detailed monitoring. Just if the server/port is up or down.

@rcy026 said:

@skorupion said:
seems like self-hosted grafana would be the best here

Please explain how? Grafana is a visualization tool, it cant even do the most basic monitoring without help.
You would need something like Telegraf or node-exporter, or some kind of custom solution that feeds a Prometheus, Influx or similar that Grafana can fetch data from. You could use the data from Icinga, Nagios or Kuma, but then Grafana would basically be totally redundant and just add unnecessary complexity.
All are good solutions when working together, no question about that, but Grafana alone will not make it.

Grafana + Telegraf + Influx is required to do that. I did look at the solution, but it is not for me. Way too high resource usage.

Mason · July 2023

Haven't seen this mentioned yet, so I'll throw in my solution of choice in this space--

LibreNMS
https://www.librenms.org/
https://github.com/librenms/librenms

Granted... this may be a bit overkill for what you're looking for and you'll have to evaluate if the resources required are worth the convenience, but it has a TON of capability right out of the box. I've used it quite a bit for monitoring my servers, websites, and services (via port monitoring) and get notifications through Discord. Has needed very little maintenance over the years.

Going through your requirements....

Monitor uptime for ports and websites: Check (Nagios plugin w/ TCP monitor) and check
Send message via telegram if something goes down or comes back up: Check (setup your alerts and use the Telegram transport)
Send a daily telegram message with the uptime status + monitor duration: Unknown (looks like it may be possible, but may not do everything you want)
(optional) Monitor latency: Check (included in normal monitoring/perormance graphs; can set it up for alerts too)
(optional) Does not require docker or nodejs: Check (main install method does not use Docker nor NodeJS)
(optional) PHP or python based script: Check (core is PHP-based and looks like most plugins/extensions are PHP or Python scripts)

P.S. Munzy (don't recall her account name here, but I'm pretty sure she is on LES too) has a convenient script to setup snmpd on the servers you wish to monitor with LibreNMS -> https://github.com/Munzy/recipes/blob/master/recipes/debian/snmpd-install.sh

somik · July 2023

@Mason said:
Haven't seen this mentioned yet, so I'll throw in my solution of choice in this space--

LibreNMS
https://www.librenms.org/
https://github.com/librenms/librenms

Granted... this may be a bit overkill for what you're looking for and you'll have to evaluate if the resources required are worth the convenience, but it has a TON of capability right out of the box. I've used it quite a bit for monitoring my servers, websites, and services (via port monitoring) and get notifications through Discord. Has needed very little maintenance over the years.

It would be a bit of a overkill, but compared to running docker engine + npm, it should be lower resource consuming given my server is already running nginx and php. Thanks, I'll give it a try.

@Mason said:
Going through your requirements....

Monitor uptime for ports and websites: Check (Nagios plugin w/ TCP monitor) and check

Wow, thanks a lot for linking the plugins! The main issue of going into a software ecosystem with plugins is that you dont know which are good and which are just "meh". With this, my main issue is solved.

@Mason said:
2. Send message via telegram if something goes down or comes back up: Check (setup your alerts and use the Telegram transport)

Awesome! Thanks!

@Mason said:
3. Send a daily telegram message with the uptime status + monitor duration: Unknown (looks like it may be possible, but may not do everything you want)

As I said, this is cause I was running only 1 monitoring server and wanted to know if the monitoring server itself is up or not. So this is solved by using another monitoring server/service on different network to monitor my main monitoring server.

@Mason said:
4. (optional) Monitor latency: Check (included in normal monitoring/perormance graphs; can set it up for alerts too)

I realize that unlike last time, most tends to support this feature. This is a very important feature and I dont know why last time it was a "premium" feature...

@Mason said:
5. (optional) Does not require docker or nodejs: Check (main install method does not use Docker nor NodeJS)

And that's at least 50% lower load

@Mason said:
6. (optional) PHP or python based script: Check (core is PHP-based and looks like most plugins/extensions are PHP or Python scripts)

Great! No need to install additional resource heavy services to run this!

@Mason said:
P.S. Munzy (don't recall her account name here, but I'm pretty sure she is on LES too) has a convenient script to setup snmpd on the servers you wish to monitor with LibreNMS -> https://github.com/Munzy/recipes/blob/master/recipes/debian/snmpd-install.sh

Thanks, I took a short look at it, but will dive deep over this weekend.

Thanks everyone for your comments.

Currently, after a short look, I am leaning towards uptime kuma and LibreNMS. I'll probably deploy uptime kuma in the internal system where I have enough resource to run docker containers and deploy librenms in the public server to monitor my public servers as well as the server running uptime kuma.

If you have a better suggestion, do let me know. All comments are appreciated.

rcy026 · July 2023

LibreNMS is a totally awesome piece of software. I manage an installation that monitors a big network of layer 2 switches. It has full monitoring and alerting on every single port (something like 60k ports). It also monitors cpu, temperature, memory etc on every device. It monitors routing (both ospf and bgp) and spanning tree changes. It creates graphs over bandwidth and load on every single port in the network. And the best thing is that it is very close to fully automated, it detects newly installed switches and adds them to the correct monitors all by itself and the success rate is probably something like 99.9%.
It is a 4 node cluster, but the clustering in Libre is a childsplay to setup compared to most other solutions.

So not to take anything away from Libre, but I would not recommend it for a case like this. I consider Libre more of a system monitor, not a service monitor. If you want to know exactly what's happening with your webservers, such as the temperature of the motherboard, how much memory it uses, current io throughput or if someone accidently unplugs a network cable and things like that, use Libre. If you just want to know if something is responding on port 443, use something else. I'm not saying Libre cant do it, I'm just saying that it is not where it shines.

And if you think LibreNMS will use less resources then Uptime Kuma for doing the same thing, I'm afraid you are mistaken. No way, not even close. Not for small scale. Libre would probably scale like a million times better than Kuma when you get to large installations, but for small easy tasks Kuma's lack of features and sqlite backend will outrun Libre by magnitudes.
I'm guessing the database alone in Libre requires more resources then the entire Kuma docker instance ever will do.
And this is coming from someone that absolute loves LibreNMS and has never been a big fan of Kuma, so if I were to be biased towards anything it would be towards Libre.

TerraNode · July 2023

With prtg, the free version of 100 sensors, you can do that. When I did my thesis, I even created a task that monitored a service every 30 seconds and if it was down, I would send a script to be executed on the server to start the service, you can send telegram messages, You have an app, and integration with various platforms

iandk · July 2023

@somik
I'm currently hacking together a Python script that implements this

Monitoring types implemented so far:

ping
port
http(s) (also validates status code and SSL cert)
keyword

I'll push it to GitHub in the coming days, just need to clean up the code a bit
Feel free to tag or DM me in case I forget

dwight · July 2023

You can write your own script that automatically sends requests to the website every 5 minutes.

sh97 · July 2023

@iandk said:
@somik
I'm currently hacking together a Python script that implements this

Monitoring types implemented so far:

ping

port

http(s) (also validates status code and SSL cert)

keyword

I'll push it to GitHub in the coming days, just need to clean up the code a bit
Feel free to tag or DM me in case I forget

I'm also interested in this since it's python. Do put a message here when it's done. Thanks!

AuroraZero · July 2023

I use a small program call Tiny Status. Basically bash scripting to ping sites and IPs, then it outputs to an index.html file.

I added a bit if scripting to hit my email of something happens and forward it through gmail to send a message to my phone in text form.

I set up a cron to run it on one minute intervals and it has worked well thus far, for monitoing just the uptime of things.

Link if you are interested:

https://github.com/bderenzo/tinystatus

Link where I use it:

https://status.yetinode.com

Edit: on mobile had to fix some spelling and added an example

Mason · July 2023

@iandk said:
@somik
I'm currently hacking together a Python script that implements this

Monitoring types implemented so far:

ping

port

http(s) (also validates status code and SSL cert)

keyword

I'll push it to GitHub in the coming days, just need to clean up the code a bit
Feel free to tag or DM me in case I forget

Nice! Please share when available

I took a stab at this several years ago. Feel free to steal any code -- https://github.com/masonr/pyPatrol-node. Implemented ping (v4 and v6), http (with or without keyword), ssl certs, tcp ports, and steam servers. Code and docs are all in there to get ideas.

Eventual goal was to have a self hosted distributed status service that one could deploy on their own. With 3+ distributed "nodes" being controlled by a central "server" (unfinished code here: https://github.com/masonr/pyPatrol-server). I kinda lost interest and priorities shifted -- gee, who would have thought having kids would suck up all your free time...

somik · July 2023

@iandk said:
@somik
I'm currently hacking together a Python script that implements this

Monitoring types implemented so far:

ping

port

http(s) (also validates status code and SSL cert)

keyword

I'll push it to GitHub in the coming days, just need to clean up the code a bit
Feel free to tag or DM me in case I forget

Sure! Would love to try out your script! A basic python based uptime monitoring would be the best as you can run it on almost any server. Do you have any plans to implement a notification service either by email or telegram?

@dwight said:
You can write your own script that automatically sends requests to the website every 5 minutes.

If you read my first post, I already have written my own script, but the issue is it is no longer working and I do not have the time required to debug or troubleshoot it. So I am looking for a off the shelf solution here.

@AuroraZero said:
I use a small program call Tiny Status. Basically bash scripting to ping sites and IPs, then it outputs to an index.html file.

I added a bit if scripting to hit my email of something happens and forward it through gmail to send a message to my phone in text form.

I set up a cron to run it on one minute intervals and it has worked well thus far, for monitoing just the uptime of things.

Link if you are interested:

https://github.com/bderenzo/tinystatus

Link where I use it:

https://status.yetinode.com

Edit: on mobile had to fix some spelling and added an example

That looks like a good script, but I already have my own PHP script that can monitor the status. What broke is the logic that determines when to send the notification to the user when something is down. So I am more interested in that.

@Mason said:

@iandk said:
@somik
I'm currently hacking together a Python script that implements this

Monitoring types implemented so far:

ping

port

http(s) (also validates status code and SSL cert)

keyword

I'll push it to GitHub in the coming days, just need to clean up the code a bit
Feel free to tag or DM me in case I forget

Nice! Please share when available

I took a stab at this several years ago. Feel free to steal any code -- https://github.com/masonr/pyPatrol-node. Implemented ping (v4 and v6), http (with or without keyword), ssl certs, tcp ports, and steam servers. Code and docs are all in there to get ideas.

Eventual goal was to have a self hosted distributed status service that one could deploy on their own. With 3+ distributed "nodes" being controlled by a central "server" (unfinished code here: https://github.com/masonr/pyPatrol-server). I kinda lost interest and priorities shifted -- gee, who would have thought having kids would suck up all your free time...

Sounds like a complete solution for uptime monitoring. Too bad you were unable to finish it. But looking at the bright side, your time was taken up by something much more important, so good on you for that. If you have kids it's always better to give them as much time as possible so they can grow up having loving parents.

Anyway, I got the uptime kuma setup on my internal server and it has been monitoring uptime for a few days. I have yet to try out the alerts/notifications but on it's own, this seems ok. Now just need a simple monitoring script to monitor my uptime monitor server...

Here's to hoping @iandk got the script that can do it!

iandk · July 2023

https://github.com/iandk/pymon

I‘m not a python dev, so there’s probably some room for improvements
So far only telegram is implemented for notifications, I’ll add SMTP next week

somik · July 2023

@iandk said: I‘m not a python dev

Lets see... custom installer script, use of python venv, install script sets up required python modules, use of seperate json files for servers and settings . . . are you sure you are NOT a python developer? You seem to be good enough to call yourself one though!

Great work on the script. I'll give it a try once i get some free time to test it out, probably over the next weekend.

pupilcc · July 2023

I always learn a lot from LES.

@msatt said:
Sounds like you want uptime kuma.
Works fine on my internal systems.

I greatly appreciate the valuable knowledge I gain from LES. Thank you for providing such helpful resources.

contactwajeeh · July 2023

@AuroraZero said:
I use a small program call Tiny Status. Basically bash scripting to ping sites and IPs, then it outputs to an index.html file.

I added a bit if scripting to hit my email of something happens and forward it through gmail to send a message to my phone in text form.

I set up a cron to run it on one minute intervals and it has worked well thus far, for monitoing just the uptime of things.

Link if you are interested:

https://github.com/bderenzo/tinystatus

Link where I use it:

https://status.yetinode.com

Edit: on mobile had to fix some spelling and added an example

Had a quick question, I had set this up and serving the HTML page with nginx located at /var/www/html/index.html. I am running the following command in cron to run this script every minute.

* * * * * /root/tinystatus/tinystatus > /var/www/html/index.html

But all it misses the checks file and just outputs the page without any checks performed. But if ran manually, correct page gets generated. Any idea why the different behavior

AuroraZero · July 2023

There is a program called tinystatus-runner. You should be using that and not calling tiny-status directly.

https://github.com/dusansimic/tinystatus-systemd

Grab that package unzip, read the Github instructions, call tinystatus-runner instead of tiny-status. The package should have tinystatus.service and tinystatus-runner files.

Guess I should have added these details, but honestly did not think anyone would be interested really.

smart2host · August 2023

uptimerobot can be used for free
has free 1-minute monitoring

as options is exactly what you are looking for

somik · August 2023

@smart2host said:
uptimerobot can be used for free
has free 1-minute monitoring

as options is exactly what you are looking for

You missed the part about me not looking for a public service, but a script i can self host.

@somik said: PS. I am NOT looking for a public service that does the same, unless I can self host it.

smart2host · August 2023

ok sorry for mistake

Monitor website uptime

Requirements:

Comments