Nagios is a network monitoring application which itself provides no actual monitoring but rather specializes in scheduling checks and notifications. As a module framework, it works very well and there are a lot of monitoring plugins and all told, there aren’t many (or any) systems that really compare, F/OSS, proprietary, or otherwise.
Since it’s not a complete solution in and of itself, I know at least I found it a bit daunting to get in to. So I got this book:
Building a Monitoring Infrastructure with Nagios by David Josephsen
It’s not a huge book like say, HP’s OpenView manual(s), so read it first.
Nagios is super cool. You build definitions for each host on your network and each service on each host. Nagios checks each service recording the service’s status. When a service fails, nagios will send a notification once it is sure the service is down and then periodically until it comes back up.
Fine, that’s the basic premise. Now the configuration works pretty well because any host can inherit it’s configuration from any other host definition including host definition templates. So set your general parameters once, and then override where necessary. It’s the same for services. And you also have host and service groups which allow you logically group hosts (or services).
Nagios doesn’t have any built-in way to check services, it’s all through plugins. A plugin is simply an external script or program which exits with a status of 0 for OK, 1 for warning, or 2 for critical and optional 1 line of standard output for status text. Nagios has many standard plugins available, for example the check_ping plugin. This plugin is a little wrapper script which is invoked with arguments specifying the warning and critical thresholds for response time and packet loss. So in testing a plugin, you can simply invoke the plugin with the arguments that Nagios would be feeding it.
Now if a service goes down, Nagios will check if the host is down. Again, this is a plugin of the same type as for the service. Typically, this means check_ping. So you don’t really need to have a check_ping “service” check, just for the host. So if your host runs a webserver, you would use check_http and if that fails, Nagios will check_ping on the host to see if that’s down. If the host is down, well then obviously all services on that host are a write-off so Nagios will send a notification for that host once rather than for each individual service.
And when I say Nagios will send a notification, it doesn’t know how to do that either. Notifications are also defined but typically the stock notification will suffice. On Fedora, it uses the “mail” program to send a mail message.
Ah, so who does it notify? Well, each service and each host defines contact groups and also contact hours. So Nagios will notify everyone in a contact group if it’s during notification hours. So you can monitor your development systems as well as production ones and only get notifications when appropriate.
Nagios also provides escalations. So if a service (or host) remains down, you can define an escalation path. Maybe level one is help desk, and if they don’t respond, it escalates to supervisors as well, and if they still don’t respond, then it escalates to on-call staff, managers and eventually the head cheese.
What else is cool? Oh yeah, parent-child relationships. On each host, you can define parent hosts. So if you have say several routers throughout your network, connectivity to hosts would depend on connectivity to their routers. So if a router goes down, as with services on a host, Nagios will know to only notify of the router being down and not all the children individually.
There is also an agent for Nagios called the NRPE. It is totally optional but if you want local system checks, like disk, CPU, checking running processes, and not just network service checks, then NRPE lets you do this. Install NRPE on your monitored hosts, and NRPE is available for “Linux” and Windows, and it, I think, is like a little baby Nagios invoked by the mothership. So you install service check plugins with the NRPE and then on the server, your service checks are like check_nrpe!check_disk … or something like that so the server sends the service check to the NRPE on the monitored system. I haven’t used this yet, but will definitely be doing so.
The NEB is another cool part of Nagios. The Nagios Event Broker is an interface where can write programs which hook into Nagios’s regular operations. There’s a couple dozen callback functions you can hook into and this makes the possibilities for Nagios virtually endless.
The part I’ve left for last is the user interface. Well, once again, there is none. You configure it, it fires off notifications, that’s the core. There is a web interface Nagios provides you can use if you want. It will show you host and service status and you can acknowledge alarms through it and schedule downtime for hosts. Now you can hook in more functionality, for example, historical graphs can be very useful. If you’re checking disk usage with Nagios, why not keep a record of it? Well, when you get a service check result back, it comes back with one line of text from standard output, right? Well, there’s packages that will build graphs from this data so that you can have your service status and historical reports too! Josephsen has a pretty extensive discussion on doing this kind of stuff and some great info on some of the options out there.
So, yeah. Get that book! Use Nagios! Monitor everything with it! Let it tell you when your toast is toasted or your beer needs a refill!
- Arch