nagios-event-triger-AutoRecover
思路：
use NRPE to execute the necessary commands on the remote hosts
In order to adapt the scheme from the Nagios docs to work on remote servers as well three things need to be done:
1.The command that is executed by the event handler script should be changed to use NRPE
2.On the remote machine the nagios user (under which the NRPE service is running) should be given some sudo rights so that it is actually allowed to start a service.
3.The NRPE configuration on the remote machine should of course be changed to include the new command(s) for starting services.
1. nagios manage server
（1）vi localhost.cfg
define service{
use generic-service
host_name test2.bigdata.com
service_description gmond
check_command check_nrpe_eventhandler!check_gmond
notifications_enabled 1
notification_interval 0
max_check_attempts 4
event_handler restart-service!gmond
}
define service{
use generic-service
host_name test2.bigdata.com
service_description mysqld
check_command check_nrpe_eventhandler!check_mysqld
notifications_enabled 1
notification_interval 0
max_check_attempts 5
event_handler restart-service!mysqld
}（2）vi commands.cfg
define command{
command_name check_nrpe_eventhandler
command_line $USER1$/check_nrpe -H $HOSTADDRESS$ -t 40 -c $ARG1$
}
define command {
command_name restart-service
command_line $USER1$/eventhandlers/event_handler_script.sh $SERVICESTATE$ $SERVICESTATETYPE$ $SERVICEATTEMPT$ $HOSTADDRESS$ $ARG1$ $SERVICEDESC$
}利用后面编写的通用的事件处理脚本模块文件 /usr/local/nagios/libexec/eventhandlers/event_handler_script.sh 传递监控服务所需的参数$ARG1$
2. on remote machine with NRPE running
（1）vi nrpe.cfg
command[check_gmond]=/usr/local/nagios/libexec/check_gmond command[restart_gmond]=/usr/bin/sudo /etc/init.d/gmond restart command[check_mysqld]=/usr/local/nagios/libexec/check_mysqld command[restart_mysqld]=/usr/bin/sudo /usr/local/nagios/libexec/restart_mysqld
(2)edit your service manage script on remote machine.
/usr/local/nagios/libexec/check_mysqld
/usr/local/nagios/libexec/restart_mysqld
3. 通用的事件处理脚本模块文件 /usr/local/nagios/libexec/eventhandlers/event_handler_script.sh内容如下
#!/bin/sh # # Event handler script for restarting the web server on the local machine # # Note: This script will only restart the web server if the service is # retried 3 times (in a "soft" state) or if the web service somehow # manages to fall into a "hard" error state. # update 2015/10/23 # version: 0.2 date=`date` # What state is the HTTP service in case "$1" in OK) # The service just came back up, so don't do anything... ;; WARNING) # We don't really care about warning states, since the service is probably still running... ;; UNKNOWN) # We don't know what might be causing an unknown error, so don't do anything... ;; CRITICAL) # We don't really care about warning states, since the service is probably still running... # Aha! The HTTP service appears to have a problem - perhaps we should restart the server... # Is this a "soft" or a "hard" state? case "$2" in # We're in a "soft" state, meaning that Nagios is in the middle of retrying the # check before it turns into a "hard" state and contacts get notified... SOFT) # What check attempt are we on? We don't want to restart the web server on the first # check, because it may just be a fluke! case "$3" in # Wait until the check has been tried 3 times before restarting the web server. # If the check fails on the 4th time (after we restart the web server), the state # type will turn to "hard" and contacts will be notified of the problem. # Hopefully this will restart the web server successfully, so the 4th check will # result in a "soft" recovery. If that happens no one gets notified because we # fixed the problem! 3) echo -n "Restarting service $6 (3rd soft critical state)...\n" # Call NRPE to restart the service on the remote machine /usr/local/nagios/libexec/check_nrpe -H $4 -c restart_$5 echo "$date -restart $6 on server $4 -at retry $3 times -SOFT" >> /tmp/eventhandlers ;; esac ;; # The HTTP service somehow managed to turn into a hard error without getting fixed. # It should have been restarted by the code above, but for some reason it didn't. # Let's give it one last try, shall we? # Note: Contacts have already been notified of a problem with the service at this # point (unless you disabled notifications for this service) HARD) echo -n "Restarting $6 service...\n" # Call the init script to restart the NRPE server echo "$date -restart $6 on server $4 -at retry $3 times -HARD" >> /tmp/eventhandlers /usr/local/nagios/libexec/check_nrpe -H $4 -c restart_$5 ;; esac ;; esac exit 0
另外有需要云服务器可以了解下创新互联scvps.cn，海内外云服务器15元起步，三天无理由+7*72小时售后在线，公司持有idc许可证，提供“云服务器、裸金属服务器、高防服务器、香港服务器、美国服务器、虚拟主机、免备案服务器”等云主机租用服务以及企业上云的综合解决方案，具有“安全稳定、简单易用、服务可用性高、性价比高”等特点与优势，专为企业上云打造定制，能够满足用户丰富、多元化的应用场景需求。
