Check and Restart Apache if it is malfunctioning
(not returning HTML content) shell script
One of the company Debian Lenny 5.0 Webservers, where I'm working
as sys admin sometimes stops to properly server HTTP
requests.
Whenever this oddity happens, the Apache server seems to be running
okay but it is not failing to return requested content
I can see the webserver listens on port 80 and establishing
connections to remote hosts - the apache processes show normally as
I can see in netstat ...:
apache:~# netstat -enp 80
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address Foreign Address State User Inode
PID/Program name
tcp 0 0 xxx.xxx.xxx.xx:80 46.253.9.36:5665 SYN_RECV 0 0 -
tcp 0 0 xxx.xxx.xxx.xx:80 78.157.26.24:5933 SYN_RECV 0 0 -
...
Also the apache forked child processes show normally in process
list:
apache:~# ps axuwwf|grep -i apache
root 46748 0.0 0.0 112300 872 pts/1 S+ 18:07 0:00 \_ grep -i
apache
root 42530 0.0 0.1 217392 6636 ? Ss Mar14 0:39 /usr/sbin/apache2 -k
start
www-data 42535 0.0 0.0 147876 1488 ? S Mar14 0:01 \_
/usr/sbin/apache2 -k start
root 28747 0.0 0.1 218180 4792 ? Sl Mar14 0:00 \_ /usr/sbin/apache2
-k start
www-data 31787 0.0 0.1 219156 5832 ? S Mar14 0:00 | \_
/usr/sbin/apache2 -k start
In spite of that, in any client browser to any of the Apache
(Virtual hosts) websites, there is no HTML content
returned...
This weird problem continues until the Apache webserver is
retarted.
Once webserver is restarted everything is back to normal.
I use
Apache Check
Apache shell script set on few remote hosts to regularly check
with nmap if port 80 (www) of my server is open and responding,
anyways this script just checks if the open and reachable and thus
using it was unable to detect Apache wasn't able to return back
HTML content.
To work around the malfunctions I wrote
tiny script -
retart_apache_if_empty_content_is_returned.sh
The scripts idea is very simple;
A request is made a remote defined host with lynx text browser,
then the output of lines is counted, if the output returned by
lynx -dump http://someurl.com is less than the number
returned whether normally invoked, then the script triggers an
apache init script restart.
I've set the script to periodically run in a cron job, every 5
minutes each hour.
# check if apache returns empty content with lynx and if yes
restart and log it
*/5 * * * * /usr/sbin/restart_apache_if_empty_content.sh
>/dev/null 2>&1
This is not perfect as sometimes still, there will be few minutes
downtime, but at least the downside will not be few hours until I
am informed ssh to the server and
restart Apache manually
...
A quick way to download and set from cron execution my script every
5 minutes use:
apache:~# cd /usr/sbin
apache:/usr/sbin# wget -q
http://pc-freak.net/bscscr/restart_apache_if_empty_content.sh
apache:/usr/sbin# chmod +x restart_apache_if_empty_content.sh
apache:/usr/sbin# crontab -l > /tmp/file; echo '*/5 * * * *'
/usr/sbin/restart_apache_if_empty_content.sh 2>&1
>/dev/null