Correct my script that "kill Apache and all sub-processes" if server load is over 7 for the 1 minute...
Budget: $10 – $30 CAD
I'm looking for help to correct the following script that does the following if the server is under a DDOS attack and the server's 1 minute load is over 7...
- kill Apache and all subprocesses
- wait 30 seconds
- restart Apache
...I currently have the following script...
----------------------------
#!/bin/bash
load=$(w | grep load | awk '{print $10}' | cut -d , -f1 | cut -d . -f1)
load2=7
if [[ $load -ge $load2 ]]; then
echo "$load"
systemctl restart httpd
for pid in $(ps -ef | grep httpd | awk '{print $2}'); do kill -9 $pid;done
sleep 30; systemctl restart httpd
else
echo "normal all ok"
fi
----------------------------
...however, when the above script does execute, it kills Apache and all subprocesses as it should; however, after waiting 30 seconds, Apache does not restart and returns the errors mentioned below when I try to restart Apache manually at the command prompt. The only way I can get Apache restarted again is by using WHM and selecting "Home » Restart Services » HTTP Server (Apache)"
============================================
[root@primary2 ~]# /scripts/restartsrv_httpd
Waiting for “httpd” to start ……info [restartsrv_httpd] systemd failed to start the service “httpd” (The “/usr/bin/systemctl restart httpd.service --no-ask-password” command (process 7538) reported error number 1 when it ended.): Job for httpd.service failed because the control process exited with error code. See "systemctl status httpd.service" and "journalctl -xe" for details.
…Waiting for httpd,litespeed,lscgid to shutdown ... not running.
info [restartsrv_httpd] systemd failed to start the service “httpd” (The “/usr/bin/systemctl restart httpd.service --no-ask-password” command (process 7583) reported error number 1 when it ended.): Job for httpd.service failed because the control process exited with error code. See "systemctl status httpd.service" and "journalctl -xe" for details.
…info [restartsrv_httpd] The service 'httpd' failed to restart at least three times. The system will now rebuild the httpd.conf file.
info [restartsrv_httpd] systemd failed to start the service “httpd” (The “/usr/bin/systemctl restart httpd.service --no-ask-password” command (process 7941) reported error number 1 when it ended.): Job for httpd.service failed because the control process exited with error code. See "systemctl status httpd.service" and "journalctl -xe" for details.
…failed.
Cpanel::Exception::Services::StartError
Service Status
undefined status from Cpanel::ServiceManager::Services::Httpd
Service Error
(XID a8y8dp) The “httpd” service failed to start.
Startup Log
Jan 04 01:52:00 primary2.aliasservername.com systemd[1]: Starting Apache web server managed by cPanel EasyApache...
Jan 04 01:52:02 primary2.aliasservername.com restartsrv_httpd[7944]: [Tue Jan 04 01:52:02.382417 2022] [core:emerg] [pid 8017] (28)No space left on device: AH00023: Couldn't create the mpm-accept mutex
Jan 04 01:52:02 primary2.aliasservername.com restartsrv_httpd[7944]: (28)No space left on device: could not create accept mutex
Jan 04 01:52:02 primary2.aliasservername.com restartsrv_httpd[7944]: AH00015: Unable to open logs
Jan 04 01:52:02 primary2.aliasservername.com systemd[1]: httpd.service: control process exited, code=exited status=1
Jan 04 01:52:02 primary2.aliasservername.com systemd[1]: Failed to start Apache web server managed by cPanel EasyApache.
Jan 04 01:52:02 primary2.aliasservername.com systemd[1]: Unit httpd.service entered failed state.
Jan 04 01:52:02 primary2.aliasservername.com systemd[1]: httpd.service failed.
httpd has failed. Contact your system administrator if the service does not automagically recover.
============================================
- kill Apache and all subprocesses
- wait 30 seconds
- restart Apache
...I currently have the following script...
----------------------------
#!/bin/bash
load=$(w | grep load | awk '{print $10}' | cut -d , -f1 | cut -d . -f1)
load2=7
if [[ $load -ge $load2 ]]; then
echo "$load"
systemctl restart httpd
for pid in $(ps -ef | grep httpd | awk '{print $2}'); do kill -9 $pid;done
sleep 30; systemctl restart httpd
else
echo "normal all ok"
fi
----------------------------
...however, when the above script does execute, it kills Apache and all subprocesses as it should; however, after waiting 30 seconds, Apache does not restart and returns the errors mentioned below when I try to restart Apache manually at the command prompt. The only way I can get Apache restarted again is by using WHM and selecting "Home » Restart Services » HTTP Server (Apache)"
============================================
[root@primary2 ~]# /scripts/restartsrv_httpd
Waiting for “httpd” to start ……info [restartsrv_httpd] systemd failed to start the service “httpd” (The “/usr/bin/systemctl restart httpd.service --no-ask-password” command (process 7538) reported error number 1 when it ended.): Job for httpd.service failed because the control process exited with error code. See "systemctl status httpd.service" and "journalctl -xe" for details.
…Waiting for httpd,litespeed,lscgid to shutdown ... not running.
info [restartsrv_httpd] systemd failed to start the service “httpd” (The “/usr/bin/systemctl restart httpd.service --no-ask-password” command (process 7583) reported error number 1 when it ended.): Job for httpd.service failed because the control process exited with error code. See "systemctl status httpd.service" and "journalctl -xe" for details.
…info [restartsrv_httpd] The service 'httpd' failed to restart at least three times. The system will now rebuild the httpd.conf file.
info [restartsrv_httpd] systemd failed to start the service “httpd” (The “/usr/bin/systemctl restart httpd.service --no-ask-password” command (process 7941) reported error number 1 when it ended.): Job for httpd.service failed because the control process exited with error code. See "systemctl status httpd.service" and "journalctl -xe" for details.
…failed.
Cpanel::Exception::Services::StartError
Service Status
undefined status from Cpanel::ServiceManager::Services::Httpd
Service Error
(XID a8y8dp) The “httpd” service failed to start.
Startup Log
Jan 04 01:52:00 primary2.aliasservername.com systemd[1]: Starting Apache web server managed by cPanel EasyApache...
Jan 04 01:52:02 primary2.aliasservername.com restartsrv_httpd[7944]: [Tue Jan 04 01:52:02.382417 2022] [core:emerg] [pid 8017] (28)No space left on device: AH00023: Couldn't create the mpm-accept mutex
Jan 04 01:52:02 primary2.aliasservername.com restartsrv_httpd[7944]: (28)No space left on device: could not create accept mutex
Jan 04 01:52:02 primary2.aliasservername.com restartsrv_httpd[7944]: AH00015: Unable to open logs
Jan 04 01:52:02 primary2.aliasservername.com systemd[1]: httpd.service: control process exited, code=exited status=1
Jan 04 01:52:02 primary2.aliasservername.com systemd[1]: Failed to start Apache web server managed by cPanel EasyApache.
Jan 04 01:52:02 primary2.aliasservername.com systemd[1]: Unit httpd.service entered failed state.
Jan 04 01:52:02 primary2.aliasservername.com systemd[1]: httpd.service failed.
httpd has failed. Contact your system administrator if the service does not automagically recover.
============================================