How to Auto-Restart Failed Linux Services Without Systemd

Linux
1

Linux system administrators often need to ensure critical services remain running, even if they crash unexpectedly. While systemd makes this straightforward on modern distributions, older systems using SysVinit or Upstart require different approaches. Let’s explore how to configure automatic service restarts on these non-systemd init systems.

Using Monit with SysVinit

Monit is a versatile monitoring tool that can restart failed services on SysVinit-based systems.

Step 1: Install Monit

# Debian/Ubuntu
sudo apt update && sudo apt install monit

# CentOS/RHEL
sudo yum install monit

Step 2: Configure service monitoring

Edit the Monit configuration file:

sudo nano /etc/monit/monitrc

Add a service definition, for example to monitor Apache:

check process apache2 with pidfile /var/run/apache2/apache2.pid
    start program = "/etc/init.d/apache2 start"
    stop program = "/etc/init.d/apache2 stop"
    if failed host 127.0.0.1 port 80 protocol http then restart
    if 5 restarts within 5 cycles then timeout

This configuration tells Monit to:

  • Check the Apache process using its PID file
  • Use the init script to start/stop the service
  • Restart Apache if it can’t reach localhost on port 80
  • Stop attempting restarts after 5 failures in 5 check cycles

Step 3: Enable and start Monit

sudo service monit start
sudo update-rc.d monit defaults

Monit will now monitor Apache and restart it if it fails.

Upstart Service Recovery

Upstart, used in older Ubuntu versions, has built-in service recovery options.

Step 1: Create an Upstart job configuration

Create a file for your service in /etc/init/, for example:

sudo nano /etc/init/myapp.conf

Step 2: Configure automatic restarts

Add the following to your Upstart job:

start on runlevel [2345]
stop on runlevel [!2345]

respawn
respawn limit 10 5

exec /path/to/your/application

This configuration:

  • Starts the service on normal runlevels
  • Enables automatic restarts with respawn
  • Limits restarts to 10 attempts within 5 seconds

Step 3: Start and enable the service

sudo start myapp

Upstart will now automatically restart your service if it crashes.

Cron-based Service Monitoring

For systems without Monit or Upstart, a simple cron job can periodically check and restart services.

Step 1: Create a monitoring script

sudo nano /usr/local/bin/check_service.sh

Add the following content:

#!/bin/bash
SERVICE_NAME="apache2"
if ! pgrep -x "$SERVICE_NAME" > /dev/null
then
    /etc/init.d/$SERVICE_NAME start
    echo "Restarted $SERVICE_NAME at $(date)" >> /var/log/service_restarts.log
fi

Step 2: Make the script executable

sudo chmod +x /usr/local/bin/check_service.sh

Step 3: Add a cron job

Edit the root crontab:

sudo crontab -e

Add a line to run the script every 5 minutes:

*/5 * * * * /usr/local/bin/check_service.sh

This method is less immediate than Monit or Upstart, but provides a simple fallback option for service monitoring.

While these methods work well for older systems, consider upgrading to a systemd-based distribution for more robust and integrated service management capabilities. Modern init systems offer improved logging, dependency management, and parallel startup processes that can significantly improve system reliability and performance.