How to automatically restart your crashed Golang server

Problem:

You need to restart your Golang(or any programming languages) server application after reboot. Also, you need to monitor your server program status, whether it is RUNNING or STOPPED and restart the program in case it was killed or crashed. How to do that?

Solution:

On Windows, register your Golang program with NSSM (the Non-Sucking Service Manager). For example :

 nssm install myGolangServer c:\myGolangServer.exe

then

1. Open the Services control panel.

2. Select your service.

3. Go to the Recovery tab.
4. You can select the responses for the first, second and subsequent service failures.

---

On Unix/Linux operating systems, you can use the supervisord client/server system to monitor your server program and take action in case one of the defined events happen. Basically, Supervisor is software that allows you to manage and monitor long running or daemon processes.

supervisord can be installed on Ubuntu like OS with

$sudo apt-get install -y supervisor

and add new configuration for your server program, which in this example we will name goserver.conf and the server executable file goserver

MODIFY the configuration to suit your machine configuration

$vi /etc/supervisor/conf.d/goserver.conf

For example, edit goserver.conf file to include:

 [program:myGolangServer]
 directory=/var/www/go
 command=/var/bin/goserver
 autostart=true
 autorestart=true
 redirect_stderr=true

Save the configuration file and then:

$supervisorctl add myGolangServer

$supervisorctl start myGolangServer

or

$supervisorctl reread

$supervisorctl update

In case your supervisorctl is not up and running. Simply start it with this command:

$supervisorctl

to see the status of your new configuration - myGolangServer. If everything goes well, you should see the status RUNNING.

---

For Cent-OS like OS, install supervisord with

$sudo yum install python-setuptools

$sudo easy_install pip

$sudo pip install supervisor

after installing supervisor, we need to configure it

$echo_supervisord_conf > supervisord.conf

$sudo cp supervisord.conf /etc/supervisord.conf

$sudo mkdir /etc/supervisord.d/

$sudo vi /etc/supervisord.conf

Edit /etc/supervisord.conf to include:

 [include]
 files = /etc/supervisord.d/*.conf

Next, we need to ensure supervisord is started every time the machine is restarted.

$sudo vi /etc/rc.d/init.d/supervisord

Edit /etc/rc.d/init.d/supervisord to include:

 #!/bin/sh
 #
 # /etc/rc.d/init.d/supervisord
 #
 # Supervisor is a client/server system that
 # allows its users to monitor and control a
 # number of processes on UNIX-like operating
 # systems.
 #
 # chkconfig: - 64 36
 # description: Supervisor Server
 # processname: supervisord

 # Source init functions
 . /etc/rc.d/init.d/functions

 prog="supervisord"

 prefix="/usr/"
 exec_prefix="${prefix}"
 prog_bin="${exec_prefix}/bin/supervisord"
 PIDFILE="/var/run/$prog.pid"

 start()
 {
 echo -n $"Starting $prog: "
 daemon $prog_bin --pidfile $PIDFILE
 [ -f $PIDFILE ] && success $"$prog startup" || failure $"$prog startup"
 echo
 }

 stop()
 {
 echo -n $"Shutting down $prog: "
 [ -f $PIDFILE ] && killproc $prog || success $"$prog shutdown"
 echo
 }

 case "$1" in

  start)
 start
  ;;

  stop)
 stop
  ;;

  status)
 status $prog
  ;;

  restart)
 stop
 start
  ;;

  *)
 echo "Usage: $0 {start|stop|restart|status}"
  ;;

 esac

then let's start the supervisord

$sudo chmod +x /etc/rc.d/init.d/supervisord

$sudo chkconfig --add supervisord

$sudo chkconfig supervisord on

$sudo service supervisord start

and add new configuration for your server program, which in this example we will name goserver.conf and the server executable file goserver

MODIFY the configuration to suit your machine configuration

$vi /etc/supervisord.d/goserver.conf

For example, edit goserver.conf file to include:

 [program:myGolangServer]
 directory=/var/www/go
 command=/var/bin/goserver
 autostart=true
 autorestart=true
 redirect_stderr=true

Save the configuration file and then:

$supervisorctl add myGolangServer

$supervisorctl start myGolangServer

or

$supervisorctl reread

$supervisorctl update

In case your supervisorctl is not up and running. Simply start it with this command:

$supervisorctl

to see the status of your new configuration - myGolangServer. If everything goes well, you should see the status RUNNING.

Hope this helps!

References:

https://www.socketloop.com/tutorials/golang-how-to-run-golang-application-such-as-web-server-in-the-background-or-as-daemon

http://supervisord.org/introduction.html#supervisor-components

https://medium.com/dev-bits/building-a-go-web-application-deployment-tool-chain-with-nginx-gulp-and-supervisor-bef6a5d454b9