Running Maxwell (MySQL Binlog Reader) As Service In Ubuntu

As Many of DBA's are already running Maxwell (http://maxwells-daemon.io/) if not Let me introduce you to Maxwell. It reads MySQL binlogs and writes row updates as JSON to Kafka, Kinesis, or other streaming platforms. Maxwell is very nice tool to build pipelines from MySQL to other NoSQL DB's like (Elastic Search, AeroSpike, Cassandra etc).

A common approach which is being used widely is to run maxwell on a box which reads bin-logs from MySQL box and write it to Kafka (A Distributed Streaming Platform) Later consumer's consume data from kafka and write it to its own DB (Elastic Search, AeroSpike etc).

Now major issue is to run maxwell as service on Ubuntu VM. Which we tried to solve using Upstart Job for Ubuntu 14.x and Systemd service for Ubuntu 16.x.

Maxwell Binaries can be downloaded from https://github.com/zendesk/maxwell/releases.

Upstart for Ubuntu 14.x (Assuming maxwell binary placed in /opt/maxwell/ directory):

Create file: /etc/init/maxwell.conf

       
description "Start and stop the maxwell service "

author "Ankit Kumar"

env APP_HOME=/opt/maxwell/

start on (net-device-up

          and local-filesystems

          and runlevel [2345])

stop on runlevel [016]

respawn

respawn limit 5 60

limit nofile 24000 24000

pre-start script

    test -x $APP_HOME/bin/maxwell || { stop; exit 1; }

    test -e $APP_HOME/config.properties || { stop; exit 1; }

end script

script

    echo $$ > /var/run/maxwell.pid

    chdir $APP_HOME

    exec $APP_HOME/bin/maxwell

end script

post-stop script

    rm -f /var/run/maxwell.pid

end script 

Now maxwell can be started using command service maxwell start & can be stopped using command service maxwell stop.

Upstart for Ubuntu 16.x (Assuming maxwell binary placed in /opt/maxwell/ directory):

Create file: /etc/systemd/system/maxwell.service

       
[Unit]

Description=Maxwell service

After=network.target

[Service]

WorkingDirectory=/opt/maxwell/

Restart=always

RestartSec=3

ExecStart=/opt/maxwell/bin/maxwell

#ExecStartPost=/bin/bash "touch /var/run/maxwell.pid && `echo $MAINPID` > /var/run/maxwell.pid"

ExecStop=/bin/kill -9 $MAINPID

KillMode=mixed

TimeoutSec=180

[Install]

WantedBy=multi-user.target 

Enable Maxwell systemd service: systemctl enable maxwell.service

Now maxwell can be started using command systemctl start maxwell.service & can be stopped using command systemctl stop maxwell.service.

Monitoring Maxwell Service: Using /var/run/maxwell.pid monitoring on maxwell service can be enabled.