How to Monitor HAProxy with ELK Stack



Analyze HAProxy logs with the Logstail ELK stack

Introduction

HAProxy is an open source reverse proxy load balancer for community/enterprise users and has become the standard in the load balancing and high-availability management industry due to its availability in almost every Linux distribution and of course, its effectiveness. Some of its main features:

  • HAProxy utilizes frontend (client side) and backend (server side) proxies.
  • Note that HAProxy can be fully functional even with one or more failed backend servers by regularly checking their health and rerouting the traffic if one server is not responding.
  • Clients use sessions in order to interact with HAProxy.Once the client connects to the frontend and the frontend connects to the backend, the client can begin sending requests.

For the needs of that guide we will use the Logstail hosted ELK stack.

HAProxy Installation

  • Debian/Ubuntu

  • CentOS

HAProxy Configuration

First of all, we should keep a backup of the default configuration file before making any changes.

In the next step, we need to add to /etc/haproxy/haproxy.cfg the below configuration along with the rest reverse proxy configuration.

Now, we need to configure rsyslog to listen for HAProxy logs at 127.0.0.1:514 over UDP and forward them to apps.logstail.com (or your logstash machine ip).

To accomplish that, we need to create /etc/rsyslog.d/21-haproxy.conf with the below configuration.

  • Don’t forget to replace USER_TOKEN with your logstail private user token.

In the next step we will develop some custom grok patterns that will be used by Logstash to parse the received HAProxy logs.

We can save them in a separate file (in our example /usr/share/logstash/patterns/log-patterns) in order to maintain a simple and clear logstash pipeline.

Custom grok patterns

 

Pipeline

Kibana Visualization

We are now ready and our properly parsed HAProxy logs are already going to our Logstail  ELK stack, so we can create new Logstail Kibana visualizations and dashboards for our logs and instantly get a clear view of its health, security and status.

Example Dashboard

 

For even more HAProxy Visualisations and Dashboards you can start your free trial.

Installing ELK Stack (latest) on OSX (Sierra)<< >>Analyze Apache logs with ELK stack

Leave a Reply

Your email address will not be published.