development and systems administration

September 20, 2012 6:54 pm

3 ways to configure HAProxy for WebSockets

Currently there aren’t many options when it comes to proxying WebSockets. Nginx doesn’t yet fully support WebSockets out of the box, though some people have opted to take an older version and patch it. I wont go into why I eventually decided to go with HAProxy, but I will link you to an article which does a nice job of summarizing the current state of Proxies and WebSockets.

Thankfully, there’s HAProxy, however for some reason it doesn’t seem to be so well known at the moment. There aren’t many overviews detailing it’s configuration, so I thought it would be useful to list a few common use cases and describe their setup.

Some potential ways to proxy to a WebSocket backend:

  • proxy based on sub-domain

  • proxy based on a URI

  • proxy using automatic detection

First, let’s get the top portion of our haproxy.cfg file out of the way. This is generally what I use for most configurations:

# this config needs haproxy-1.1.28 or haproxy-1.2.1
global
  log  127.0.0.1  local0
  log  127.0.0.1  local1 notice
  maxconn  4096
  chroot   /usr/share/haproxy
  uid  99
  gid  99
  daemon

defaults
  log   global
  mode  http
  option  httplog
  option  dontlognull
  retries  3
  option  redispatch
  option  http-server-close
  maxconn  2000
  contimeout  5000
  clitimeout  50000
  srvtimeout  50000

One very important thing to point out in this config is the ‘option http-server-close’ line. Without this, some of the examples below can behave incorrectly. The option tells HAProxy to ignore the servers ‘keepalive’ setting. If it were not specified, then in some cases the conditional rules (used below) would not be re-evaluated every time there is a new request. Instead HAProxy would use the previously established connection for the new request(s) and so therefore would fail to notice that the new request might be a socket request. In short, If you are using WebSockets in a mixed environment, always make sure ‘option http-server-close’ is set.

note: I previously used ‘option httpclose’ which disables keepalive on both the client and server, whereas http-server-close just disables keepalive on the server. Thanks to Willy Tarreau for pointing out the difference in the comments below. See the docs on http-server-close for more information

Now, let’s move on to the various common types of configuration. For the sake of simplicity, we’re going to assume that your socket server is running on port 8000 on localhost. However, this could easily be a completely different server or port.

note: although HAProxy is great for load-balancing, in these examples I’m not covering that kind of setup and so I have no health checks running on the servers. Perhaps I will cover load balancing with HAProxy in a future blog post.

1. Proxy based on sub-domain

Some people like to have their WebSocket server running on an entirely separate sub-domain (ie. websockets.example.com). This makes it easy to keep the services on completely separate machines, and also allows you to run your sockets on port 80 directly instead of using a second-proxy on the sub-domain.

frontend public
  bind *:80
  acl is_websocket hdr_end(host) -i ws.example.com
  use_backend ws if is_websocket
  default_backend www

backend www
  timeout server 30s
  server www1 127.0.0.1:8080

backend ws
  timeout server 600s
  server ws1 127.0.0.1:8000

This will direct all traffic going to ws.example.com to your websocket server (In this example it’s localhost, but you could easily plug in the IP to a different server).

note: if you are using 1.5-dev or higher, you can use the ‘timeout tunnel’ option which sets up larger timeout for WS connections than for normal HTTP connections automatically. This means you don’t have to set overly long timeouts on the client side. I haven’t used this option yet, which is why I don’t include it in the examples.

note: since the two ‘backend’ config blocks will not change for any of the examples, we’ll leave them out in the following two configurations.

2. Proxy based on URI

An alternative to the above setup is to proxy based on the URI (ie. example.com/websockets). This allows you to keep everything operating within the same virtual(or non-virtual) domain.

frontend public
  bind *:80
  acl is_example hdr_end(host) -i example.com
  acl is_websocket path_beg -i /websockets
  use_backend ws if is_websocket is_example
  default_backend www

This is how I usually set up my servers right now, because the application I’m working on is, for the most part, a standard PHP application, however I use a python socket server (Tornado) for handling certain tasks. We want to keep everything operating on the same domain, therefore when a socket connection is made with a specific URI, we proxy that straight to Tornado, whereas otherwise we send the request to our Nginx + Apache backend (see my previous blog post, 5 reasons to add Nginx to your LAMP stack right now).

3. Proxy using WebSocket detection

The final example uses an automatic detection of the websocket request by examining the HTTP header for the Upgrade: WebSocket line. My personal preference is to use this along with a secondary test to make sure we really want to pass the request along to the socket server.

frontend public
  bind *:80
  acl is_websocket hdr(Upgrade) -i WebSocket
  acl is_websocket_server hdr_end(host) -i ws.example.com
  use_backend ws if is_websocket is_websocket_server
  default_backend www

This config will ensure that the request not only has the Upgrade: WebSocket header, but also that it’s accessing the correct location for the websocket server (ws.example.com).

These are 3 common ways to configure HAProxy for WebSockets. Perhaps the setup you’ve found works best for you is different? Please share your comments, corrections or alternate configurations in the comments!

This theme was originally based on the Office theme by Alex Penny, with modifications by Nick Jennings