Air Traffic Control: Routing microservices with a single Nginx server

In my last post I talked about linking microservices together inside a single Docker network for easy communication between the services. For web developers, this poses an interesting problem: if we want to access the services from our local machines, we need to deal with the fact that we can’t have multiple machines bound to the same port. If you have two microservices running at the same time, both trying to bind to ports 80/443 on the local host, you’ll be unable to run both.

At Laminar Research we’ve solved this with a handy chart showing which microservice gets which port starting in the 808x`range. This helps us when we’re developing locally, but presents an interesting problem: we have to access our development environments using a port number, which makes it challenging to test real-life URLs that don’t have a port number (and rely on the defaults).

We also use mock services (like WireMock) to mock different components that we don’t want to have running but need APIs for. To do this, we stand up a service on a separate port to provide the endpoints. This makes routing difficult.

To solve this problem, I developed a service called Air Traffic Control.

Introducing Air Traffic Control

Air Traffic Control (ATC for short) does four things:

1. It binds to port 80 and 443 on the local machine.
2. It routes to the correct microservice based on the Host header, meaning we can connect multiple micorservices through a single Nginx endpoint.
3. It has the capability to hide a mock service behind the correct domain name (e.g, domain.dev.x-plane.com routes to mockserver.dev.x-plane.com.
4. It is responsible for and owns creation of the laminar_store network that we use for multiple microservices.

ATC is a fairly simple service. It’s a Docker Nginx container, with a Makefile containing commands for starting, stopping, restarting, activating and deactivating endpoints. It uses sed to change the domains inside the configuration file. And it is designed to be cross-platform.

Configuring Endpoints

To configure endpoints, we start with a default configuration file, like so:

upstream ##service##-upstream {
    server ##service##:443;
}

server {
    server_name ##service##.dev.x-plane.com;

    listen 443 ssl;

    ssl_certificate /ssl/cert.pem;
    ssl_certificate_key /ssl/key.pem;

    charset utf-8;

    location = /favicon.ico { log_not_found off; access_log off; }
    location = /robots.txt  { log_not_found off; access_log off; }

    location / {
         proxy_pass https://##service##-upstream;

         proxy_set_header   Host $host;
         proxy_set_header   X-Real-IP $remote_addr;
         proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;
         proxy_set_header   X-Forwarded-Host $server_name;
    }

    location ~ /\.ht {
        deny all;
    }
}

The `##service##` variable is what we replace with the various service names (`ident`, `tower`, etc). We use a Makefile with a command that executes a script for us that does the replace and puts the files onto the Nginx server:

enable-%: ## Enable a service (one of catalog, ident, tower)
	sed -e 's/##service##/'$*'/g' ./docker/nginx/sites-available/basic_config.conf > ./docker/nginx/sites-enabled/$*.conf
	make restart

We mount the `sites-enabled` directory to the Nginx container so that only the sites we turn on are available. We use a wildcard self-signed certificate for `*.dev.x-plane.com`.

Open Source Plans

Currently, ATC is a private repository. However, it’s simple enough that I’m planning to open source the code at some point in the future. It need not be limited to containers within the same network; you can use any upstream that you like, including a fully qualified domain name. This means you can host multiple sites on your local machine, and not worry about port binding conflicts, and the machines can be in separate networks as well.

I plan to release this sometime in mid-June.

Conclusion

The ability to reach different services on the same computer without experiencing port collisions is an important one, and one that developers have sought to fix for a long time. Hopefully, Air Traffic Control can help solve this problem for anyone who needs to solve it, and provide one of a number of solutions to this long-standing issue.