Traefik Configuration for Docker Compose Setups

• Sun, Jan 7, 2024
• 4-minute read

Recently I played around with letting multiple smaller applications run, containerized, and published via Traefik as the central Gateway and SSL handler.

Initial setup was simple - but adding more applications and separating the applications and their backend services into separate docker networks caused problems.

The problem

I ran into an error that was initially hard to get a grip on, because it was

• sometimes there, sometimes not
• a restart via docker-compose up changed the situation all the time (even without changes to the docker-compose.yml config file)
• the log-output I saw was inexistent or not helpful
• the error that was output when accessing the application through Traefik was just a “504 Gateway Timeout” after reaching the timeout
• the Traefik Dashboard showed all services, routers, everything - and everything as “healthy” without problems

While trying several things to solve this issue, I two times thought I’d have solved it - then restarted the containers just to find out I was wrong and the problem was back.

It looked so unlogical that I was even close to swapping Traefik with something else. But I kind of wanted Traefik to stay…

I pretty soon found hints that my problems could be related to how Traefik routes traffic to the containers and which docker network is chosen for this - worst case is what happens by default: Traefik chooses the network randomly - ending up trying to talk to a container via it’s IP address of a different network - a way that intentionally does not work.

Researching the Traefik documentation

I checked the Traefik documentation on this issue, but e.g. their Docker Compose example is too simple and does not contain multiple networks.

Getting closer to it after reading Traefik Docker Documentation, especially the Section about networking where it states:

Defines a default docker network to use for connections to all containers.

This option can be overridden on a per-container basis with the traefik.docker.network label.

Well, I tried that, but it didn’t work with having a service in docker-compose.yml like this (massively shortened):

  matomo:
    image: matomo
    container_name: "matomo"
    networks:
      - published-apps
      - backend-matomo
    (...)
    labels:
      - "traefik.enable=true"
      - "traefik.docker.network=published-apps"
      (...)

OK, to be honest it worked, then it didn’t work, then it worked again… Each docker compose up was a gamble, which of the apps was reachable and which not…

After an extensive consumation of google results, I finally found the relevant hint I was missing.

The network name in the label must not be 1:1 as the defined network in the docker-compose.yml file, but exactly as seen by Docker. Check docker network ls and see the “real” name of the networks.

OK, that sounds a bit stupid to me, having one thing referenced by two different names in the same config file.

Tested that nevertheless and it works, and survives a bunch of restarts, consistently now.

Solution found

In my setup, the docker-compose stack has no specific name for the whole stack, so the prefix is just “docker_”. Resulting in a network named docker_published-apps that needs to referenced in the service’s labels exactly like this.

But add this prefix only within the labels that configure Traefik for this service, not the other parts of the config file that tell Docker what to do (e.g. the definition of the networks on top of the file, or the config which app container is bound to what network(s).

So the resulting, now working config (again, massively shortened to illustrate it better) is:

version: '3'

networks:
  published-apps:
  backend-matomo:

services:
  reverse-proxy:
    image: traefik:v2.10
    (...)
    command:
      (...)
      - "--providers.docker"
      - "--providers.docker.exposedbydefault=false"

    labels:
      - "traefik.enable=true"
      - "traefik.docker.network=docker_published-apps"
      - "traefik.http.routers.dashboard.rule=Host(`traefik.domain.tld`)"
      (...)
    networks:
      - published-apps
    ports:
      - "443:443"
    (...)

  matomo-db:
    image: mariadb:10.11
    command: --max-allowed-packet=64MB
    container_name: "matomo-db"
    restart: always
    networks:
      - backend-matomo
    (...)

  matomo:
    image: matomo
    restart: always
    container_name: "matomo"
    networks:
      - published-apps
      - backend-matomo
    links:
      - matomo-db
    (...)
    
    labels:
      - "traefik.enable=true"
      - "traefik.docker.network=docker_published-apps"
      - "traefik.http.routers.matomo.rule=Host(`analytics.domain.tld`)"
      (...)

You can see that the matomo and traefik containers are bound to the published-apps network, referenced as docker_published-apps in the label configuration and thus make sure Traefik can always reach those containers.

The matomo-db container in turn is not exposed to the outside, neither via Docker, nor via Traefik and is only reachable from the single other host that shares the same network (which is the matomo container that consumes this database)