complex docker-compose templates using bash (or anything you can execute)

A cool trick I have not seen applied much (maybe it’s a bad idea, and that’s why) is using bash as a templating language for complex compose yml files.

docker-compose is a nifty entry level tool to work with Docker, but it has its own caveats and limitations and is not really under active feature development. By using templates and exploiting file descriptors on the -f flag it’s possible to supercharge and extend docker-compose functionality.

To date, compose files only allow for variable substitution, and only for values. That works for simple straightforward setups, but a more complex environment might need its own runtime logic into it. This becomes useful particularly on development scenarios.

#!/usr/bin/env bash
# some.compose.yml.sh

answer() {
  echo 42
}

cat << EOF
services:
  app:
    image: scratch
    environment:
      ALL_THINGS_ANSWERED: $(answer)
EOF

$ ./some.compose.yml.sh
services:
  app:
    image: scratch
    environment:
      ALL_THINGS_ANSWERED: 42
$ docker compose -f <(./some.compose.yml.sh) -p foo config
services:
  app:
    environment:
      ALL_THINGS_ANSWERED: "42"
    image: scratch
    networks:
      default: null
networks:
  default:
    name: foo_default

This example used bash, but note that any executable producing a valid compose yml output will work!

#!/usr/bin/env python3
# some.py
def answer():
    return 42
print(f"""
services:
  app:
    image: scratch
    environment:
      ALL_THINGS_ANSWERED: {answer()}
""")

$ ./some.py
services:
  app:
    image: scratch
    environment:
      ALL_THINGS_ANSWERED: 42
$ docker compose -f <(./some.py) -p foo config
services:
  app:
    environment:
      ALL_THINGS_ANSWERED: "42"
    image: scratch
    networks:
      default: null
networks:
  default:
    name: foo_default

Hope this silly example makes sense. For a real world case of this pattern being applied, check out https://github.com/Kong/gojira/.

Kudos to Rai and Niji for their part on figuring out this pattern.

See also: A docker compose command that brings up a service without a command