Watchtower – Simple solution for automatic updates to all your Docker containers
Docker is great – keeping all your containers up to date not so much. Fortunately there is an easy fix to that: Watchtower. Docker and docker-compose allow you to easily run a bunch of services on a computer as cheap as the Raspberry Pi. Unfortunately with a lot of power comes a lot of responsibility. You need to keep everything secure and up to date. Watchtower is another container which allows you to automate this process to any degree you want.
Running a server with Docker
I personally chose Docker and docker-compose to run all the software on my smart home server on a Raspberry Pi as well as other software like Bitwarden on a more powerful server. It comes with many advantages: easy installation and configuration, (mostly) independence of the actual platform it is running on and it makes sharing your setup and backup much easier.
Unfortunately the level of abstraction and the easy of use means that I am running a lot of services now. Keeping them all up to date is not that hard but a very annoying task. Fortunately Watchtower promises to fully automate this process for you.
How to update a Docker container
Lets step back for a second and look at how Docker containers are updated in general. In contrast to “normal” software you don’t update some single files. Instead you just download a newer version of the full Docker image (there are features like layers that mean you don’t actually have to download the full amount of data each time).
Then you stop your container, replace it with the new image and start it again. The whole software is replaced by a newer version, actual (user) data should be persisted in separate volumes or bindings.
If you are running docker-compose the way to update looks something like this:
docker-compose down // stop containers
docker-compose pull // pull/download newest images
docker-compose up -d // start containers based on the new imagesHow does Docker know which new image to download though? It asks a container registry, a directory of a set of available containers and there versions. Usually it is the public Docker-hub but you can host your own (for private docker images for example.
Your Docker daemon still needs to find the right version of an image though. This is done via tags. Each image has a set of possible tags which you can indicate after the image name like this: TARGET_IMAGE[:TAG]. If I am looking to use the MQTT broker mosquitto in version 1.6 I can indicate it like this: eclipse-mosquitto:1.6.
If you leave out the version number Docker implicitly uses the "latest" tag which means it just pulls the newest available version. It is also important to know that every part of the version you leave empty will be replaced by the newest one. So if you define version 1 the container version 1.6.7 or later 1.8.8 might be downloaded. If you define version 1.6 though then 1.6.9 might be the newest version for you, 1.8.8 will not be installed.
Knowing this it is important to select the right version tags for your containers before automating the update process. For any containers that might have breaking changes (like HomeAssistant or core infrastructure like the Traefik reverse proxy) I prefer to fix them to a major version and only allow minor updates. Other software like PiHole for example can always be updated to the newest version, the latest tag might be fine here.
Automating container updates with Watchtower
Once you have prepared your Docker containers and/or docker-compose files with proper version tags we can install Watchtower and configure it to automatically update all containers for us.
Installing Watchtower
Watchtower itself can be installed as a Docker container and defined in docker-compose. My personal setup looks like this:
version: "3"
services:
// ... other services
watchtower:
image: containrrr/watchtower
volumes:
- /var/run/docker.sock:/var/run/docker.sock
environment:
- TZ=${TZ}
- WATCHTOWER_CLEANUP=true
- WATCHTOWER_LABEL_ENABLE=true
- WATCHTOWER_NOTIFICATIONS=email
- WATCHTOWER_NOTIFICATION_EMAIL_FROM=${EMAIL_FROM}
- WATCHTOWER_NOTIFICATION_EMAIL_TO=${WATCHTOWER_EMAIL_TO}
- WATCHTOWER_NOTIFICATION_EMAIL_SERVER=${SMTP_SERVER}
- WATCHTOWER_NOTIFICATION_EMAIL_SERVER_PORT=${SMTP_PORT}
- WATCHTOWER_NOTIFICATION_EMAIL_SERVER_USER=${SMTP_USER}
- WATCHTOWER_NOTIFICATION_EMAIL_SERVER_PASSWORD=${SMTP_PASSWORD}Watchtower does not need to persist any data here but it has to have access to the Docker socket to be able to detect other running containers and update them. Of course this is a possible security risk but personally I am fine with running the official images.
All values in ${} are provided as environment variables like explained in my smart home server setup here. The timezone is provided to the container for proper times in the logs and scheduling.
Now you can just start the container and it will regularly update all outdated containers.
Configuring Watchtower
If you just want Watchtower to update all your containers there is not much you need to do. You can just skip everything after the timezone setting and run it like that. I did add a couple of custom settings though.
Notifications
Because Watchtower does all the container updates automatically I was looking for a way to still find out when changes occur. Fortunately Watchtower supports notifications. Slack and Teams are supported but I personally use Pushover for my smart home related notifications. Watchtower does not support Pushover directly (actually there is a better solution described here) or even custom scripts (bummer) for notifications so I used Email instead.
Pushover provides a generated email address that you can send messages to. These will then be send to your devices as Pushover messages. The settings above will send all Watchtower messages to Pushover which will push them to my smartphone. All you need is your own SMTP server to actually send those emails.
Excluding containers
Sometimes you might have some containers that you don’t want to be updated automatically. Even though you control the behavior to a certain degree with the right version label some systems should just not automatically change. In this case I do not want HomeAssistant to randomly restart as it controls my smart home.
Watchtower supports two ways of implementing this. You can exclude all containers and specifically mark some for updates via the LABEL com.centurylinklabs.watchtower.enable: true label and by setting the environment variable WATCHTOWER_LABEL_ENABLE.
By default Watchtower updates all containers instead and you can label those you do not want to have updated with “com.centurylinklabs.watchtower.enable: false". More details on this can be found in the documentation.
Cleanup
Watchtower will always pull the newest applicable images for your containers and then start new containers based on those updated images. This means that over time you will collect a huge amount of unused images. Of course you can clean those manually but the goal here is to automate the work. Fortunately we can add the WATCHTOWER_CLEANUP=true variable to watchtower to make it clean up after itself by deleting old unused images.
This setup has been running on my smart home server for some time now and is really a great time saver. You can find the full set of files for running a similar system in my GitHub repository and the full documentation in this article.
