Nowadays, most of us have containerized our services, and effectively managing and upgrading containers without affecting existing services is a critical issue. Two steps are required in the CI/CD process: first, packaging the environment as a Docker image and uploading it to the company’s private Docker registry; and second, after the upload is complete, possibly connecting to the machine via SSH, pulling the new image, and restarting the running service via the Graceful Shutdown mechanism. You can learn more about Graceful Shutdown in this article. I am going to introduces Watchtower, a brand new tool that automatically upgrades and updates running containers, allowing for further CD process streamlining. Developers only need to upload the Docker image, and the remote servers can update the running container automatically.
The infrastructure diagram will become as below:
Watchtower is an application developed in Go language that monitors running Docker containers and observes whether the Docker images used when these containers were initially started have been changed. If Watchtower detects a change in the Docker image, it will automatically restart the container using the new image.
Through Watchtower, developers can easily update the running version of containerized applications by pushing new Docker images to Docker Hub or their own Docker registry. Watchtower will download your new image, gracefully shut down the existing container, and then restart it using the same options used during the initial deployment.
For example, suppose you are running Watchtower and an instance of an image called
Every few minutes, Watchtower will download the latest
ghcr.io/go-training/example53 image and compare it to the image used to run the “example53” container. If it finds that the image has been changed, it will stop/delete the “example53” container and then restart it using the new image and the same docker run options used when the container was initially started.
How to use?
Watchtower is packaged as a Docker container itself, so installation is very simple; just pull the
containrrr/watchtower image. If you’re using an ARM-based architecture, pull the appropriate
containrrr/watchtower:armhf-tag image from Docker Hub.
Since Watchtower’s code needs to interact with the Docker API to monitor running containers, the
/var/run/docker.sock needs to be mounted into the container using the -v flag when running the container.
Use the following command to run the Watchtower container:
If pulling images from a private Docker registry, use the
REPO_PASS environment variables, or mount the host’s Docker configuration file into the container (located at the root of the container filesystem /).
Also, if you have 2FA authentication set up on Docker Hub, providing only your account and password won’t be enough. Instead, you can run the docker login command to store the credentials in the
$HOME/.docker/config.json file, and then mount this configuration file to make it available to the Watchtower container:
Here we use the docker-compose method to test the running container.
After starting, you can see the following log messages:
You can adjust the
--interval 5 parameter based on the time interval you want to monitor. Here, we set it to 5 seconds. Watchtower monitors all containers on the host by default. If you don’t want certain containers to be updated, you can set the
label in docker-compose file.
After upgrading, the old containers or image files still exist on the host and take up some space. You can use the
--cleanup parameter to let Watchtower delete the old image files after using the new ones to restart the containers.
During the process, if a new image is pulled, you will see an error message below, and Watchtower will send a SIGTERM signal to the container for graceful shutdown.
If you want to change the signals, you can use the label. Please modify the
Or add them when starting the container.
Our team can focus on packaging the image and uploading it to the Docker Registry in the CI/CD process in the future. Watchtower monitors all services on the machine, and uploaded images follow the semver principles. This reduces a lot of the work involved in writing shell scripts.