Let's say you want to distribute a Python tool with docker using known good dependency versions ready to be used by end users... In this article you will see how to continuously keeping up to date a Docker Hub container with minimal managing effort (because I'm a lazy guy) using github, TravisCI and pyup.
The goal was to reduce as much as possible any manual activity for updates, check all works fine before pushing, minimize build times and keep docker container always secure and updated with a final high quality confidence.
As an example let's see what happens under the hood behind every pytest-play Docker Hub update on the official container https://cloud.docker.com/u/davidemoro/repository/docker/davidemoro/pytest-play (by the way if you are a pytest-play user: did you know that you can use Docker for running pytest-play and that there is a docker container ready to be used on Docker Hub? See a complete and working example here https://davidemoro.blogspot.com/2019/02/api-rest-testing-pytest-play-yaml-chuck-norris.html)
RepositoriesThe docker build/publish stuff lives on another repository, so https://github.com/davidemoro/pytest-play-docker is the repository that implements the Docker releasing workflow for https://github.com/pytest-dev/pytest-play on Docker Hub (https://hub.docker.com/r/davidemoro/pytest-play).
WorkflowThis is the highly automated workflow at this time of writing for the pytest-play publishing on Docker Hub:
- a new pytest-play version or any other pinned sub-dependency is published on PyPI.
By the way pytest-play is automatically released on PyPI thanks to travis when a new tag comes on the master branch.
If you are curious see https://github.com/pytest-dev/pytest-play/blob/master/.travis.yml. Instead for tags, versions and changelog management I use the always evergreen https://pypi.org/project/zest.releaser described in High quality automated package releases for Python with zest.releaser
- pyup service opens a pull request on github for pytest-play-docker to be merged on master (if you want to reduce the pull requests rate configure pyup properly with a daily or weekly policy, see pyup docs)
- travis runs the build for the above pull request against the pull request branch and pull request status updated (pass or failing)
- receive an email notification about a new pull requests
- if the pull request status is pass merge pull request on master on https://github.com/davidemoro/pytest-play-docker/pulls otherwise investigate
- travis runs tests again on master merge, if the current branch is master and tests passed:
- docker push to Docker Hub on https://hub.docker.com/r/davidemoro/pytest-play
So now every time you run a command like the following one (see a complete and working example here https://davidemoro.blogspot.com/2019/02/api-rest-testing-pytest-play-yaml-chuck-norris.html):
docker run --rm -v $(pwd):/src davidemoro/pytest-playyou know what was the workflow for every automated docker push for pytest-play.
AcknowledgementsMany thanks to Andrea Ratto for the 10 minutes travis build speedup due to Docker cache, from ~11 minutes to ~1 minute is a huge improvement indeed! It was possible thanks to the docker pull davidemoro/pytest-play command, the build with the --cache-from davidemoro/pytest-play option and running the longest steps in a separate and cacheable step (e.g., the very very long cassandra-driver compilation moved to requirements_cassandra.txt will be executed only if necessary).
Relevant technical details about pytest-play-docker follows (some minor optimizations are still possible saving in terms of final size).
FeedbackAny feedback will be always appreciated.
Do you like the Docker hub push process for pytest-play? Let me know becoming a pytest-play stargazer! Star