All posts by loomchild

Backup & Restore Docker Named Volumes

I finally started implementing backup & restore feature for Puffin. The first issue I encountered was to make a backup of named volumes.

The official Docker documentation mentions only data volume containers and –volumes-from option. There’s also docker cp command, but it requires knowing the path where the volumes are mounted in the container that uses them.

It turns out it’s pretty easy to do using volume mounts and tar.

To backup some_volume to /tmp/some_archive.tar.bz2 simply run:

And to restore run:

I have chosen alpine image since it’s lightweight and contains everything what’s needed. One potential issue might be preserving file ownership since different users and groups exist on different containers. Classical solution to this problem is to run the tar command using the same image as the one normally using the volume instead of alpine, but what if there’s no tar there? Using numeric owner generally preserves permissions correctly, unless you also use user namespaces. Also you need to remember to stop all the containers using the volume being backed up or restored, otherwise the data on it might be damaged.

Ultimately I wrote my own little volume-backup utility for backup and restore of volumes that simplifies the process even further and offers some improvements. Example usage (see README for more details):

Feel free to check it out and let me know what do you think.

Docker Can Create Only 31 Networks per Machine

I have just learned that in Docker there is a limit of 31 networks for default network driver on a single machine:

docker-black

This is due to the fact that it uses hardcoded list of broad network ranges – 172.17-31.x.x/16 and 192.168.x.x/20 – for bridge network driver. Look into ipamutils and allocator for more details. For overlay network driver 64K networks can be created.

There seems to be no solution to circumvent this limitation apart from manually specifying subnet ranges for each created network – see Docker network create subnet option and Docker Compose network configuration reference. In Puffin, which needs to create a separate network for each application, I implemented a simple address allocator.

Puffin

puffin-scaledI am happy to announce Puffin, a project I have been working on since December.

The idea behind it is to create lightweight web application catalog based on Docker containers, offering the users smooth experience à la mobile app store.

The reason I think it’s interesting is that containers allow packing hundreds of relatively well isolated applications on a single server, which could bring the price of hosting them to almost zero. In addition easy-to-use orchestration technology lets developers easily describe complex applications in terms of microservices.

The whole thing is free / open source and I am hoping to build a small community around it – see README and CONTRIBUTING for details.

Start Only Dependencies via Docker Compose

Docker Compose is great, among other things, for demoing your web applications. It allows you to have consistent runtime environment, download dependencies without polluting the host system and automatically start external services like databases, search engines, mail servers, message queues, caches, etc. Many projects put docker-compose.yml configuration file in the source repository root, to be able to start the app by just typing:

However, during development, when you are constantly changing and debugging the code, it’s sometimes useful to keep the app running natively outside of Docker, but still running dependencies via Docker. This can be achieved in the same docker-compose.yml file by adding a special deps service (or however you want to call it) that will just start the dependencies and exit. It can be invoked as follows:

I am using this technique in my Puffin project – see my docker-compose.yml for an example.

Edit: I have changed how it’s done in Puffin, since I needed more configuration options. I linked above the original file.

Curriculum Vitae with HTML, CSS and JavaScript

Some time ago I needed to update my CV / resume because I was searching for a new job. When I looked at my current one created many years ago in Microsoft Word and later adapted to Open / Libre Office, I realized that it looks pretty outdated. Also it’s maintenance is time consuming – based on a big fat table, page breaks appear in unexpected places and there is no option to conditionally show or hide parts of the document, so I need to maintain multiple versions. While I guess solutions to these problems exist in LibreOffice, I wanted to solve them in non-WYSIWYG manner to gain greater control over the PDF generation process.

After a bit of searching, I decided to stick with familiar, ubiquitous, simple yet flexible technology – HTML, CSS with a bit of JavaScript. I found out that this combination has quite powerful typesetting capabilities – you can define manual page breaks, margins, avoid widows and orphans and do all sorts of stuff normally found in text editors. Minor rendering deficiencies, like correct hyphenation or avoiding breaking text after a header, can be fixed with a bit of JavaScript.

Resulting project is called CV, source code can be found on GitHub and here’s an example document generated by it.

I styled the output after this website, which is powered by Twenty Thirteen WordPress theme. I don’t think it’s particularly pretty, but the concept is interesting. For consistent, scriptable rendering I used wkhtmltopdf, which uses WebKit a rendering engine.