Tuesday, 18 April 2017

Introduction to Docker Compose Tool for Multi-Container Applications

Docker Compose is a “tool for defining and running your multi-container Docker applications”. Your applications can be defined in a YAML file where all the options that you used in `docker run` are now defined. Compose also allows you to manage your application as a single entity rather than dealing with individual containers.
In this tutorial we give you a brief introduction to Docker Compose, by building, you may have guessed...a Blog site.

Installing Docker Compose 

Just like the Docker engine, Compose is extremely easy to install. First verify that you have the Docker engine installed, since Compose will use it. Then if you are comfortable with it you can simply use `curl` to download the Compose binary. If you struggle with the following commands or need additional details, check the very good documentation
$ docker version
$ curl -L https://github.com/docker/compose/releases/download/1.6.2/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose
$ docker-compose version

Running a Ghost blog

While you can read the entire documentation and go through the compose reference manual. Nothing beats trying this out to discover a new tool. To dive straight into using Compose we are going to run a Ghost blog using containers.
You can run Ghost in a standalone mode which uses an embedded SQlite database in a single container. It is simple, and you do not need Compose for this, but it breaks the principles of single service functionality per container and will not allow you to scale any components of your blog if you need to. Let’s see how to do it anyway:
$ docker pull ghost
$ docker run -d --name ghost -p 80:2368 ghost
Once the above commands are successful, you should be able to access Ghost with your browser on port 80 of the Docker host you are using. Using a small trick, we will use this single container deployment to get the Ghost configuration file and modify it for a multi-container setup. Copy the Ghost configuration file located in the container to your local file system using the `docker cp` command like so:
$ docker cp -L ghost:/usr/src/ghost/config.js ./config.js
$ cat config.js
Edit the development section of the config.js file, to point to a Mysql database. We will assume that you can reach a Mysql database with a DNS name of `mysql`. We will setup a ghost database, with a Ghost user and a password set to `password`. You could also use a config file that takes advantage of environment variable. For simplicity, in this blog, we override the Ghost config file like so:
[config.js]
database: {
           client: 'mysql',
           connection: {
               host     : 'mysql',
               user     : 'ghost',
               password : 'password',
               database : 'ghost',
               charset  : 'utf8'
           }
       },
For that new configuration to be used, you need to create a Dockerfile that will be used to build your own local image of Ghost using your custom config file. You could do this several different ways, but building your own image with a two line Dockerfile is as easy as it gets. Here is the Dockerfile: 
FROM ghost
COPY ./config.js /var/lib/ghost/config.js
This new Docker image will be built automatically in your Docker Compose file using the `build` argument.
Your Compose file takes the following form. Two services are defined, a Mysql service and a Ghost service. The Mysql service is configured via environment variables set in the docker-compose file. We use the official Mysql Docker image that Compose will automatically pull from the Docker hub. Port 3306 is exposed to other containers in the same network. The Ghost service is based on our custom image; it depends on the Mysql service to ensure that the database will start first. We expose the default port of Ghost `2368` to port 80 of our Docker host.
[yaml]
version: '2'
services:
 mysql:  
  image: mysql
  container_name: mysql
  ports:
   - "3306"
  environment:
   - MYSQL_ROOT_PASSWORD=root
   - MYSQL_DATABASE=ghost
   - MYSQL_USER=ghost
   - MYSQL_PASSWORD=password
 ghost:  
  build: ./ghost
  container_name: ghost
  depends_on:
    - mysql
  ports:
    - "80:2368"
This would be the equivalent of running the following `docker run` commands: 
$ docker run -d --name mysql -e MYSQL_ROOT_PASSWORD=root -e MYSQL_DATABASE=ghost -e MYSQL_PASSWORD=password -e MYSQL_USER=ghost -p 3306 mysql
$ docker build -t myghost .
$ docker run -d --name ghost -p 80:2368 myghost
Keeping all these steps in a single YAML configuration file will be easier to maintain and evolve than writing your own Docker commands wrapper in bash scripts. Plus compose allows you to manage the entire app and individual services. 
To start your Compose application, you just need to run `docker-compose up -d`. The two containers will get started and will be properly connected to each other on the network. You can then open your browser at `http://localhost>` and start using Ghost. To create new posts go to `http://localhost/ghost/setup>` create an account and start editing your posts. Once the containers have started you can view the state of your application as simply as with `docker-compose ps`. 
[bash]
$ docker-compose up -d
Starting mysql
Starting ghost
$ docker-compose ps
Name            Command            State            Ports          
------------------------------------------------------------------
ghost   /entrypoint.sh npm start   Up      0.0.0.0:80->2368/tcp    
mysql   /entrypoint.sh mysqld      Up      0.0.0.0:32770->3306/tcp
Note that if you have used Compose before, in this example we use version ‘2’ of Compose. Hence we do not need links. The two services will take advantage of the embedded DNS server now running on Docker engine 1.10 and will be able to find each other using their service name. Hence if you want to ping ‘ghost’ from the mysql container you can and vice versa: 
[bash]
$ docker exec -ti mysql bash
root@b1e66140ddb3:/# ping ghost
PING ghost (172.18.0.3): 56 data bytes
64 bytes from 172.18.0.3: icmp_seq=0 ttl=64 time=0.074 ms
64 bytes from 172.18.0.3: icmp_seq=1 ttl=64 time=0.222 ms
And voila! Docker Compose is a very handy tool that helps you write a distributed application definition in a single YAML file. It can handle most of the `docker run` options and since the last release also supports Docker networks and volumes. In following posts, we will dive into more advanced setup and use cases using Compose as well as the use of Docker Swarm to distribute your containers across a cluster of Docker hosts.

Quickstart: Compose and WordPress

You can use Docker Compose to easily run WordPress in an isolated environment built with Docker containers. This quick-start guide demonstrates how to use Compose to set up and run WordPress. Before starting, you’ll need to have Compose installed.

Define the project

  1. Create an empty project directory.
    You can name the directory something easy for you to remember. This directory is the context for your application image. The directory should only contain resources to build that image.
    This project directory will contain a docker-compose.yml file which will be complete in itself for a good starter wordpress project.
    Tip: You can use either a .yml or .yaml extension for this file. They both work.
  2. Change directories into your project directory.
    For example, if you named your directory my_wordpress:
     cd my_wordpress/
    
  3. Create a docker-compose.yml file that will start your WordPress blog and a separate MySQL instance with a volume mount for data persistence:
    version: '2'
    
    services:
       db:
         image: mysql:5.7
         volumes:
           - db_data:/var/lib/mysql
         restart: always
         environment:
           MYSQL_ROOT_PASSWORD: wordpress
           MYSQL_DATABASE: wordpress
           MYSQL_USER: wordpress
           MYSQL_PASSWORD: wordpress
    
       wordpress:
         depends_on:
           - db
         image: wordpress:latest
         ports:
           - "8000:80"
         restart: always
         environment:
           WORDPRESS_DB_HOST: db:3306
           WORDPRESS_DB_PASSWORD: wordpress
    volumes:
        db_data:
    
    NOTE: The docker volume db_data will persist any updates made by wordpress to the database. Learn more about docker volumes

Build the project

Now, run docker-compose up -d from your project directory.
This pulls the needed images, and starts the wordpress and database containers, as shown in the example below.
$ docker-compose up -d
Creating network "my_wordpress_default" with the default driver
Pulling db (mysql:5.7)...
5.7: Pulling from library/mysql
efd26ecc9548: Pull complete
a3ed95caeb02: Pull complete
...
Digest: sha256:34a0aca88e85f2efa5edff1cea77cf5d3147ad93545dbec99cfe705b03c520de
Status: Downloaded newer image for mysql:5.7
Pulling wordpress (wordpress:latest)...
latest: Pulling from library/wordpress
efd26ecc9548: Already exists
a3ed95caeb02: Pull complete
589a9d9a7c64: Pull complete
...
Digest: sha256:ed28506ae44d5def89075fd5c01456610cd6c64006addfe5210b8c675881aff6
Status: Downloaded newer image for wordpress:latest
Creating my_wordpress_db_1
Creating my_wordpress_wordpress_1

Bring up WordPress in a web browser

If you’re using Docker Machine, then docker-machine ip MACHINE_VM gives you the machine address and you can open http://MACHINE_VM_IP:8000 in a browser.
At this point, WordPress should be running on port 8000 of your Docker Host, and you can complete the “famous five-minute installation” as a WordPress administrator.
NOTE: The WordPress site will not be immediately available on port 8000 because the containers are still being initialized and may take a couple of minutes before the first load.
Choose language for WordPress install
WordPress Welcome

Shutdown/Clean up

docker-compose down will remove the containers and default network, but preserve your wordpress database. docker-compose down --volumes will remove the containers, default network, and the wordpress database.

Use the Docker command line

docker

To list available commands, either run docker with no parameters or execute docker help:
$ docker
Usage: docker [OPTIONS] COMMAND [ARG...]
       docker [ --help | -v | --version ]

A self-sufficient runtime for containers.

Options:
      --config string      Location of client config files (default "/root/.docker")
  -D, --debug              Enable debug mode
      --help               Print usage
  -H, --host value         Daemon socket(s) to connect to (default [])
  -l, --log-level string   Set the logging level ("debug", "info", "warn", "error", "fatal") (default "info")
      --tls                Use TLS; implied by --tlsverify
      --tlscacert string   Trust certs signed only by this CA (default "/root/.docker/ca.pem")
      --tlscert string     Path to TLS certificate file (default "/root/.docker/cert.pem")
      --tlskey string      Path to TLS key file (default "/root/.docker/key.pem")
      --tlsverify          Use TLS and verify the remote
  -v, --version            Print version information and quit

Commands:
    attach    Attach to a running container
    # […]

Description

Depending on your Docker system configuration, you may be required to preface each docker command with sudo. To avoid having to use sudo with the docker command, your system administrator can create a Unix group called docker and add users to it.
For more information about installing Docker or sudo configuration, refer to the installation instructions for your operating system.

Environment variables

For easy reference, the following list of environment variables are supported by the docker command line:
  • DOCKER_API_VERSION The API version to use (e.g. 1.19)
  • DOCKER_CONFIG The location of your client configuration files.
  • DOCKER_CERT_PATH The location of your authentication keys.
  • DOCKER_DRIVER The graph driver to use.
  • DOCKER_HOST Daemon socket to connect to.
  • DOCKER_NOWARN_KERNEL_VERSION Prevent warnings that your Linux kernel is unsuitable for Docker.
  • DOCKER_RAMDISK If set this will disable ‘pivot_root’.
  • DOCKER_TLS_VERIFY When set Docker uses TLS and verifies the remote.
  • DOCKER_CONTENT_TRUST When set Docker uses notary to sign and verify images. Equates to --disable-content-trust=false for build, create, pull, push, run.
  • DOCKER_CONTENT_TRUST_SERVER The URL of the Notary server to use. This defaults to the same URL as the registry.
  • DOCKER_HIDE_LEGACY_COMMANDS When set, Docker hides “legacy” top-level commands (such as docker rm, and docker pull) in docker help output, and only Management commands per object-type (e.g., docker container) are printed. This may become the default in a future release, at which point this environment-variable is removed.
  • DOCKER_TMPDIR Location for temporary Docker files.
Because Docker is developed using Go, you can also use any environment variables used by the Go runtime. In particular, you may find these useful:
  • HTTP_PROXY
  • HTTPS_PROXY
  • NO_PROXY
These Go environment variables are case-insensitive. See the Go specification for details on these variables.

Configuration files

By default, the Docker command line stores its configuration files in a directory called .docker within your $HOME directory. However, you can specify a different location via the DOCKER_CONFIG environment variable or the --config command line option. If both are specified, then the --config option overrides the DOCKER_CONFIG environment variable. For example:
docker --config ~/testconfigs/ ps
Instructs Docker to use the configuration files in your ~/testconfigs/ directory when running the commandps.
Docker manages most of the files in the configuration directory and you should not modify them. However, you can modify theconfig.json file to control certain aspects of how the docker command behaves.
Currently, you can modify the docker command behavior using environment variables or command-line options. You can also use options within config.json to modify some of the same behavior. When using these mechanisms, you must keep in mind the order of precedence among them. Command line options override environment variables and environment variables override properties you specify in a config.json file.
The config.json file stores a JSON encoding of several properties:
The property HttpHeaders specifies a set of headers to include in all messages sent from the Docker client to the daemon. Docker does not try to interpret or understand these header; it simply puts them into the messages. Docker does not allow these headers to change any headers it sets for itself.
The property psFormat specifies the default format for docker ps output. When the --format flag is not provided with the docker ps command, Docker’s client uses this property. If this property is not set, the client falls back to the default table format. For a list of supported formatting directives, see the Formatting section in the docker ps documentation
The property imagesFormat specifies the default format for docker images output. When the --format flag is not provided with the docker images command, Docker’s client uses this property. If this property is not set, the client falls back to the default table format. For a list of supported formatting directives, see the Formatting section in the docker images documentation
The property serviceInspectFormat specifies the default format for docker service inspect output. When the --format flag is not provided with the docker service inspect command, Docker’s client uses this property. If this property is not set, the client falls back to the default json format. For a list of supported formatting directives, see the Formatting section in the docker service inspect documentation
The property statsFormat specifies the default format for docker stats output. When the --format flag is not provided with thedocker stats command, Docker’s client uses this property. If this property is not set, the client falls back to the default table format. For a list of supported formatting directives, see Formatting section in the docker stats documentation
Once attached to a container, users detach from it and leave it running using the using CTRL-p CTRL-q key sequence. This detach key sequence is customizable using the detachKeys property. Specify a <sequence> value for the property. The format of the <sequence> is a comma-separated list of either a letter [a-Z], or the ctrl- combined with any of the following:
  • a-z (a single lowercase alpha character )
  • @ (at sign)
  • [ (left bracket)
  • \\ (two backward slashes)
  • _ (underscore)
  • ^ (caret)
Your customization applies to all containers started in with your Docker client. Users can override your custom or the default key sequence on a per-container basis. To do this, the user specifies the --detach-keys flag with the docker attachdocker execdocker run or docker start command.
Following is a sample config.json file:
{
  "HttpHeaders": {
    "MyHeader": "MyValue"
  },
  "psFormat": "table \\t\\t\\t",
  "imagesFormat": "table \\t\\t\\t",
  "statsFormat": "table \t\t",
  "serviceInspectFormat": "pretty",
  "detachKeys": "ctrl-e,e",
  "credsStore": "secretservice",
  "credHelpers": {
    "awesomereg.example.org": "hip-star",
    "unicorn.example.com": "vcbait"
  }
}

Notary

If using your own notary server and a self-signed certificate or an internal Certificate Authority, you need to place the certificate attls/<registry_url>/ca.crt in your docker config directory.
Alternatively you can trust the certificate globally by adding it to your system’s list of root Certificate Authorities.

Examples

Display help text

To list the help on any command just execute the command, followed by the --help option.
$ docker run --help

Usage: docker run [OPTIONS] IMAGE [COMMAND] [ARG...]

Run a command in a new container

Options:
      --add-host value             Add a custom host-to-IP mapping (host:ip) (default [])
  -a, --attach value               Attach to STDIN, STDOUT or STDERR (default [])
...

Option types

Single character command line options can be combined, so rather than typing,docker run -i -t --name test busybox sh you can write docker run -it --name test busybox sh.

BOOLEAN

Boolean options take the form -d=false. The value you see in the help text is the default value which is set if you do not specify that flag. If you specify a Boolean flag without a value, this will set the flag to true, irrespective of the default value.
For example, running docker run -d will set the value to true, so your container will run in “detached” mode, in the background.
Options which default to true (e.g., docker build --rm=true) can only be set to the non-default value by explicitly setting them to false:
$ docker build --rm=false .

MULTI

You can specify options like -a=[] multiple times in a single command line, for example in these commands:
$ docker run -a stdin -a stdout -i -t ubuntu /bin/bash

$ docker run -a stdin -a stdout -a stderr ubuntu /bin/ls
Sometimes, multiple options can call for a more complex value string as for -v:
$ docker run -v /host:/container example/mysql
Note: Do not use the -t and -a stderr options together due to limitations in the pty implementation. All stderr in pty mode simply goes to stdout.

STRINGS AND INTEGERS

Options like --name="" expect a string, and they can only be specified once. Options like -c=0 expect an integer, and they can only be specified once.

User Interface(UI) for Docker, Portainer

Portainer gives you access to a central overview of your Docker host or Swarm cluster. From the dashboard, you can easily access any manag...