Docker Tutorial

Docker Logo
Sebastian Gutsche


sebasguts.github.io/ICMSDockerTutorial

gutsche@mathematik.uni-siegen.de

QR code

Legal disclaimer:

  • I am not affiliated with Docker, Inc.

  • This presentation is not created by Docker, Inc.


Further notes:

  • This presentation is created using reveal.js

Outline

What is Docker


To get an impression:

An enviroment for lightweight virtual machines


But: Docker uses Linux namespaces and containers, no virtual machine at all (on a Linux system).

Why use Docker/VM's


  • Easy to get working software: less maintenance for the developer, less stress for the user
  • Software distributed via VM/Docker images works on many OS
  • Software runs in predefined enviroment
  • Easy to keep distributed software up-to-date

Why use Docker


  • Docker VM's (containers) need few resources
  • Containers can interact with the host system
  • Images can be easily distributed via DockerHub

First example

How to get a working GAP?

Without Docker


$ wget http://www.gap-system.org/pub/gap/gap48/tar.gz/gap4r8p4_2016_06_04-12_41.tar.gz
$ tar x gap4r8p4_2016_06_04-12_41.tar.gz
$ cd gap4r8
$ ./configure
$ make
                    


With Docker


$ docker run -it gapsystem/gap-docker gap
                    

Mathematical software available

gapsystem/gap-docker: Image for GAP and software it depends on.


$ docker run -it gapsystem/gap-docker gap
                  

sppcomputeralgebra/sppdocker: Image for software in the German SPP Computeralgebra. Contains GAP, Singular polymake, ...


$ docker run -it sppcomputeralgebra/sppdocker Singular
                  

sagemath/sage: Image for SageMath.


$ docker run -it sagemath/sage
                  

How does it work


On Linux: Docker creates containers to isolate processes via namespaces and cgroups. This allows software executed in the Docker enviroment to be executed native on the host system, yet be isolated.

On Windows: Docker uses similar features as in Linux. No virtual machine is required anymore.

On OS X: Docker uses a lightweight virtual machine with a Linux OS.

Virtual Machines vs. Docker

    Virtual machine

  1. Hardware

  2. Host operating system

  3. Hypervisor

  4. Virtual operating system

  5. Libraries/Binaries

    Docker

  1. Hardware

  2. Host operating system

  3. Docker Engine


  4. Libraries/Binaries

Install Docker

On Linux:
Most Linux distributions have Docker in their package repositories.


$ apt-get install docker
                      

Additionally, create the group docker and add users to it, to execute Docker commands without sudo.

Attention: Users in this docker group should be considered root.

Further information can be found on the Docker website.

Install Docker

On newer Macs (>=2010, OS X >= 10.10.3):
Download Docker for Mac and install it by dragging "Moby the whale to the Applications folder".

Then start the Docker app. Docker can now be used from any terminal.

Attention: VirtualBox prior to 4.3.30 must NOT be installed on the system.

Further information can be found on the Docker website.

Install Docker

On Macs (OS X >= 10.8):
Download Docker Toolbox for Mac and install it.

Then start the Docker Quickstart Terminal.

Further information can be found on the Docker website.

Important Notions


  • Docker images: Files that contain the whole data, file system, etc.

  • Docker containers: Instances of images

  • Host: The computer Docker is running on

Start a container

There are several images already available for Docker.

Ubuntu


$ docker run -it ubuntu /bin/bash
                    

starts a container with the latest Ubuntu. This might take some time to download, approx. size 200MB.

After the command, you get a shell inside the container which you can use as always:


$ docker run -it ubuntu /bin/bash
root@45a8b7b8dfc8:/# ls
bin  boot  dev  etc  home  lib  lib64  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var
                    

Start a container

Lets examine the command


$  docker run -i -t ubuntu /bin/bash
                    

  • docker run ... ubuntu /bin/bash: Starts the command /bin/bash inside a container created from the Ubuntu image.
  • -i: Starts the container in interactive mode, i.e. connects your terminal to the bash inside the container.
  • -t: Allocates a pseudo-tty for the container

Listing the containers

To get an overview of the containers on the host, use docker ps -a.


$ docker ps -a
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS                      NAMES
45a8b7b8dfc8        ubuntu:wily         "/bin/bash"         12 minutes ago      Running                     nauseous_hopper
ab17973b21fe        ubuntu:trusty       "/bin/bash"         11 weeks ago        Exited (130) 11 weeks ago   reverent_curie
                  
  • ID: Unique ID of the container
  • IMAGE: Image from which the container comes from
  • COMMAND: Command started first in the container
  • STATUS: Status of the container
  • NAMES: Freely assignable name via the --name option

Listing the images

To see all images currently stored on the host, use docker images.


$ docker images
REPOSITORY                       TAG                 IMAGE ID            CREATED             SIZE
ubuntu                           trusty              b72889fa879c        11 weeks ago        187.9 MB
redis                            alpine              50405530a7e5        4 months ago        15.95 MB
sagemath/sage_patchbot           latest              6d0c1ab2df6c        5 months ago        7.369 GB
sagemath/sage                    latest              068b88165f36        5 months ago        7.338 GB
ubuntu                           wily                9f99e190e4e6        6 months ago        133.5 MB
                  
  • REPOSITORY: Name of the image
  • TAG: Version of the image
  • ID: Unique hash ID

Deleting a container

To delete an old container, use docker rm.


$ docker ps -a
CONTAINER ID        IMAGE               STATUS                   NAMES
b01c24bcc569        ubuntu:wily         Exited (0) 2 hours ago   sleepy_nobel
$ docker rm sleepy_nobel
sleepy_nobel
$ docker ps -a
CONTAINER ID        IMAGE               STATUS              NAMES
                  

Both NAME and ID can be used to delete a container.

After deleting the container, all data stored in it is deleted. If you want a container to be deleted after exiting, run it with the --rm flag.

Pausing a container

Containers can be paused and then later unpaused using docker pause and docker unpause. Processes running in the container will be stalled while doing this.


$ docker ps
CONTAINER ID        IMAGE               STATUS              NAMES
e19facf9c3f8        ubuntu:wily         Up 3 seconds        sleepy_nobel
$ docker pause sleepy_nobel
sleepy_nobel
$ docker ps -a
CONTAINER ID        IMAGE               STATUS                   NAMES
e19facf9c3f8        ubuntu:wily         Up 14 seconds (Paused)   sleepy_nobel
$ docker unpause sleepy_nobel
sleepy_nobel
$ docker ps
CONTAINER ID        IMAGE               STATUS              NAMES
e19facf9c3f8        ubuntu:wily         Up 26 seconds       sleepy_nobel
                  

If you restart your Linux host while a container is paused, it is not possible to unpause the container.

Attaching a container

To attach a terminal to a running container, use docker attach.


$ docker ps
CONTAINER ID        IMAGE               STATUS              NAMES
f095a8e646db        ubuntu:wily         Up 2 seconds        sleepy_nobel
$ docker attach sleepy_nobel 
root@f095a8e646db:/#
                  

Non-attached interaction

Commands can be executed inside a container using docker exec.


$ docker exec sleepy_nobel uname
Linux
                  

History of containers command execution can be seen using docker logs.


$ docker attach sleepy_nobel 
$ root@f095a8e646db:/# ls
bin  boot  dev  etc  home  lib  lib64  media  mnt  opt  proc  root  run  sbin  srv  sys  test  test2  tmp  usr  var
$ root@f095a8e646db:/# exit
exit
$ docker logs sleepy_nobel 
root@f095a8e646db:/# ls
bin  boot  dev  etc  home  lib  lib64  media  mnt  opt  proc  root  run  sbin  srv  sys  test  test2  tmp  usr  var
root@f095a8e646db:/# exit 
exit
                  

Copying files from/into containers

One can copy files from and into a running or stopped container, using docker cp.


$ touch test  
$ docker cp test sleepy_nobel:/ 
$ docker exec sleepy_nobel bash -c 'echo Hallo > test' 
$ docker cp sleepy_nobel:/test test 
$ cat test 
Hallo
                  

Further useful commands

  • docker kill: Kills a container
  • docker export: Exports containers file system into tar-ball
  • docker rename: Rename a container
  • docker diff: See changes of containers file system
  • docker top: Displays running processes of a container

Useful options for run

docker run is the most used command, since it starts a container from an image. Here are some useful optional parameters:

  • --name: Name the container
  • --rm: Delete the container after exiting
  • --volume: Mount host volume into container
  • --cpu-quota: Set CPU quota for container
  • --mac-address: Set the hardware address of the container
  • --ip: Set IPv4 address of the container

Creating an image

Images are data files from which Docker container instances can be launched from.

If you want to launch software in a Docker container or distribute your software using Docker, you need to create an image for it.

Suppose you want to create an image containing GAP.

Install the software

A good base system for GAP is Ubuntu. So we start a shell inside an Ubuntu container.


$ docker run -it --name="gap-container" ubuntu /bin/bash
                    

Most commom Linux Distributions have base images, containing their basic binaries and libraries.

After the container gap-container has been started, one can install GAP into it.

Commiting the image

The container gap-container now contains a new GAP installation, and we are ready to create an image from it. To create an image from a container, docker commit is used.


$ docker ps -a
CONTAINER ID        IMAGE               COMMAND             STATUS                       NAMES
4aa8c1248db7        ubuntu:wily         "/bin/bash"         Exited (130) 5 seconds ago   gap-container
$ docker commit gap-container gap-image
sha256:ab6088c74c86c4ff4bdaae3ac35a9cb13c317bf50d54df8d4c12a21688156de7
$ docker images
REPOSITORY                       TAG                 IMAGE ID            CREATED              SIZE
gap-image                        latest              ab6088c74c86        About a minute ago   325.5 MB
ubuntu                           wily                9f99e190e4e6        7 months ago         133.5 MB
$ docker run -it gap-image gap
                    

Writing Dockerfiles

Recreating images by hand whenever the included software changes is not desireable. Fortunately, there are script files, so called Dockerfiles.

Dockerfiles contain the steps from a base image to an image containing your software, e.g., shell commands, added files, and standard settings for the image.

Structure of Dockerfiles

Here is a small example to create an image containing polymake


FROM ubuntu:latest
MAINTAINER Sebastian Gutsche
RUN apt-get update && apt-get install -y polymake
CMD polymake
                  
  1. FROM: Defines the base image we are using, i.e., the latest ubuntu image.
  2. MAINTAINER: Sets the name of the image maintainer.
  3. RUN: Commands that set up the image, i.e., installing polymake from the Ubuntu repositories.
  4. CMD: Sets the default command which is executed when a container is started from the image.

Building images from Dockerfiles

To build an image from a Dockerfile, the command docker build is used.


$ ls
Dockerfile
$ docker build --tag="polymake-image" .
$ docker images
REPOSITORY                       TAG                 SIZE
polymake-image                   latest              7.338 GB
$ docker run -it polymake-image
                  

Further commands in Dockerfiles

There are some more useful commands for Dockerfiles:

  • ADD: Adds files from the host system into the image. Useful to add installation scripts to not pollute the Dockerfile.
  • USER: Changes the user for all following steps in the Dockerfile. Default is root.
  • WORKDIR: Sets the directory in which the next commands in the Dockerfile are executed. Default is /.
  • ENV: Sets an enviroment variable to a certain value.
  • SHELL: Sets the default shell for the next command. Default is sh.

Please note that all the default values given here depending on the choosen base image.

Notes about the build process

Images contain layered file systems, i.e., each layer only contains the changes to the layer before. Each command produces such a layer.

Furthermore, builds are cached. If a Dockerfile has been build before, only the layers from the first changed command are newly build.

If the image needs to be rebuild completely, one can delete the old images, or use the option --no-cache.

Remember: Delete temporary files

When writing Dockerfiles, one should always make sure to delete all temporary files created in a command in the same command.

Images are layered, so each file is stored in a layer once it is created. If it is deleted in a different layer, it size will still add to the final image size.

What is DockerHub

DockerHub is a puplic registry for Docker images, which is free to use.

If you try to run an Ubuntu image like before, and it is not stored on your computer, Docker automatically downloads it from DockerHub.

The good news: You can also create an account on DockerHub and upload your images, so people can use them as easy!

Because of this, the GAP container can be run as easy as this:


docker run -it gapsystem/gap-docker gap
                  

Using DockerHub

First of all, you need to create an account on DockerHub (http://hub.docker.com).

Then you can connect your system to DockerHub using docker login.


$ docker login
Username: sebasguts
Password:
$
                  

Uploading images

To upload an image to DockerHub, one needs to name it the following way: username/image_name[:tag]. This can be done using docker tag.


$ docker tag polymake-image sebasguts/polymake-image:latest
                  

After the image is named correctly, it can be uploaded to DockerHub using docker push.


$ docker push sebasguts/polymake-image:latest
                  

After uploading, everybody can download the image and start polymake via


$ docker run -it sebasguts/polymake-image
                  

Automated builds on DockerHub

DockerHub can also build images from your Dockerfiles.

This can be used by uploading a repository containing the Dockerfile to GitHub. After that, you can configure DockerHub to build an image from the Dockerfile everytime changes are pushed to this repository.

DockerHub does not use caches, so one always gets a fresh build.

Please note that some undisclosed ressource restrictions apply to the automated builds.

Automated builds on DockerHub

DockerHub

DockerHub

DockerHub

DockerHub

DockerHub

DockerHub

DockerHub

DockerHub

DockerHub

Container networks

An important feature of Docker containers are the vast options for networks in and between containers.

By default, each container gets a different IP address and can be configured to listen on various ports. But this is configurable for each container at startup.

For security: Host and container networks are seperated, and container, host, and the outside world communicate via a virtual network bridge

Container networks

Some options for container networks:

  • Set the container network to the host network
  • Map host ports to container ports
  • Link ports of several containers together

Mapping ports to the host

Suppose there is an image polymake-jupyter, containing an installation of polymake and the jupyter server.


$ docker run -d -p 8080:8080 --name="jupyter" polymake-jupyter jupyter notebook
                  

After starting, we can access the notebook server under 127.0.0.1:8080.

To check the port bindings, use docker port.


$ docker port jupyter
8080/tcp -> 0.0.0.0:8080
                  

To use the hosts network in the container, use --net="host".

Connecting two containers

It is also possible to connect two containers using the --link option.


$ docker run -it --name="container1" ubuntu
$ docker run -it --link container1:container1 ubuntu
root@00043959c31a:/# ping container1
PING container1 (172.17.0.2) 56(84) bytes of data.
64 bytes from container1 (172.17.0.2): icmp_seq=1 ttl=64 time=0.189 ms
64 bytes from container1 (172.17.0.2): icmp_seq=2 ttl=64 time=0.165 ms
                  

An example use case is a link between a webserver and a data base

Thank you for your attention!