Docker - Deep Dive

What is this Docker thing?

A company?
A format?
An API?

Linux containers? That’s not new…

Linux-native functionality
Has been around ~ 10 years?
cgroups
kernel namespaces
chroot
Linux capabilities
Security (SELinux)

cgroups

Built into Kernel (RHEL7/Debian/etc)
Generically isolates resource usage (CPU, memory, disk, network)
Guarantee resources to app/set of apps
Can be adjusted on the fly
Can monitor the cgroup itself to see utilization

Kernel namespaces

Isolating views of the system
Can make a process think it’s the only process
Built-in way to "virtualize" a process

Kernel namespaces

mnt (mount points, filesystem)
pid (processes)
net (network stack)
ipc (inter-process comms)
uts (hostname)
user (UIDs)

Docker - CGroups/namepsace in Action

Docker brings together

Why is this important?

Image format vs golden image
API
Packaging
Separation of concerns (Devs/Ops)
Density, infrastructure utilization

Docker format

Process virtualization

Immutable infrastructure

"We’ll put it back in Ansible"
Cattle vs Pets
Don’t change it; replace it
System created fully from automation; avoid drift
Manual intervention is error prone
How does Docker help?

Basic
Docker components

Docker client
Docker daemon
Images
Registry
Containers

Basic
Docker components

Docker images

Templates from which containers are created
Layered using union filesystems
Each change to the system is a layer
Typically created with Dockerfiles/instructions
Stored in a docker registry (public/private)

Docker containers

Runtime instances of a Docker Image
Copy on write file system; changes localized
"virtualized" with namespaces, cgroups, selinux, etc
Has own IP address/networking/volumes
Intended to run single process (process virtualization)

Developer workflow

work from vagrant image
can trash and reboot it any time
locally running docker client
Source code in developer IDE
When ready, use tooling to generate docker image (or hand craft)
Run image locally (possibly with others)
Push code (or image?)
CI process kicks in

Developer works locally

Developer pushes code

Pull a docker image

Pull Centos7 from DockerHub ( https://hub.docker.com)

docker pull centos:7

output:

rawsanj@ubuntu(~) $ docker pull centos:7
        7: Pulling from library/centos

fa5be2806d4c: Pull complete
        0cd86ce0a197: Pull complete
        e9407f1d4b65: Pull complete
        c9853740aa05: Pull complete
        e9fa5d3a0d0e: Pull complete
        Digest: sha256:def5c79bc29849815dec7dddc8f75530a9115c94d5b17e0e6807f929902fab62
        Status: Downloaded newer image for centos:7

List docker images

List locally, installed images

docker images

output:

rawsanj@ubuntu(~) $ docker images
    REPOSITORY          TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
    centos              7                   e9fa5d3a0d0e        2 days ago          172.3 MB

Show all images, including itermmediate

docker images -a

output:

rawsanj@ubuntu(~) $ docker images -a
    REPOSITORY          TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
    centos              7                   e9fa5d3a0d0e        2 days ago          172.3 MB
    <none>              <none>              e9407f1d4b65        2 days ago          172.3 MB
    <none>              <none>              0cd86ce0a197        2 days ago          172.3 MB
    <none>              <none>              fa5be2806d4c        5 weeks ago         0 B

Let’s run a Docker container!

Let’s run a linux command inside a docker container:

docker run --rm centos:7 echo "hello world"

output:

rawsanj@ubuntu(~) $ docker run --rm centos:7 echo "hello world"
    hello world

Woah, what happened? It just printed out "hello, world"? So what?

Let’s run a Docker container!

Let’s run a shell inside a docker container:

docker run -it --rm centos:7 bash

output:

[root@d7dfcc490cbe /]# _

Cool! We have a bash shell, and a minimal distro of Centos 7! Did you see how fast that booted up? Typing ls -l /etc/*-release from the new bash prompt shows us we indeed have a Centos 7 distro:

[root@c2c2b8a65afe /]# ll /etc/*-release
    -rw-r--r-- 1 root root  38 Mar 31  2015 /etc/centos-release
    -rw-r--r-- 1 root root 393 Mar 31  2015 /etc/os-release
    lrwxrwxrwx 1 root root  14 Aug 14 21:00 /etc/redhat-release -> centos-release
    lrwxrwxrwx 1 root root  14 Aug 14 21:00 /etc/system-release -> centos-release

Let’s run a Docker container!

Run some other commands from within the container:

  hostname -f
  cat /etc/hosts
  ps aux
  yum -y install vim
  ip a

A real linux distro right? Did you notice that
ps faux didn’t show too many processes?

Let’s run a Docker container!

Let’s do some destructive stuff:

rm -fr /usr/sbin

Wuh? you deleted all of the sacred system tools!?

Let’s delete some user tools too

rm -fr /usr/bin

output:

[root@c2c2b8a65afe /]# ls
    bash: /usr/bin/ls: No such file or directory

Whoops… cannot ls or do anything useful anymore. What have we done!?

Let’s run a Docker container!

No worries! Just exit the container and fire up a new one:

docker -it --rm centos:7 bash

Everything is back! Phew….

Deploy Apache Tomcat

Now let’s run a JVM based application like Apache Tomcat:

docker run --rm -p 8888:8080 tomcat:8.0

Since the Tomcat 8.0 docker image doesn’t exist, Docker will try to automatically pull it from the registry. Give it a moment, and you should see tomcat start successfully:

16-Oct-2015 18:30:51.541 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web application directory /usr/local/tomcat/webapps/manager has finished in 28 ms
    16-Oct-2015 18:30:51.542 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deploying web application directory /usr/local/tomcat/webapps/examples
    16-Oct-2015 18:30:52.108 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web application directory /usr/local/tomcat/webapps/examples has finished in 566 ms
    16-Oct-2015 18:30:52.117 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deploying web application directory /usr/local/tomcat/webapps/ROOT
    16-Oct-2015 18:30:52.161 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web application directory /usr/local/tomcat/webapps/ROOT has finished in 45 ms
    16-Oct-2015 18:30:52.176 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["http-nio-8080"]
    16-Oct-2015 18:30:52.206 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["ajp-nio-8009"]
    16-Oct-2015 18:30:52.208 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in 1589 ms

Deploy Apache Tomcat

Let’s explore that command for a quick sec:

docker run --rm -p 8888:8080 tomcat:8.0

--rm tells us that we want to remove the container (delete) when it’s done running
-p 8888:8080 tells us we want to map the container’s port 8080 to the host port of 8888

So if we try to connect to http://localhost:8888 we should be able to reach our tomcat server!

Deploy Apache Tomcat

Well, not quite. Why not?

docker-ports

Our Docker Host has been mapped properly, but we cannot reach it from our host (Windows/MacOSX) because the VM does not expose those ports.

Deploy Apache Tomcat

Now navigate in a browser to http://VM_IP_ADDRESS:8888

tomcat

You can use Kitematic to map container ports and expose VM IP Address.

Stop and Remove container

Feel free to play around with the container a little bit more.
When finished, stop the container:

docker stop tomcat8

If you run docker ps you shouldn’t see the container running any more.
However, docker run -a will show all containers even the stopped ones. We can remove a container with:

docker rm tomcat8

Then neither docker ps nor docker ps -a should show the container.

Docker Images

Image tags
Points to a specific layer
Usually the last most layer gets changed
Can have multiple tags each pointing to diff layers; same base
don’t use latest if you can help it

Public/private docker registry

Docker hub: https://hub.docker.com
Can host public images
- ie, search for fedora, or jenkins, etc
Can also host private repos (like github)
Other registries:
- AWS Elastic Container Registry
- JFrog
- Quay.io
- Google Container Registry

Enterprise docker registry

Be careful with images on Docker hub
- vulnerabilities
- run as root
- http://www.infoq.com/news/2015/05/Docker-Image-Vulnerabilities
Use trusted registries

Creating Docker images

Encouraged to build docker images from Dockerfiles

FROM java:8
        
MAINTAINER @RawSanj

ADD aws-s3-file-system-0.1.0.jar app.jar

RUN sh -c 'touch /app.jar'

ENV JAVA_OPTS=""

CMD [ "sh", "-c", "java $JAVA_OPTS -jar /app.jar" ]

Dockerfile

Constructs such as the following:

FROM
ADD
COPY
USER

ENV
VOLUME
WORKDIR
CMD
ENTRYPOINT

Each step in the docker file is a new image layer! Don’t put passwords into the docker file!

Advanced Dockerfile

FROM ubuntu:14.04
        
MAINTAINER @RawSanj

ENV GERRIT_HOME /home/gerrit
ENV GERRIT_TMP_DIR /home/tmp
ENV GERRIT_USER gerrit
ENV GERRIT_VERSION 2.11

RUN \
    sed -i 's/# \(.*multiverse$\)/\1/g' /etc/apt/sources.list && \
    apt-get update && \
    DEBIAN_FRONTEND=noninteractive apt-get -y upgrade && \
    DEBIAN_FRONTEND=noninteractive apt-get install -y sudo vim-tiny git && \
    DEBIAN_FRONTEND=noninteractive apt-get install -y openjdk-7-jre-headless && \
    DEBIAN_FRONTEND=noninteractive apt-get install -y curl

# Add user gerrit & group like also gerrit to sudo to allow the gerrit user to issue a sudo cmd
RUN groupadd $GERRIT_USER && \
    useradd -r -u 1000 -g $GERRIT_USER $GERRIT_USER

RUN mkdir ${GERRIT_HOME}

# Download Gerrit
ADD http://gerrit-releases.storage.googleapis.com/gerrit-${GERRIT_VERSION}.war ${GERRIT_HOME}/${GERRIT_WAR}

# Copy the files to bin, config & job folders
ADD ./configs ${GERRIT_HOME}/configs

# Copy the plugins
ADD ./plugins ${GERRIT_HOME}/plugins

WORKDIR ${GERRIT_HOME}

EXPOSE 8080 29418
CMD ["/home/gerrit/bin/conf-and-run-gerrit.sh"]

Difference between CMD & ENTRYPOINT

CMD can be overridden at run time
- docker run -it centos:7 <command_to_run>
ENTRYPOINT fixed command, pass things in as parameters
- docker run -it centos:7 <params to add>

Registry architecture

Deploying Spring Boot App

We will follow the similar steps we used to deploy Tomcat and will re-use the port mappings we had earlier. Verify your tomcat server is no longer running on 8888 and follow these steps:

$ docker pull rawsanj/aws-s3-file-system
$ docker run -d -p 8080:8080 rawsanj/aws-s3-file-system \
$ -e cloud.aws.credentials.accessKey=ACCESS_KEY
$ 
$ docker ps

Navigate to http://VM_IP_ADDRESS:8080 to see the main page.

Quickly understand default docker networking

docker-network

Linking Containers

We want containers to communicate with each other
Each container has an IP (veth/eth0)
Containers can expose virtual ports
Docker bridge networking can link containers without going over host network
How do we discover IP addresses, etc?

docker run --link <container_name>:alias .....

Linking Containers

docker run --link <container_name>:alias .....

Name your containers!!!!!!!
links depend on this
alias is what your containers will see as environment variables

example:

docker run --rm --name web2 --link db:db training/webapp env

Will create these environment variables:

DB_NAME=/web2/db
DB_PORT=tcp://172.17.0.5:5432
DB_PORT_5432_TCP=tcp://172.17.0.5:5432
DB_PORT_5432_TCP_PROTO=tcp
DB_PORT_5432_TCP_PORT=5432
DB_PORT_5432_TCP_ADDR=172.17.0.5

Your applications can then use environment variables to discover the dependent containers/services

Examples of linking:

Run a database service which will be used by a Java EE application:

docker run --name mysqldb -e MYSQL_USER=mysql \
    -e MYSQL_PASSWORD=mysql -e MYSQL_DATABASE=sample \
    -e MYSQL_ROOT_PASSWORD=supersecret -p 3306:3306 -d mysql

Note, you may need to forward the mysql port 3306 in your VM.

Now let’s link up a Java EE application

docker run -d --name mywildfly --link mysqldb:db -p 8080:8080 \
arungupta/wildfly-mysql-javaee7

Examples of linking:

Our app is new using the DB, but let’s log into the container and verify the environment variables/DNS was set up:

docker exec -it mywildfly bash

then type the following to list environment variables:

env

You can also take a look at the /etc/hosts file

cat /etc/hosts`

[jboss@c924917fe4ad ~]$ cat /etc/hosts
    172.17.0.30     c924917fe4ad
    127.0.0.1       localhost
    ::1     localhost ip6-localhost ip6-loopback
    fe00::0 ip6-localnet
    ff00::0 ip6-mcastprefix
    ff02::1 ip6-allnodes
    ff02::2 ip6-allrouters
    172.17.0.28     db ef59a1b98326 mysqldb
    172.17.0.30     mywildfly.bridge
    172.17.0.18     registry
    172.17.0.18     registry.bridge
    172.17.0.28     mysqldb
    172.17.0.28     mysqldb.bridge
    172.17.0.30     mywildfly

Docker containers have no state!

Containers are ephemeral!!!
Nothing is saved from a container if it goes away
Containers get new IP addresses
Don’t treat containers as VMs: they are not!!
But what about stateful applications?

Docker volumes

Docker volumes to the rescue!
Persist data outside of the container
Can be mapped directly to Host locations
Can also be deployed independently of hosts/indirectly

Example:

docker run -d -P --name web -v /webapp training/webapp python app.py

Example Docker volumes

Example:

docker run -d -P --name web -v /webapp training/webapp python app.py

Docker host volumes

We can also map volumes directly to Host storage locations:

Useful for known locations on Host
Can use NFS mounts
Files are visible outside of the container and are persisted
Can restart new containers up with same location

docker run -d -P --name web -v /src/webapp:/opt/webapp \
training/webapp python app.py

Example Docker host volumes

Example:

docker run -d -P --name web -v /src/webapp:/opt/webapp \
training/webapp python app.py

Jenkins example

Let’s take an example using Jenkins. We can fire up Jenkins containers, add build jobs, etc. But if we delete the container, the jobs are lost.

docker run -d --name jenkins -p 8080:8080 jenkins

We can save the changes and jobs that jenkins creates by adding a host volume:

docker run -d --name jenkins -p 8080:8080 \
-v /your/home:/var/jenkins_home jenkins

Now when you run jenkins, you can stop, destroy, and re-run jenkins and your build jobs should be there.

Things to know

Docker runs on a single host!
/var/lib/docker needs to be managed!
Use only what you need in your images (avoid image bloat)
Don’t run as root
Be careful with docker images on Docker Hub (use trusted images only)
Container security… containers do not "contain"
- http://opensource.com/business/14/7/docker-security-selinux
always use image tags
Use sanity scripts to boot your process from within container
One task per container!

Play with Docker

Try running a containerized Angular.js app. You should see a href link for port 80 in Play with Docker dashboard.

docker run -d -p 80:80 rawsanj/ng-imdb

Lets try running a Spring MVC Application and expose it on port 8080.

docker run -d -p 8080:8080 spring-tiles-sample-app

Lets try something cool, run Ubuntu Desktop inside a Container and access it on browser.

docker run -it --rm -p 6080:80 dorowu/ubuntu-desktop-lxde-vnc

Resources and Examples

Repository for Spring Boot S3 App

Repository for Angular.js App

Repository for Spring MVC Application

Interesting article on Java in Docker - Nobody puts Java in a Container

JDK 10 improvements for Containers - Better containerized JVMs in JDK 10

	E-Mail
	@RawSanj
	@Raw_Sanj
	stack overflow

Play with Docker

Questions?