Docker Intro - Brandon Mitchell

name: empty
layout: true
---
name: base
layout: true
template: empty
background-image: none
<div class="slide-footer">@sudo_bmitch</div>
---
name: ttitle
layout: true
template: empty
class: center, middle
background-image: url(img/containers_bg.png)
background-size: cover
---
name: inverse
layout: true
template: base
class: center, middle, inverse
background-image: none
---
name: impact
layout: true
template: base
class: middle, impact
background-image: url(img/containers_bg.png)
background-size: cover
---
name: picture
layout: true
template: base
class: center, middle
background-image: none
---
name: code
layout: true
template: base
class: terminal
background-image: none
---
name: terminal
layout: true
template: base
class: center, middle, terminal
background-image: none
---
name: default
layout: true
template: base
class: bg-blur
background-image: url(img/containers_bg.png)
background-size: cover
---
layout: false
template: default
name: agenda

# Agenda

.left-column[
- [What is a Container?](#container)
- [Running Containers](#docker-run)
- [Volumes](#volumes)
- [Networking](#networking)
- [Compose](#compose)
- [Security](#security)
]
.right-column[
- [Best Practices](#best-practices)
- [Building Images](#building)
- [Multi-Stage Builds](#multi-stage)
- [Registry](#registry)
- [Orchestration](#orchestration)
]

---
layout: false
template: ttitle
name: title

# Docker Intro

.left-column[
.pic-circle-70[![Brandon Mitchell](img/bmitch.jpg)]
]
.right-column[.align-right[.no-bullets[
<br><br>
- Brandon Mitchell
- Twitter: @sudo_bmitch
- GitHub: sudo-bmitch
]]]
???
- My twitter and github handles are what any self respecting sysadmin does
  when you get a permission denied error on your favorite username.
- This presentation is on github and I'll have a link to it at the end,
---
template: default

```no-highlight
$ whoami
- Solutions Architect @ BoxBoat
- Docker Captain
- Frequenter of StackOverflow
```

.align-center[
.pic-30[![BoxBoat](img/boxboat-logo-color.png)]
.pic-30[![Docker Captain](img/docker-captain.png)]
.pic-30[![StackOverflow](img/stackoverflow-logo.png)]
]

???

---

# What is a Container?

---

# Containers

- App vs OS isolation
- Namespaces: Limit what the app sees
- cgroups: Limit what resources it can use
- Images vs Containers

???

- Shared kernel isolating apps vs shared hardware isolating OSs
- Namespaces: Mount, PID, Network, User, UTS (hostname), IPC (inter process comm)
- cgroups: CPU, memory, future may include I/O
- Images vs Containers: Class vs Object, Definition vs Instance
- Images: filesystem and metadata
- Containers: extend image with config, RW FS layer, logs, namespaces and cgroups

---

<asciinema-player src="demo-1-1-container.cast" cols=100 rows=26 preload=true font-size=16></asciinema-player>

---

# Images

- Filesystem and Metadata to create a container
- Application packaging format
- Application code/binaries and runtime are shipped together
- Do not include configuration and data
- Built with a Dockerfile
- Immutable layers

???

- Filesystem is packaged as a set of tar files
- Metadata includes the defaults for the container: command to run, env,
  username, etc, which can be overridden
- Runtime includes JRE, Python interpreter, Node, plus libraries and other dependencies
- Separate your app into:
  - Binaries, libs, deps: goes in image
  - Configuration: env vars and conf files injected at runtime
  - Data: volumes mounted into container
- Dockerfile is the list of instructions to process to create an image reproducibly 
- Each step of a Dockerfile creates a filesystem delta, packaged as a tar, sha checksum, never changed
  - Allows images to extend other images
  - Changes require creating a new image, fundamentally diff from VMs
  - Eliminates state drift

---

<asciinema-player src="demo-1-2-image.cast" cols=100 rows=26 preload=true font-size=16></asciinema-player>

---

# Why?

- Ops: Less overhead than a VM
- Dev: Works on my machine
- DevOps: Portability and faster deploys
- Ephemeral containers prevent state drift
- Enable microservices
- Horizontal Scaling

???

- Containers don't need entire OS, so you can run more
- Devs create same filesystem that will exist in prod, eliminate entire class of testing issues
- Containers are very fast to release, and to roll back, seconds vs ansible playbook time
- State drift:
  - Who likes uptime?
  - who wants to manage a server that's been running without a reboot for 10 years and needs patches for spectre/meltdown
  - We now have a reverse uptime metric, companies intentionally stop containers that have been running for too long
  - Uptime can go down, but availability can go up with rolling updates and LB's
- Microservices:
  - Conways Law: orgs design structures that mirror their communication structure
  - Containers are designed for different groups to dev their own app, communicate across network, REST APIs
- Google doesn't scale vertically, there's no single host running www.google.com, they run lots of small servers
  - When you need more scale, you add more servers to share the load rather than a bigger server

---

# Which Docker

- Docker Community Edition (CE - for Linux)
- Docker Desktop (Windows and Mac)
- Docker Enterprise
- Docker for Windows Server

???

- On servers, it's Linux and focus of this presentation
- Desktop versions for Windows and Mac include an embedded Linux environment
  - Windows also supports Windows Containers
- Enterprise is CE with a few more features, now owned by Mirantis
- Windows Server supports both Win and Linux on the server, Enterprise
  - First 3rd party service shipped by MS with their OS

---

# Related Projects

- Moby
- containerd
- runc
- buildkit / buildx
- linuxkit
- CNAB
- Docker App

???
- Moby: open source upstream of Docker, similar to Fedora vs RHEL
- containerd forked out of Docker, just images/containers, no networking or volume code
- runc: is one of many runtimes, runsc (gvisor), kata (VMs). Takes FS+Net, adds other namespaces and runs/watches single container
- buildkit/buildx: revamped image build tooling
- linuxkit: linux OS as containers, part of Docker Desktop
- CNAB: cloud native application bundle, packages a multi-container app, images, and scripted installer together
- Docker App: CNAB implementation that is specific to Docker Tooling
- These are just the Docker projects, see also CNCF landscape

---

# Running Containers

---

# Running Containers

- Uses an image to generate a container
- CLI is order sensitive

```no-highlight
$ docker [opts] container run [run_opts] <image_name> [cmd] ...
```

???
- First docker command most learn is `run`
- This take the image and turns it into a running container
- CLI order matters, if you put options after the image name they become the command

```no-highlight
$ docker container run -it --rm busybox echo hello world
hello world
$ 
```

---

# Container Lifecycle

```no-highlight
$ docker container create <image_name>
$ docker container start <container_name>
```

- Create a container from an image
- Start that container

???
- `create` sets up the docker metadata for the container, sets up disk for the
  container RW layer and log files
- `start` generates the various namespaces, attaches to the network (IPAM),
  and executes the command within the namespaces/cgroup

---

# Container Lifecycle

```no-highlight
$ docker container run <image_name>
```

- Run: combination create and start

???
- `run` is a combination of `create` and `start`

---

# Container Lifecycle

```no-highlight
$ docker container run <image_name>
$ docker container ls -a
```

- `ls` shows containers

???

- By default, `ls` only shows running containers

---

# Container Lifecycle

```no-highlight
$ docker container run <image_name>
$ docker container ls -a
$ docker container restart <container_name>
```

- Restart a running container (stop and start)

???

- Restarting a container doesn't replace the container, you still have all
  the same namespaces, logs, and filesystem changes.
- This also doesn't change the image being used, so not useful after building
  a new image

---

# Container Lifecycle

```no-highlight
$ docker container run <image_name>
$ docker container ls -a
$ docker container restart <container_name>
$ docker container stop <container_name>
```

- Stop a running container

???

- Kills the process with a SIGTERM, 10 second sleep, followed by SIGKILL

---

# Container Lifecycle

```no-highlight
$ docker container run <image_name>
$ docker container ls -a
$ docker container restart <container_name>
$ docker container stop <container_name>
$ docker container kill <container_name>
```

- Forcefully kill a container

???

- Immediately runs a SIGKILL, could result in data loss

---

# Container Lifecycle

- Delete a container

???

- Delete container logs
- Delete any filesystem changes from the image
- Delete metadata stored in docker

---

<asciinema-player src="demo-1-3-lifecycle.cast" cols=100 rows=26 preload=true font-size=16></asciinema-player>

???
---

# Debugging Containers

```no-highlight
$ docker container logs <container_name>
```

- Show STDOUT/STDERR

???

- Write your app logs to stdout/stderr and Docker will handle them for you
- Do not write to files inside the container if avoidable
- Docker can be configured to forward to log aggregators (Elastic/Splunk)

---

# Debugging Containers

```no-highlight
$ docker container logs <container_name>
$ docker container inspect <container_name>
```

- Inspect: shows metadata for an object in docker

???

- Metadata includes what image, start/stop times, exit code, and any options passed when running
- Very powerful for scripting and automation, output is json formatted and
  format options let you select/filter/format using Go templates

---

# Debugging Containers

```no-highlight
$ docker container logs <container_name>
$ docker container inspect <container_name>
$ docker container exec <container_name> <cmd>
```

- Exec: Run a command in the same container environment

???

- The most common exec is a shell to get a command prompt inside the container
- Useful for developers, code smell in production

---

<asciinema-player src="demo-1-4-debugging.cast" cols=100 rows=26 preload=true font-size=16></asciinema-player>

???
---

# Persistent Data?

---

# Volumes

- Ephemeral Containers = Data Loss
- Volumes mount an external source into containers
- Not versioned or backed up by docker
- Many to many relationship between containers and volumes

???

- Container FS is deleted with container rm
- Volumes: mount external source into container
  - Mounting hides the previous contents
- External location isn't controlled by docker, no versioning or backups
  - Deleted files in a volume are gone, recreating the container won't reset
- Containers may have multiple volumes, mounting different directories
- Multiple containers may mount the same volume (user manages file locking)

---

# Volume Types

- Named:
  - Docker manages with a name
  - Initialized when empty to image contents
- Host:
  - Bind mount of directory on the host for external access
  - No initialization, permission issues
- Anonymous:
  - Same as named volume, but with guid
- Tmpfs:
  - Not really a volume, data stored in RAM, never persisted

???

- Named volumes are preferred
- Host volumes only when you need them
  - Permission issues are common
  - UID/GID mismatch between container and host user
- Anonymous volumes are often accidents and where data is lost
  - Typically when image defines a VOLUME target and container doesn't provide a source
- tmpfs is useful for data that doesn't need to be saved
  - Lost on container restart

---

# Managing Volumes

```no-highlight
$ docker volume
...
Commands:
  create      Create a volume
  inspect     Display detailed information on one or more volumes
  ls          List volumes
  prune       Remove all unused local volumes
  rm          Remove one or more volumes
...
$ docker volume create [opts] <name>
$ docker container run -v <src>:<tgt>[:opts] <image>
```

???

- `docker volume` has similar commands, create, inspect, ls, rm
- We can create a volume in advance with `create`, useful with options
- Running a container with `-v` includes a volume, `src:tgt`
  - excluding src results in anonymous vol
  - options include RO, sync options for Mac

---

# Local Volume Driver

```no-highlight
$ docker volume create --driver local \
    --opt type=none \
    --opt device=/home/user/test \
    --opt o=bind \
    bind_vol

$ docker volume create --driver local \
    --opt type=nfs \
    --opt o=nfsvers=4,addr=nfs.example.com,rw \
    --opt device=:/path/to/dir \
    nfs_vol
```

Options are passed through to `mount` syscall

???

- By default, local named volume stores locally in /var/lib/docker
- We can change the default with options to local driver
- This is the mount syscall, not cmd, but most cases are identical
- `type` is typically the filesystem type, ext4, nfs, with bind it's none
- `o` is for options
- `device` is often the drive, e.g. /dev/sda1
- Why do a bind mount this way instead of a host mount?
  - Named bind mount will initialize host from image, including uid/gid perms

---

<asciinema-player src="demo-1-5-volumes.cast" cols=100 rows=26 preload=true font-size=16></asciinema-player>

???
---

# Networking

???

- Microservies communicate over the network
- So running multiple containers needs a network to communicate
- And external access needs a way to reach container ports

---

# Networking

- Network is namespaced
  - Private IP
  - Localhost is internal to the container
- Bridge: Linux software switch
  - Multiple containers on the same bridge communicate on container ports
- Overlay network: bridge that spans multiple hosts
- DNS: enabled on user created networks
  - Resolves container name, service name, aliases, container ID, not hostname
- Publishing Ports: port forward from docker host into container
- Expose: metadata/documentation from image creator to user

???

- Namespaced: 127.0.0.1 inside a container will not talk to the host
  (by default, host networking)
- Bridge: how containers talk to each other
- Overlay: setup by swarm and K8s to allow containers to talk between hosts
- DNS: container IP's change, DNS is built in service discovery
  - must be a user created network
- Publish: used for external access, don't connect to container IP
- Expose: not required, documents port image creator expects app to listen on inside container

---

# Networking

This doesn't work

```no-highlight
$ docker network create usernet
$ docker container run -d \
    --net usernet -p 8080:80 --name web \
    nginx
$ docker container run -it busybox ping web
$ docker container run -it --net usernet \
    busybox curl http://web:8080/
$ curl http://localhost/
```

???

- None of these ping and curl commands work, why?
- ping is not on the same network, no DNS, diff bridge
- curl inside a container needs the container port
- curl on host needs the published port

---

# Networking

This will work

```no-highlight
$ docker network create usernet
$ docker container run -d \
    --net usernet -p 8080:80 --name web \
    nginx
$ docker container run -it --net usernet \
    busybox ping web
$ docker container run -it --net usernet \
    busybox curl http://web/
$ curl http://localhost:8080/
```

???
- Same network, and DNS, on user created networks for container networking
- Container port when doing container networking
- Host port outside of the container

---

# Networking

- Link: legacy feature, based on static hostname entries
- Publish on a single interface: `-p 127.0.0.3:80:8080`
- Reverse proxies / Ingress
- Container IP address

???

- Few miscellaneous points...
- Don't use links, when your target container is replaced, it gets a new IP, and the link no longer works.
- There is more than one loopback IP, but this could be any IP on the host. Host port 80, container port 8080
- Many reverse proxy options: nginx, haproxy, traefik
  - Publish a port (HTTP/HTTPS) to the proxy, that talks to each container
  - Access containers by hostname/path/header/etc
  - Key component of microservices deployment, useful for internal LB
- Containers will have a private IP, don't use it, publish ports to host or DNS between containers
  - IP addresses change as containers are replaced
  - Static IP's for a container eliminate the ability to scale or do rolling updates
  - Static IP's are a code smell, treating container as a VM, cattle not pets

---

<asciinema-player src="demo-1-6-networking.cast" cols=100 rows=26 preload=true font-size=16></asciinema-player>

???
---

# Compose
---

# Compose

- `docker` is a CLI to REST API for `dockerd` daemon
- `docker-compose` is another CLI to same API
- Configuration is stored in YML file: `docker-compose.yml`
- Args to `docker` get moved into YML
- Multi-container projects
- Scaling replicas of a container
- Networking with DNS
- Project name is prefixed on objects
- Declarative vs Imperative

???

- We've been using a CLI to talk to a REST API
- Compose avoids the need to remember all the args, moves into YML
- Default settings are useful for most projects
- Multi-container projects need networking, DNS, scaling
- Project name is a kind of namespacing of an app
- Named volumes are not deleted without extra flags
- Declarative: define target state and compose compares to current, only runs
  needed commands, YML is the target state

---
class: small

# docker-compose.yml

```no-highlight
version: '2'

volumes:
  data:

networks:
  frontend:
  backend:

services:
  app:
...
  db:
...
```

???
- YML is white space sensitive, JSON compatible
- Top level includes version, volumes, networks, and services
  - Must have version and services
- Version 2 is a minimum, 2.x for single node docker-compose, 3.x to support swarm mode
- Volumes and Networks define values to be used within services
---
class: small

# docker-compose.yml

```no-highlight
...
services:
  app:
    image: app:latest
    environment:
    - "DB_HOST=db"
    ports:
    - 8080:80
    networks:
    - frontend
    - backend
  db:
    image: mysql
    networks:
    - backend
    volumes:
    - "data:/data"
```

???
- Multiple services, app and db
- Values within services are thinks you could create on `docker run`

---

# docker-compose

```no-highlight
$ docker-compose up -d
$ docker-compose ps
$ docker-compose restart app
$ docker-compose logs app
$ docker-compose down
```
???
- Most of the CLI's match `docker` commands
- `up` to start everything instead of `run`
- `down` to stop and cleanup everything except for named volumes

---

<asciinema-player src="demo-1-7-compose.cast" cols=100 rows=26 preload=true font-size=16></asciinema-player>

???
---

# Security

---

# Security

- Capabilities
- Seccomp
- RO Filesystem
- Scratch / Distroless
- AppArmor / SELinux
- Secrets

???
- Capabilities limit what root can do (shutdown, mount, change time, etc)
- Seccomp limit what syscalls are permitted
- Container root filesystem's can be made Read Only
- Minimal images can include those without any normal commands like a shell or libc
- Linux Security Modules AppArmor and SELinux restrict file and network operations
- There are several ways to manage secrets:
  - Avoid ENV vars (not scoped within app, exposed to child processes, dumped in
    logs, visible in inspect)
  - Use Swarm or K8s secrets, Vault, or Key Mgmt Svc (KMS)

---

<asciinema-player src="demo-1-8-security.cast" cols=100 rows=26 preload=true font-size=16></asciinema-player>

???
---

# Best Practices

---

# Best Practices

- Separate code/deps, config/secrets, data
- One app per container
- Don't run as root
- Named volumes
- Minimize published ports
- Reverse proxies

???
- Code/Deps go in Image, config/secrets injected in compose/manifest, data stored in volumes
- Multi-app containers lose readable logs, error handling, and ability to update a single app
- Use user namespaces or run containers as non-root to minimize escape risk (e.g. runc escape)
- Named volumes > Host: UID/GID perm and initialization
- Published ports are open to the internet
- Reverse proxies are useful when running http microservices, publish one port, route by hostname/path/header
  - Nginx, Traefik, HAProxy

---

# Building Images

---

# Image Fundamentals

- Overlay Filesystem
  - Filesystem type in the Linux kernel
  - Merges one or more base layers with read-write layer

???

- Overlay filesystem takes one or more base layers, merges them together
- Each layer is a directory
- Add a work directory that is empty, just for overlay
- Read write upper layer captures changes, lower layers are read-only
- Read write layer could be added to list of lower layers in a future mount
- Each layer is a step of an image build

---

<asciinema-player src="demo-2-1-overlay.cast" cols=100 rows=26 preload=true font-size=16></asciinema-player>

???
---

# Image Fundamentals

- Layered Filesystem
- Base Image
  - scratch
  - Alpine
  - Debian, CentOS
  - distroless
  - Any other Image

???
- Scratch: empty filesystem, base of everything
  - No shell, libraries, etc
- Alpine is minimal
  - musl libc rather than glibc
- Debian, CentOS, etc ships minimal filesystem for those distros
  - Not a full install, stripped down
  - Not the kernel, that's shared
- Google distroless includes minimal filesystem for various languages
- Anything can be a base image, extend images from upstream

---

# Image Fundamentals

- Layered Filesystem
- Base Image
- Dockerfile Steps
  - FROM
  - RUN
  - COPY
  - CMD

???

- Dockerfile's are a list of steps
- Each step changes metadata or adds a filesystem layer
- FROM defines base image, all layers inherited
- RUN performs command in temp container, captures filesystem change
- COPY copies files from context to new layer
- CMD sets default command when container is run, only one

---

# Dockerfile

```no-highlight
FROM python:3

WORKDIR /app

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY . .

EXPOSE 80

ENV NAME=Docker User
CMD [ "gunicorn", "app:app", "-b", "0.0.0.0:80", \
      "--log-file", "-", "--access-logfile", "-", \
      "--workers", "4", "--reload", "--keep-alive", "0" ]
```

???
- FROM python: depend on something other than latest, semver
- WORKDIR is how you cd to a directory, affects all relative commands
- COPY a single file, or later the entire context
- RUN captures filesystem change with pip install
- EXPOSE is documentation
- ENV defines environment variables for commands, both RUN and CMD
- LABEL is metadata, useful documentation to users
- Why two COPY commands? Layer caching...

---

# Layer Caching

- Docker reuses cache of previous builds when possible
- Compares:
  - Previous layer
  - Command being run
  - Checksum on files/metadata

???

- Previous layer: once the cache misses, all following steps are uncached, order matters
- Command: RUN, this includes environment/args
- Checksum: COPY, this looks at file contents, permissions, ownership
- By copying requirements.txt separately, we only run pip install if requirements changes

---

# FROM

- Imports metadata and pointers to filesystem layers
- Additional lines after FROM extend the new image

???

- Never modify the parent image, only add new layers to define different metadata
- To change the base image layers, you need to pull or build base image itself

---

```no-highlight
$ cat Dockerfile
FROM busybox

$ docker image build -t mybusybox .
Sending build context to Docker daemon  24.06kB
Step 1/1 : FROM busybox:latest
 ---> 6d5fcfe5ff17
Successfully built 6d5fcfe5ff17
Successfully tagged mybusybox:latest

$ docker image ls | grep busybox
REPOSITORY     TAG       IMAGE ID        CREATED         SIZE
busybox        latest    6d5fcfe5ff17    2 months ago    1.22MB
mybusybox      latest    6d5fcfe5ff17    2 months ago    1.22MB
```
???

- If we make no changes, it's the same image id

---

# Exec vs Shell Syntax

```no-highlight
RUN [ "echo", "hello", "world" ]
RUN echo hello world
```

- Exec: json array
  - Executes binary directly with syscall
- Shell: any string
  - Expands environemnt variables
  - I/O redirection
  - Chaining commands

???

- Shell format is any string, docker will run with `/bin/sh -c "your string"`
- Shells give lots of features we like
- Exec directly runs the command, Linux exec syscall
  - Each arg is a separate array entry, often spaces in shell command
  - You can exec a shell script or any other command
- Shell's take pid 1, and block syscalls
- ENTRYPOINT+CMD syntax breaks when using shell
- What happens when you have a json parsing error?
  - A: it runs broken json string as a shell command

---

# ENTRYPOINT vs CMD

- Both are ways to sest the default command
- Override CMD: `docker run debian echo hello world`
- Override ENTRYPOINT: `docker run --entrypoint /bin/bash some_image`
- Both together concatenate into one command

```no-highlight
ENTRYPOINT [ "/bin/app", "--common", "option" ]
CMD [ "server", "--default", "arg" ]
```

???
- Individually, each option is the same except for how you override
- CMD is easy to override, ENTRYPOINT is often a script to parse CMD
- What command(s) does docker run when you start a container with the above overridden with a new CMD?

```no-highlight
$ docker container run yourimg client --custom arg
>>> /bin/app --common option client --custom arg
```
???

- How many commands will docker run to start your container? 1
- If one command / script runs lots of processes in the background and returns, what happens? Container exits

---
class: small

# ENTRYPOINT vs CMD

```no-highlight
ENTRYPOINT /bin/app --common option
CMD --default arg
```

```no-highlight
$ docker container run yourimg
>>> /bin/sh -c "/bin/app --common option" /bin/sh -c "--default arg"
```

???
- If you use the string syntax, you get this command executed in the container
- `/bin/sh -c` only takes a single arg, everything else is `$*` passed to that arg string

--
```no-highlight
/bin/sh -c "echo input is $1 $2" hello world
```

???
- Valid syntax would look like `/bin/sh -c "echo $1 $2" hello world`

---
class: small

# ENTRYPOINT vs CMD

```no-highlight
ENTRYPOINT ["/bin/app", '--common', 'option' ]
CMD [ "--default"  "arg" }
```

```no-highlight
$ docker container run yourimg
>>> /bin/sh -c "[\"/bin/app\", '--common', 'option' ]" /bin/sh -c \
    "[ \"--default\"  \"arg\" }"

docker: Error response from daemon: OCI runtime create failed:
  container_linux.go:349: starting container process caused
  "exec: \"[/bin/app,\": stat [/bin/app,: no such file or directory": unknown.
```

???

- Any error parsing the json, results in running string with a shell

---

# ENV

- Environment variable is defined in the image
- Expanded by docker in:
  - COPY / ADD
  - ENV
  - FROM
  - LABEL
- Affects any RUN commands, and containers

???

- RUN only captures changes to the filesystem, if you want to set an env var, you must tell docker
- Also expanded in:
  - EXPOSE
  - STOPSIGNAL
  - USER
  - VOLUME
  - WORKDIR
- Note: docker does not expand variable in run, only injects variable and shell in container expands it
  - Exec syntax does not expand

---

# Build ARGs

- ARG behaves identical to ENV during build
- ARG is scoped
  - Global
  - Current stage
- Can be overridden at build time

```no-highlight
docker image build --build-arg APP_VER=5.0 -t app:5.0 .
```

---

# ENV vs ARG

???

- How do you pick?

---

# ENV vs ARG

```no-highlight
ARG APP_VER=5.0
ENV APP_VER=${APP_VER}
RUN installer.sh -v ${APP_VER}
```

???

- By doing both, we can change a variable at build time, and still have it when running the container
- If you don't need to change it at build time, but want it when container is run, use ENV
- If you only want it during build, use ARG

---

# Demo / Lab

- Create the mobysay image
- Based on "debian:10"
- Set arg "DEBIAN_FRONTEND=noninteractive"
- Run `apt-get update` and `apt-get install cowsay`
- Copy in moby.cow to `/usr/share/cowsay/cows/`
- Update PATH to include `/usr/games`
- When run, execute `cowsay -f moby.cow Whale hello there` where the message can change

???

- Give group a few minutes to work on lab

---

<asciinema-player src="demo-p1-example.cast" cols=100 rows=26 preload=true font-size=16></asciinema-player>

???
- mobysay image
- We'll go over Dockerfile in detail next
- Show build, run, and run with a different command

---

# Dockerfile

```no-highlight
FROM debian:10

ARG DEBIAN_FRONTEND=noninteractive
RUN apt-get update \
 && apt-get install -y --no-install-recommends cowsay

COPY moby.cow /usr/share/cowsay/cows/

ENV PATH=$PATH:/usr/games

ENTRYPOINT [ "cowsay", "-f", "moby.cow" ]
CMD [ "Whale hello there" ]
```

???

- `debian:10` binds to a major version but allows patches
- `ARG` ensures that apt-get doesn't prompt while not affecting users in interactive containers later
- `RUN` is chained: ensures both commands run together, why?
  - If separate, an update could cache the apt repos, and a later install could pull stale or missing packages
- Where does `$PATH` come from?
  - From the parent image, set in `debian:10`, not from user building container
- `ENTRYPOINT` uses the json syntax so we can merge `CMD`
- `CMD` allows message to be easily overridden

---

# ADD vs COPY and the Build Context

- ADD will expand tar files, with various compression
- ADD will pull from URL's, and track if those URL's have changed
- When source is a directory, contents of the directory are copied
- Both will copy from the build context
- Neither will copy files from outside of the build context

```no-highlight
docker image build -t img_name .
```

???

- ADD has more features, and should only be used when those features are needed
- If you copy multiple directories, the contents of those directories are all merged into image target dir
- The build context is the last arg to build, often `.` aka the current directory
- You cannot pull files from the parent directory

---

# EXPOSE

- Documentation from image builder to user
- Indicates ports the container should listen
- Does not impact ability to publish ports
- Not needed for container networking

???

- Configuration of the listening port could be changed in app
- Docker doesn't control what port app listens on

---

# WORKDIR

- A `cd` inside a `RUN` command is lost
- Use `WORKDIR` to define the current directory
- Applies to `RUN`, containers, `COPY`, and `ADD`

???

- RUN only captures changes to the filesystem, `cd` only affects current shell
- WORKDIR is like a persistent `cd`, even affects containers

---

# USER

- Do not use `sudo` with containers
- `USER` defines a user and group for future containers (including `RUN`)

```no-highlight
USER root
RUN useradd -u 5000 app
USER app:app
```

???

- Sudo would allow privilege escalation
- User needs to exist inside of container
- Good best practice to not run containers as root (or use user namespaces)
- You can `docker container exec -u` to run commands as another user

---

# VOLUME

- Forces a volume at that path on all future containers
  - Results in anonymous volumes
- Breaks ability to modify directory with RUN
  - Breaks downstream images
- Not needed to define a volume later
  - Best done with `docker-compose.yml`

???

- VOLUME has more bad features than good, avoid
- The volume is forced, but if the user doesn't define the volume, you get an anonymous volume
- Long guid, without any indication of source container/image
- During RUN, temp volume is created, initialized with image contents, changes apply in volume, and image layer is only the changes to the container filesystem (note buildkit behaves differently)
- Users are best off with docker-compose.yml to allow them to extend image (initialize DB)
- Can also create entrypoint to initialize volume from another image directory

---

# Multi-Stage Builds

```no-highlight
FROM redis
FROM nginx
FROM php
```

???

- What would happen if this was built?
  - You'd get a php image
- Multi-stage builds have multiple FROM lines
- Each from defines a new image being built, a new stage, separate from the previous stage
- Multi-stage does not merge images
- So what is multi-stage good for? ...

---

# Multi-Stage Builds

```no-highlight
FROM maven:latest
COPY src/ /usr/src/app/
RUN mvn package
CMD jre /usr/src/app/target/app.jar
```

???

- What happens when we compile our app inside the container?
  - We ship the entire source code, compiler, and compiled code to prod

---

# Multi-Stage Builds

```no-highlight
ARG JAVA_VER=11
# stage 1: build
FROM maven:3-jdk-${JAVA_VER} as mvn
COPY src/ /usr/src/app/
RUN mvn package -Dmaven.test.skip=true
CMD jre /usr/src/app/target/app.jar

# stage 2: test
FROM mvn as test
RUN mvn test

# stage 3: package
FROM openjre:${JAVA_VER}-jre as release
COPY --from=mvn /usr/src/app/target/app.jar /usr/src/app/target/app.jar
CMD jre /usr/src/app/target/app.jar
```

???

- Multi-stage allows us to build multiple images but only ship one
- Can copy files from previous image
- Full featured compile in docker + minimal image to ship
- `FROM` has "as" to give stage an alias
- `FROM` can also be the result of another stage
- `COPY --from` can pull from previous stages or other images

---

# Developer Workflow

- Run app inside a container with a volume mount to code
- Configure app to dynamically update with code change for interpreted languages
- Run a compile as entrypoint for compiled languages

```no-highlight
docker container run -d -p 8080:80 --name flask \
  -v "$(pwd):/app:ro" flask-app
```

???

- Do developers need to build a new image for every change? No.
- With multi-stage builds, developers often run a build stage rather than a release stage
- Flask is a python based web server framework, has a development mode to load changes
- The `-v` gives us a volume mount
- We need the full path, so we run `$(pwd)` to get the current directory
- What does `:ro` do? Makes the volume read-only inside the container, only make changes from the host

---

<asciinema-player src="demo-p1-example.cast" cols=100 rows=26 preload=true font-size=16></asciinema-player>

???
- Flask app developer workflow

---

# I was promised a ship

???

- We did run
- Now we did build
- What about the ship?
- Note: Docker's slogan changed to Build, Share, Run. Ships make better slides.

---

# Registry

- Artifact repository for docker images
- Docker Hub, GCR, ECR, quay.io, etc 
- Registry container, Harbor, Artifactory, Nexus
- Based on image tag, push and pull commands
- Only push and pull new layers

```no-highlight
$ docker image build -t my-app:1.0 .
$ docker image tag my-app:1.0 registry.example.com:5000/my-app:1.0
$ docker image push registry.example.com:5000/my-app:1.0
$ docker image pull registry.example.com:5000/my-app:1.0
```

???

- Registry is an open spec from docker, lots of implementations
  - Some cloud/SaaS based, others are self hosted
  - Harbor is best featured self hosted option, CNCF
- This is how we move images from CI servers to dev through prod
- Docker knows what registry based on the image tag, prefix with host:port
  - Hub is default, only needs your username
- You can save/load images, but this includes every layer, lose sharing/efficiency

---

# Orchestration

---

# Orchestration

- Manages containers
- Quorum High Availability
- Declarative vs Imperative
- Networking handles replicas and multiple nodes

???

- Running one container is easy, 100's is error prone
- Raft is the preferred HA quorum model
  - Requires a majority of managers to be available
  - If you have 2 nodes and 1 goes down, do you have a majority? No
  - Quorum: "I can't even", 3 allows 1, 5 allows 2
  - More managers increases availability but decreases throughput, wait for each manager to sync
- Declarative: this is my target state, "make it so"
- Networking:
  - RR container to container when there are multiple replicas
  - Publishing a port needs to RR to those replicas too
  - Networks need to span multiple hosts, overlay networking

---

# Recap

---

# Recap

- Buffer Overflow

---

# Thank You

github.com/sudo-bmitch/presentations

.content[
.left-column[
.pic-80[![Slides QR](img/github-qr.png)]
]
.right-column[.align-right[.no-bullets[
<br><br>
- Brandon Mitchell
- Twitter: @sudo_bmitch
- GitHub: sudo-bmitch
]]]
]
???
- This presentation is online, you can watch it again