Docs / InfraDocker · Containers

Docker & Containers

Containers are the unit of reproducibility for ML systems. They package the model, runtime, system libraries, and inference contract into a single immutable image that behaves identically across laptops, CI runners, and production clusters.

Engine

Docker / containerd

Format

OCI image spec

Runtime

runc / crun

Orchestration

Compose · K8s

Images

Layered, content-addressed artifacts

A Docker image is an immutable bundle of read-only layers plus a manifest. Each instruction in a Dockerfile produces a cacheable layer identified by a SHA-256 digest. Containers are runtime instances of an image with a thin writable layer on top.

For ML, use multi-stage builds to separate heavy build dependencies (CUDA toolkit, compilers) from the slim runtime layer that actually ships. This keeps inference images small, fast to pull, and free of attack surface.

Pin base images to a digest, never :latest.
Order layers from least-changing to most-changing to maximize cache reuse.
Run as a non-root user; drop capabilities you don't need.
Use BuildKit cache mounts for pip / apt.

Dockerfile · ML inference

# syntax=docker/dockerfile:1.7
FROM python:3.11-slim AS builder
WORKDIR /build
COPY requirements.lock .
RUN --mount=type=cache,target=/root/.cache/pip \
    pip install -r requirements.lock

FROM python:3.11-slim AS runtime
RUN useradd -m -u 10001 app
WORKDIR /app
COPY --from=builder /usr/local/lib/python3.11 \
     /usr/local/lib/python3.11
COPY ./model ./model
COPY ./server ./server
USER app
EXPOSE 8080
CMD ["python", "-m", "server.handler"]

Compose

Multi-service local stacks

Docker Compose declares multi-container stacks in a single YAML file. For ML, it's the fastest way to wire a model server with its feature store, vector DB, observability sidecar, and a local Postgres for metadata — all reproducibly on a developer laptop.

depends_on with healthchecks for ordered startup.
Profiles to gate optional services (e.g. --profile gpu).
Per-service resource limits to mimic prod constraints.
Use .env files; never commit secrets.

compose.yaml · inference stack

services:
  model:
    image: aegis/model:v4.2.1
    ports: ["8080:8080"]
    environment:
      FEATURE_STORE_URL: http://features:6566
    depends_on:
      features:  { condition: service_healthy }
      registry:  { condition: service_started }
    networks: [serving]
    volumes:
      - model-cache:/app/.cache

  features:
    image: feastdev/feature-server:0.40
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:6566/health"]
      interval: 5s
    networks: [serving, data]

  registry:
    image: ghcr.io/mlflow/mlflow:v2.16
    command: mlflow server --host 0.0.0.0
    networks: [data]

  prometheus:
    image: prom/prometheus:v2.54
    volumes: ["./prom.yml:/etc/prometheus/prometheus.yml:ro"]
    networks: [serving]

networks:
  serving: {}
  data:    {}

volumes:
  model-cache: {}

Networking

Service discovery & isolation

bridge

Default user-defined network. DNS-based service discovery — reach peers by service name.

host

Shares the host network stack. Lowest latency, no port mapping — Linux only.

overlay

Spans multiple hosts in Swarm / multi-node setups. Encrypted control plane.

none

No connectivity. Useful for batch jobs that must never reach the network.

Inside a user-defined bridge network, containers resolve each other by name via Docker's embedded DNS. The model service reaches features:6566 without any hard-coded IPs — exactly the pattern Kubernetes mirrors with its DNS service.

Split your stack into isolated networks (e.g. serving for the public path, data for backend systems). Only attach services to the networks they actually need — least privilege at the network layer.

$ docker network create --driver bridge serving
$ docker network inspect serving --format '{{ json .Containers }}'

Volumes

Persistent & shared state

Named volume

Docker-managed storage. Survives container restarts. Preferred for DB data, model caches, metric stores.

volumes:
  - model-cache:/app/.cache

Bind mount

Maps a host path into the container. Great for live-reloading source during local dev — avoid in prod.

volumes:
  - ./src:/app/src:ro

tmpfs

In-memory filesystem. Use for secrets at runtime or scratch space that should never hit disk.

tmpfs:
  - /run/secrets:size=8m

ML-specific guidance: mount the model-weights cache as a named volume so containers don't re-download multi-gigabyte checkpoints on every restart. Mount feature parquet datasets read-only. Never bake large datasets into the image — pull them at start time from object storage and cache on a volume.

Back up named volumes with docker run --rm -v vol:/data -v $PWD:/backup alpine tar czf /backup/vol.tgz /data. Don't rely on bind mounts in production — host filesystem layout becomes a deployment dependency.

Best Practices

Production-grade containers

Pin to digest

Use image@sha256:… in prod. Tags are mutable; digests aren't.

Smallest viable base

distroless or python:slim. Less surface, faster pulls.

One process per container

Let the orchestrator handle restarts and scaling.

Healthchecks always

HTTP /healthz + /ready. Orchestrators need both signals.

Read-only root FS

Set read_only: true; write only to declared volumes.

Drop capabilities

cap_drop: [ALL] then add back only what's needed.

← Toolchain Phase 01 · Build & Package →