DevOps Hub/Docker & K8s
All systems operational
Track 5 • Cloud-Native Infrastructure & Orchestration

Docker Containers & Kubernetes Architecture

Master production multi-stage container optimization, Docker Compose multi-service topology networking, Kubernetes control plane architecture, and Helm & ArgoCD GitOps continuous deployment.

Module 1 • Image Engineering & Security

Multi-Stage Dockerfile Builder & Optimization Inspector

Base Image:node:20-alpine
Multi-Stage Build
Separate build tools from lightweight runtime
Layer Cache Strategy
Copy package.json before source code
Non-Root Execution User
Run container process as unprivileged user
Prune DevDependencies
Strip build tools, tests, and dev binaries
Final Image Size
120 MB
-92% reduced
CVE Vulnerabilities
2 CVEs
Trivy scan report
Rebuild Time
4s
Cache Hit ⚡
Security Grade
Grade A
CIS Benchmark
# MULTI-STAGE OPTIMIZED PRODUCTION DOCKERFILE # Stage 1: Build Dependencies & Asset Compilation FROM node:20-alpine AS builder WORKDIR /app # Optimize Layer Caching: Copy package manifests first COPY package*.json tsconfig*.json ./ RUN npm ci COPY . . RUN npm run build # Stage 2: Minimal Secure Runtime Environment FROM node:20-alpine AS runner WORKDIR /app ENV NODE_ENV=production RUN addgroup --system --gid 1001 nodejs && adduser --system --uid 1001 nextjs # Copy isolated production node_modules from builder COPY --from=builder /app/package*.json ./ RUN npm ci --only=production && npm cache clean --force # Copy compiled build output with explicit non-root ownership COPY --from=builder --chown=nextjs:nodejs /app/dist ./dist EXPOSE 3000 USER nextjs CMD ["node", "dist/index.js"]
Module 2 • Multi-Container Orchestration

Docker Compose Service Stack Generator & Topology Engine

nginx-proxy(80:80)
frontend-net
Image: nginx:1.25-alpine
nextjs-app(3000:3000)
frontend-net
Image: company/nextjs-frontend:v1.4
backend-api(8080:8080)
both
Image: company/backend-api:v2.1
postgres-db(5432:5432)
backend-net
Image: postgres:16-alpineVol: pgdata
redis-cache(6379:6379)
backend-net
Image: redis:7-alpineVol: redisdata
Enable Healthcheck Conditions
wait for DB healthy before app start
Interactive Network Isolation Topology2 Bridge Networks Active
🌐 frontend-net (Bridge)Public Facing
nginx-proxy80:80
nextjs-app3000:3000
backend-api8080:8080
🔒 backend-net (Isolated)Internal Only
backend-api8080:8080
postgres-dbVol: pgdata
redis-cacheVol: redisdata
docker-compose.yml
version: "3.8" services: nginx-proxy: image: nginx:1.25-alpine container_name: nginx-proxy ports: - "80:80" networks: - frontend-net depends_on: - nextjs-app - backend-api nextjs-app: image: company/nextjs-frontend:v1.4 container_name: nextjs-app ports: - "3000:3000" environment: - NODE_ENV=production - API_URL=http://backend-api:8080 networks: - frontend-net depends_on: - backend-api backend-api: image: company/backend-api:v2.1 container_name: backend-api ports: - "8080:8080" environment: - DB_HOST=postgres-db - REDIS_HOST=redis-cache networks: - frontend-net - backend-net depends_on: postgres-db: condition: service_healthy redis-cache: condition: service_healthy postgres-db: image: postgres:16-alpine container_name: postgres-db ports: - "5432:5432" environment: - POSTGRES_DB=prod_db - POSTGRES_USER=db_user - POSTGRES_PASSWORD=secret_pass volumes: - pgdata:/var/lib/postgresql/data networks: - backend-net healthcheck: test: ["CMD-SHELL", "pg_isready -U db_user -d prod_db"] interval: 10s timeout: 5s retries: 5 redis-cache: image: redis:7-alpine container_name: redis-cache ports: - "6379:6379" volumes: - redisdata:/var/lib/redis networks: - backend-net healthcheck: test: ["CMD", "redis-cli", "ping"] interval: 5s timeout: 3s retries: 3 networks: frontend-net: driver: bridge backend-net: driver: bridge internal: true # Isolated from public internet volumes: pgdata: driver: local redisdata: driver: local
Module 3 • Production Kubernetes Cluster Mechanics

Kubernetes Control Plane vs Worker Node Architecture

Routing Dataplane:
🧠

Control Plane (Master Node)

Global cluster state, scheduling & API management

Active Cluster Leader
⚙️

Worker Nodes (Node 01..03)

Runs application pods, container runtime & networking

3 Nodes Online
kube-apiserverControl Plane
Port / Spec: 6443 / TCP (HTTPS / REST)

Central API gateway and orchestration engine. Validates and processes all REST requests from kubectl, controllers, and kubelets.

HA & Redundancy Strategy:Active-Active behind external Load Balancer (stateless).
Failure Impact:Cluster management frozen; kubectl commands fail, but running Pods continue operating.

Ingress ➔ Kubernetes Service ➔ Pod Endpoints Traffic Flow

Simulate live HTTP ingress packet decapsulation and load balancing

Pod 01 (pod-web-1)
10.244.1.15:8080
0 Requests
Pod 02 (pod-web-2)
10.244.2.45:8080
0 Requests
Pod 03 (pod-web-3)
10.244.3.75:8080
0 Requests
Module 4 • Continuous Deployment & GitOps

Helm Charts Packaging & ArgoCD GitOps Sync Visualizer

Sync State:● Synced
1. Helm Chart values.yaml Parameters
replicaCount3 Pods
ingress.enabled
Provision NGINX Ingress rules
# HELM VALUES.YAML replicaCount: 3 image: repository: registry.company.io/web-service pullPolicy: IfNotPresent tag: "v1.2.0" resources: limits: cpu: "500m" memory: "512Mi" requests: cpu: "100m" memory: "128Mi" ingress: enabled: true className: "nginx" annotations: cert-manager.io/cluster-issuer: "letsencrypt-prod" hosts: - host: app.company.com paths: - path: / pathType: Prefix
2. ArgoCD GitOps Continuous Reconciliation Dashboard
🐙 Git Source RepoSHA: a4f9b2c
feat: bump image tag to v1.2.0
Target Tag: v1.2.0
☸️ Live K8s ClusterSHA: a4f9b2c
Replicas: 3 active pods
Health: Healthy
ArgoCD GitOps Live Unified Diff ViewerIn Sync (0 diffs)
✓ Desired Git State matches Live Cluster State perfectly. No drift detected.
DOCKER · CONTAINER SECURITY

Trivy Scanner Simulator — CVE Hunt Inside Your Image

Pick an image, run a simulated trivy image scan, and watch the vulnerability report build up: severity counts, per-package fixes, an SBOM (CycloneDX) preview, and an exportable JSON report.

Scanner Console

Select an Image Target & Scan

ready
$ trivy image --format table --severity CRITICAL,HIGH,MEDIUM,LOW,UNKNOWN catalog-service:1.2.3
awaiting target…
Track 5 • Cloud-Native Infrastructure & Orchestration

Kubernetes Resource Quotas & Limits

Master CPU & memory request/limit math, namespace-wide ResourceQuota enforcement, LimitRange defaults, QoS class mechanics (Guaranteed · Burstable · BestEffort), and efficiency metrics that keep costs and evictions in check.

Module 1 • Scheduling Math

CPU & Memory Requests/Limits Calculator

Node:
CPU (millicores)
250m

Guaranteed minimum for scheduling

500m

Burst ceiling before CPU throttling

Memory
512Mi

Reserved on nodes at placement

1Gi

OOM-kill threshold

CPU vs Nodereq 250mc / lim 500mc

Spare node headroom: 7.75 cores

Memory vs Nodereq 512Mi / lim 1Gi

Spare node headroom: 31.5Gi

limit/request ratio
2×
CPU burst headroom
250m
Memory burst headroom
512Mi
Est. pods / node
32
resources: requests: cpu: "250m" memory: "512Mi" limits: cpu: "500m" memory: "1Gi"
✅ Schedulable on m5.2xlarge: request 250m / 512Mi · limit 500m / 1Gi
Module 2 • Namespace Guardrails

ResourceQuota Builder

Compute resources
Other resources
Scope

Scope restricts the quota to pods matching that QoS class (BestEffort / NotBestEffort) or the whole namespace (Default).

resourcequota.yaml
apiVersion: v1
kind: ResourceQuota
metadata:
  name: compute-quota
  namespace: default
spec:
  hard:
    pods: 20
    requests.cpu: "2"
    limits.cpu: "4"
    requests.memory: "4Gi"
    limits.memory: "8Gi"
Enforcement rules

• Admission is rejected when the pod's declared request/limit would exceed the remaining quota for that resource.

• Resources counted: requests.* and limits.* are summed across all containers in the pod.

• If a namespace has a quota for requests.cpu, every pod must declare it (ingress) or be rejected.

Module 3 • Defaults & Bounds

LimitRange Configurator

CPU (millicores)
50m

Lowest allowed request/limit

100m

Injected when request omitted

250m

Injected when limit omitted

4000m

Hard ceiling per container

Memory (Mi)
64Mi

Lowest allowed request/limit

128Mi

Injected when request omitted

512Mi

Injected when limit omitted

8Gi

Hard ceiling per container

maxLimitRequestRatio
4
2
limitrange.yaml
apiVersion: v1
kind: LimitRange
metadata:
  name: container-limits
  namespace: default
spec:
  limits:
    - max:
        cpu: "4000m"
        memory: "8192Mi"
      min:
        cpu: "50m"
        memory: "64Mi"
      default:
        cpu: "250m"
        memory: "512Mi"
      defaultRequest:
        cpu: "100m"
        memory: "128Mi"
      maxLimitRequestRatio:
        cpu: "4"
        memory: "2"
      type: Container
How LimitRange applies

• When a container omits a limit, the namespace default is injected; when it omits a request, defaultRequest is injected.

• Containers declaring values outside [min, max] are rejected at admission.

• maxLimitRequestRatio caps how far a container's limit may exceed its request for the same resource.

Module 4 • Pod Priority Tiers

QoS Class Calculator

cpu (m)
memory (Mi)
cpu (m)
memory (Mi)
QoS: Burstable

At least one container declares a request or limit, but the Guaranteed criteria are not fully met. Middle tier: can burst, may be evicted when pressure hits.

Decision rules

1. No requests or limits anywhere → BestEffort.

2. Every container sets CPU request = CPU limit AND memory request = memory limit → Guaranteed.

3. Anything else with at least one request/limit → Burstable.

pod.yaml
apiVersion: v1
kind: Pod
metadata:
  name: qos-demo
  namespace: default
spec:
  containers:
    - name: api-server
      resources:
        requests:
          cpu: "250m"
          memory: "256Mi"
        limits:
          cpu: "500m"
          memory: "512Mi"
    - name: cache
      # no requests/limits — contributes to BestEffort
  restartPolicy: Always
# Pod QoS class: Burstable
Module 5 • Cost & Capacity

Resource Efficiency Metrics

Node size:
Workload shape (per pod)
4
500m
1c
512Mi
1Gi
Measured utilization (% of request)
35%

From metrics: sum(rate(container_cpu_usage_seconds_total))/requests

55%

From metrics: container_memory_working_set_bytes vs requests

Efficiency score
55/100
Total CPU request
2c
CPU overcommit
2×
Mem overcommit
2×
CPU fleet (request vs limit)req 2c / lim 4c

Spare node headroom: 6 cores

Memory fleetreq 2Gi / lim 4Gi

Spare node headroom: 30Gi

CPU in use
0.7c
CPU waste
1.3c
Mem in use
1126.4Mi
Pods fit / node
16
✓Requests track actual utilization — steady-state efficiency looks healthy.
K8s · Network Policies

Kubernetes Network Policy Visual Builder 🛡️

Build ingress/egress allowlists with pod, namespace and CIDR selectors — then preview the traffic flow, generated YAML, and a live allow/deny simulator.

🛡️ Ingress / Egress Rule Builder

4 active
IN
Peers (any match)
pod
namespace
port
IN
Peers (any match)
namespace
port
OUT
Peers (any match)
namespace
port
OUT
Peers (any match)
cidr
port

🧪 Policy Effect Simulator

📡 Visual Traffic Flow

● allowed● denied (default)→ rule direction
INGRESS→ protected pod
app=frontend→80/TCPALLOW
app=payments→80/TCPALLOW
role=monitoring→9090/TCPALLOW
unmatched traffic→any portDENY
Pod(s)
web-api-allowlist
app=web, tier=frontend
ns: prod
status: 🔒 deny-by-default
EGRESSprotected pod →
3306/TCP→app=dbALLOW
443/TCP→203.0.113.0/24ALLOW
any port→unmatched trafficDENY
How to read: arrows show the direction a packet travels. ALLOW rows come from enabled rules; the unmatched traffic row shows the default-deny verdict when the namespace is isolated. Kubernetes NetworkPolicies are allowlist-only — rules never explicitly "deny", the isolation default does.

📄 Generated YAML

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: web-api-allowlist
  namespace: prod
spec:
  podSelector:
    matchLabels:
      app: web
      tier: frontend
  policyTypes: [Ingress, Egress]
  ingress:
    # rule: allow-frontend-web
    - from:
        - podSelector:
            matchLabels:
              app: frontend
        - namespaceSelector:
            matchLabels:
              app: payments
      ports:
        - port: 80
          protocol: TCP
    # rule: allow-monitoring
    - from:
        - namespaceSelector:
            matchLabels:
              role: monitoring
      ports:
        - port: 9090
          protocol: TCP
  egress:
    # rule: allow-db-out
    - to:
        - namespaceSelector:
            matchLabels:
              app: db
      ports:
        - port: 3306
          protocol: TCP
    # rule: allow-payments-lb
    - to:
        - ipBlock:
            cidr: 203.0.113.0/24
      ports:
        - port: 443
          protocol: TCP

🔒 Namespace Isolation Mode

policyTypes
[Ingress, Egress] → default-deny both directions
Track 5 • Kubernetes Edge Routing & Service Mesh

Kubernetes Ingress & Service Mesh

Route external traffic into the cluster with NGINX Ingress, then go deeper: path-based routing builders, TLS termination, mutual-TLS mesh identity, canary weight splitting and traffic mirroring with Istio. Compare the two control planes side by side.

NGINX vs IstioPath RoutingTLS TerminationmTLS MeshCanary WeightsTraffic Mirroring
Module 1 • Control Plane Comparison

NGINX Ingress Controller vs Istio Service Mesh

Dimension
NGINX Ingress
Istio Mesh
L7 North-South Edge Routing
Both terminate TLS at the edge; NGINX is the battle-tested default for plain Ingress.
Native Ingress controller: host-based rules, path prefixes, TLS at the edge
92/100
Routed through Istio IngressGateway (Envoy) — also exposed as a Gateway API
70/100
mTLS & Workload Identity
The core reason teams adopt a mesh: encrypted east-west traffic with zero app changes.
No mesh identity — services must handle their own certs, if at all
10/100
Automatic mutual TLS with SPIFFE workload certs rotated by istiod (PERMISSIVE/STRICT)
98/100
Traffic Splitting & Weighted Canary
NGINX canaries are per-Ingress (one split); Istio splits across the whole mesh per-service.
Header / cookie canary annotations + canary-weight distribution between two backends
55/100
Weighted route subsets, header conditions, A/B splits, fault injection, retries, timeouts
95/100
Traffic Mirroring (Shadowing)
Istio's mirrorWeight makes shadowing precise; NGINX mirrors blindly at request level.
mirror-* annotations copy requests to a shadow backend (no percentage control)
45/100
mirror + mirrorWeight on any VirtualService route — exact % control, optional response discard
90/100
TLS Termination Flexibility
NGINX is the workhorse for classic certs; Istio adds SNI-aware routing and mesh-native certs.
Ingress tls block, cert-manager annotations, custom TLS protocols & proxy protocols
90/100
TLS termination at gateway + per-node deeper: SNI routing, mTLS passthrough options
78/100
Retries, Fault Injection, Timeouts
Chaos-engineering tools only exist in the mesh data plane.
Annotations for simple retries/timeouts; no fault injection simulation
30/100
First-class HTTP fault injection (abort/delay), per-route retries and timeout policies
88/100
East-West (Pod-to-Pod) Routing
If you only need edge routing, NGINX is enough; east-west traffic requires a mesh.
Not applicable — Ingress only handles external traffic
5/100
Service entries, headless service routing, locality load balancing, DNS proxying
90/100
Operational Complexity
NGINX 'just works'; Istio needs a mesh-operations discipline to run safely.
One Deployment + ConfigMap; simple, few moving parts, huge community
85/100
istiod control plane, sidecar injection, RBAC, mTLS policies — real learning curve
40/100
Module 2 • Edge Traffic Router

Ingress Resource — Path-Based Routing Builder

Host:
Backend:
Backend:
Backend:
Class:
Request simulation
ingress.yaml (generated live)
apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: app-ingress annotations: nginx.ingress.kubernetes.io/ssl-redirect: "true" spec: ingressClassName: nginx tls: - hosts: - app.example.com secretName: app-tls rules: - host: app.example.com http: paths: - path: / pathType: Prefix backend: service: name: web-svc port: number: 80 - path: /api pathType: Prefix backend: service: name: api-svc port: number: 8080 - path: /api/v1/orders pathType: Exact backend: service: name: orders-svc port: number: 3000
How NGINX picks the backend
1. Host match → 2. most specific segment-aware prefix match (or Exact) → 3. Service port → EndpointSlice load balancing.
Precedence: Exact rules beat Prefix; among Prefix rules, the longest path wins — so /api/v1/orders beats /api.
Module 3 • Edge Cryptography

TLS Termination Configuration

tls block + annotations → controller behavior
Redirect HTTP → HTTPS
ssl-redirect / force-ssl-redirect annotations
Minimum TLS version
pin the handshake floor (TLSv1.2+ recommended)
Backend transport
http = plaintext pod-to-pod (cluster net)
Request flow — TLS terminates at the ingress edgeminSsl TLSv1.2
Client
HTTPS :443
TLS 1.2 ↦
Ingress Controller
terminates TLS here
cert-manager auto-issue + renew
301 → https://app.example.com
Service (VIP)
podIP routing
cluster-internal
Pod (web-svc)
plain HTTP :80
trust cluster network
ingress.yaml — TLS block + annotations
apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: app-ingress annotations: cert-manager.io/cluster-issuer: "letsencrypt-prod" nginx.ingress.kubernetes.io/ssl-redirect: "true" nginx.ingress.kubernetes.io/force-ssl-redirect: "true" nginx.ingress.kubernetes.io/ssl-protocols: "TLSv1.2 TLSv1.2" spec: ingressClassName: nginx tls: - hosts: - app.example.com secretName: app-tls rules: - host: app.example.com http: paths: - path: / pathType: Prefix backend: service: name: web-svc port: number: 443 # cert-manager watches this Ingress and issues the certificate # -- validated with DNS-01 / HTTP-01 challenge, auto-renewed before expiry (Let's Encrypt: 90 days)
Module 4 • Mesh Security

mTLS Service Mesh Visualization

Imagined mesh — prod namespace (sidecars injected)Automatic mTLS — REQUIRED
istio-ingressgateway
Public :443
tls Secret
mutual TLS
frontend-svc
sa/frontend
SPIFFE://cluster.local/ns/prod/sa/frontend
mutual TLS
checkout-svc
svc/checkout
SPIFFE://cluster.local/ns/prod/sa/checkout
mutual TLS
payments-svc
svc/payments
SPIFFE://cluster.local/ns/prod/sa/payments
Sidecar handshake (Envoy → Envoy)
[1] Workload certs signed by mesh root (istiod); presented on EVERY request from the sidecar
[2] Peer verifies cert against mesh CA — cert SAN carries SPIFFE identity
[3] Short-lived certs (24h) rotate automatically; no app code changes needed
PeerAuthentication + DestinationRule
apiVersion: security.istio.io/v1beta1 kind: PeerAuthentication metadata: name: default namespace: prod spec: mtls: mode: STRICT --- # Scoped to a specific workload instead? Use selector: # selector: # matchLabels: # app: checkout # And pair with DestinationRule to REQUIRE the client side: apiVersion: networking.istio.io/v1beta1 kind: DestinationRule metadata: name: checkout-dr namespace: prod spec: host: checkout-svc.prod.svc.cluster.local trafficPolicy: tls: mode: ISTIO_MUTUAL
Why mTLS matters
With sidecars, every workload gets a SPIFFE identity (k8s service account). istiod signs 24-hour workload certificates, so:
1. Traffic is encrypted pod-to-pod — the cluster network is no longer trusted.
2. Replay/middlebox attacks fail: a client must present a cert bound to its identity.
3. STRICT rejects legacy clients until they adopt mTLS — migrate with PERMISSIVE first.
Module 5 • Progressive Delivery

Canary Deployment Weights

Istio VirtualService weights • NGINX canary annotations
Canary traffic weight10%
0% (smoke)10% (safe)25% (bold)50% (A/B cap)
Header-based split
x-canary: "true" always goes to canary
Live split
stable v1 — 90%canary v2 — 10%
No requests yet — click to observe weighting.
frontend-vs.yaml — weighted split (VirtualService)
apiVersion: networking.istio.io/v1beta1 kind: VirtualService metadata: name: frontend-vs namespace: prod spec: hosts: - frontend-svc http: - route: - destination: host: frontend-svc subset: stable weight: 90 - destination: host: frontend-svc subset: canary weight: 10
Module 6 • Shadow Traffic

Traffic Mirroring (Shadowing)

copy live requests to a canary/shadow without affecting users
Enable mirroring
shadow traffic fires-and-forgets
Mirror weight (Istio mirrorWeight)50%
Shadow log empty — simulate a request to see mirrored copies.
Request path with mirror
Client
→
Envoy (checkout-svc)
route weight 50%
→ checkout-svc version v1 (real request)
⤷ shadow copy: version shadow (fire-and-forget, no client response)
checkout-vs.yaml — mirror + mirrorWeight
apiVersion: networking.istio.io/v1beta1 kind: VirtualService metadata: name: checkout-vs namespace: prod spec: hosts: - checkout-svc http: - route: - destination: host: checkout-svc subset: v1 mirror: host: checkout-svc subset: shadow mirrorWeight: 50
Mirroring vs Canary: mirrored copies never see the client response — they validate requests, replay, or warm caches. Canaries answer real traffic with a percentage split. Both are used together for risk-free releases.
Track 5 • Cloud-Native Storage on Kubernetes

Persistent Volumes & State on EKS

Interactive lab for the PV/PVC contract: watch volumes travel from Pending to Bound (and beyond), compare EBS and EFS StorageClasses, check which access mode fits your workload, choose a reclaim strategy, right-size capacity — then snapshot and restore.

Module 1 • PV/PVC Lifecycle Visualizer

Pending → Bound → Deleted → (what happened to my data?)

PVC Created
→
Provisioning
→
PV Bound
→
Pod Mount
→
PVC Deleted
→
Reclaim…
PVC Phase
Pending
PV Phase
Available
Storage
EBS gp3
gp3 (General Purpose SSD)
Reclaim
Retain
kube-controller-manager event log
— awaiting simulation steps —
what you are watching
  • • PVC = a claim; PV = the actual volume (EBS volume or EFS file system).
  • • The StorageClass provisioner (ebs.csi.aws.com) creates PVs on demand — dynamic provisioning.
  • • With WaitForFirstConsumer (gp3 / io2), the PV is only created after a Pod is scheduled — so the EBS volume lands in the node's AZ.
  • • Deleting the PVC detaches the PV: phase → Released. The reclaim policy picks what happens to the bytes.
  • • EBS PVs attach to exactly one node; EFS PVs are mounted over NFS from any node.
⚠ Gotcha: a Released PV is not automatically reusable — Retain leaves the bytes untouched but detached, Delete removes them permanently, Recycle wipes and re-offers them. That's why cloud providers default to Delete.
Module 2 • StorageClass Provisioner Selector

EBS vs EFS — choose your provisioner

Amazon EBS — gp3 (Default) — live detailsdynamic provisioning
IOPS
3K + 0.5/GiB · ≤ 16 K
Throughput
125–1 000 MiB/s
Latency
< 1 ms (p99)
Durability
99.9% (EBS)
Best for
General K8s stateful apps · Databases up to 16K IOPS · CI caches

Baseline performance is size-independent — a 1 GiB gp3 already has 3 000 IOPS; only pay extra IOPS above the baseline.

apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: ebs-gp3 annotations: storageclass.kubernetes.io/is-default-class: "true" provisioner: ebs.csi.aws.com reclaimPolicy: Retain volumeBindingMode: WaitForFirstConsumer allowVolumeExpansion: true parameters: type: gp3 fsType: ext4 iops: "3000" throughput: "125"

reclaimPolicy follows Module 4's selector — a class is created once; edit means recreate.

Module 3 • Access Mode Calculator

RWO / ROX / RWX — how many pods can touch it?

ReadWriteOnce (RWO)
PostgreSQL, MySQL, Redis — classic single-writer databases.
Nodes: 1 node
Writers: 1 writer (single pod)
EBS is zone-bound: the PV attaches only to nodes in the AZ where the volume lives.
✓ EBS gp3 supports RWO — pods can attach legally.
cluster view — 1 node
Node 1
volume attached
Node 2
mount blocked
Node 3
mount blocked
ModeFull nameNodesWritersEBSEFS
RWOReadWriteOnce1 node1 writer (single pod)✓✓ (as container root volume)
ROXReadOnlyManyMany nodes0 writers — all readers✓✓
RWXReadWriteManyMany nodesMany writers✗✓
rendered PVC manifest
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: apps-rwo
  namespace: app-prod
spec:
  accessModes:
    - RWO
  storageClassName: ebs-gp3
  resources:
    requests:
      storage: 100Gi
---
ReadWriteOnce (RWO) — PostgreSQL, MySQL, Redis — classic single-writer databases.
Module 4 • Reclaim Policy Selector

The fate of your bytes when the PVC is deleted

when PVC is deleted → Retain outcome

PVC deleted → PV phase becomes Released → the EBS volume / EFS file system is untouched. PV is NOT automatically rebound — an admin must delete the PV (and optionally re-market the volume). Data is safe but orphaned without manual cleanup; the cloud storage keeps billing you.

PV phase → AvailablePVC phase → Pendingstorage → EBS gp3

Tip: reclaimPolicy is fixed at StorageClass creation — choose Delete for auto-cleanup or Retain for insurance policies, and pair either with scheduled snapshots (Module 6).

apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: ebs-gp3 annotations: storageclass.kubernetes.io/is-default-class: "true" provisioner: ebs.csi.aws.com reclaimPolicy: Retain volumeBindingMode: WaitForFirstConsumer allowVolumeExpansion: true parameters: type: gp3 fsType: ext4 iops: "3000" throughput: "125"
Module 5 • Capacity Planner

Right-size the cluster storage budget

Volume size per PVC: 100 GiB100 GiB
10 GiB2 000 GiB
PVCs in the cluster: 3
120
Total provisioned300 GiB
Monthly cost (EBS gp3)$24.00/mo
≈ 6% of 5 TiB budget
Class$/GiBCost for 300 GiBIOPS est.ThroughputZones
EBS gp2$0.1$30.00300 IOPS≤ 250 MiB/sSingle-AZ
EBS gp3$0.08$24.003,050 IOPS125 MiB/sSingle-AZ
EBS io2$0.125$37.5050,000 IOPSup to 4 GiB/sSingle-AZ
Amazon EFS$0.3$90.0010+ GiB/s burstelasticMulti-AZ
Yearly estimate
$288
12 × monthly config
Per-PVC unit
100 GiB
× 3 claims
Snapshot margin
45 GiB
+15% for backups

IOPS estimates are per volume of the selected size (gp2: 3×GiB · gp3: 3 000 + 0.5×GiB · io2: 500×GiB) — EFS scales elastically regardless of reported capacity.

Module 6 • Volume Snapshot Simulator

Point-in-time recovery in three clicks

snapshot class: ebs-backup
snapshot & restore manifests
# No snapshots yet — press "Take Snapshot" to simulate an AWS EBS snapshot
# (EFS snapshots behave the same via efs-backup).
snapshot registry
No snapshots yet — press 📸 Take Snapshot (an EBS snapshot is saved to S3 within ~1 s for this sized volume).
Snapshot semantics: EBS snapshots are crash-consistent (stored in S3, incremental, restorable to any size); EFS snapshots cover a full file system and restore into the same region. In Kubernetes, restore means creating a brand-new PVC whose dataSource points at the VolumeSnapshot — the original volume is never touched.

Container Platform / Image Supply Chain

Docker Image Registry & Tag Strategies

Tag immutability, retention economics, multi-arch distribution, and layer anatomy — everything that decides whether an image is safe, findable, and cheap to store.

4 tag models8 lifecycle stages3 registries priced
🏷️

Image Tag Strategy Selector

Each strategy is a promise about repo:tag → digest stability. Pick one and inspect its failure modes.

Module 1
🏷️ Semantic Versioningrisk: mediumDetected may skip stale
Recommended for releases
✅ Best forRelease trains, feature versions, rollbacks by re-pinning an older version tag. Consumers read intent (major/minor/patch) directly from the tag.
⚠️ PitfallsOnly immutable if you never re-tag an existing version. Re-tagging v2.4.1 with a different digest silently rewrites history — pin each semver to exactly one digest, and use '+build' metadata for variant markers.

Example references

v2.4.0v2.4.1v2.5.0-rc.1
🔁

Image Lifecycle Visualizer

Every image travels build → registry → cluster → garbage. Drag the stage slider.

Module 2
🛠️

Build

docker build

docker build composes immutable layers (filesystem diffs). Each instruction (FROM, RUN, COPY) adds one layer; a rebuild reuses unchanged layers unless COPY'd files changed.

Registry stateLayers only exist locally; nothing published yet.
Cluster impactNo registry state; overlay mounts, cache keys = layer digests.
🧹

Retention Policy Builder

Policies fight registry sprawl. Tune rules below and watch the prune preview.

Module 3
2

per repository

7d

orphaned manifests

30d

beyond newest 2

comma-separated; never expires

Prune preview — 14 images

keep 10expire 4
✅payment-api:v2.4.12d old
480 MB·Protected tag matches policy
✅payment-api:v2.4.09d old
475 MB·Protected tag matches policy
✅payment-api:v2.3.228d old
470 MB·Protected tag matches policy
✅payment-api:v2.3.145d old
465 MB·Protected tag matches policy
🗑️payment-api:<untagged>61d old
460 MB·Untagged, older than 7 days
✅payment-api:v2.2.090d old
450 MB·Protected tag matches policy
✅web-frontend:1.4.05d old
340 MB·Within latest 2 tags
✅web-frontend:1.7.412d old
335 MB·Within latest 2 tags
✅web-frontend:1.7.320d old
330 MB·Within retention horizon
🗑️web-frontend:<untagged>35d old
325 MB·Untagged, older than 7 days
🗑️web-frontend:1.6.275d old
450 MB·Older than 30 days and beyond count
✅notifications-worker:0.4.04d old
56 MB·Within latest 2 tags
✅notifications-worker:0.3.160d old
52 MB·Within latest 2 tags
🗑️notifications-worker:<untagged>78d old
48 MB·Untagged, older than 7 days
Freed on next run: 1.25 GB≈ $0.13/mo at $0.10/GB

Generated policy

Mutable tags (branch deployments, re-tagged versions) escape age rules — pin policies to digest when traceability matters

🧬

Multi-Arch Manifest Viewer

One tag, many architectures: the manifest list (index) points to per-arch manifests.

Module 4

registry.example.com/team/payment-api:latest (index)

Selected manifest

platformlinux/amd64
mediaTypemanifest.v2+json
size61.4 MB (4 layers)
configsha256:9a4f4f5e6f7a8b9c0d1e2f3a4b5c6d7e8f9a0b1c2d3e4f5a6b7c8d9e0f1a2b3c
entrypoint["node", "server.js"]
Kubelet (arm64) asks for the index, receives this digest, and pulls only the arm64 manifest — amd64 nodes never download arm layers. Upstream: docker pull --platform linux/arm64
1 pushtag v2.4.1 published once
3 manifestsone per architecture
1 release traincluster-wide same version
💸

Registry Cost Calculator — ECR / ACR / GCR

Storage is billed per unique byte; egress per pull crossing the cloud boundary.

Module 5
1200 MB
120
18 GB
20%

Basic = $0, Standard/$25, Premium/$100

Amazon ECR

AWS
storage (112.5 GB @$0.10)$11.25
egress (18 GB @$0.090)$1.62

$12.87/mo

Flat $0.10/GB-mo storage; egress billed at standard AWS data-transfer rates. Free tier: 500 MB/mo private for new accounts (12 months).

Google Artifact Registry (GCR)

GCP
storage (112.5 GB @$0.10)$11.25
egress (18 GB @$0.120)$2.16

$13.41/mo

GCR is deprecated in favor of Artifact Registry. Storage $0.10/GB-mo; egress to internet at GCP network pricing (higher first-TB tier). Free tier 0.5 GiB/mo.

cheapest

Azure Container Registry

Azure
storage (112.5 GB @$0.10)$11.25
egress (18 GB @$0.087)$1.57

$12.82/mo

ACR storage ~$0.10/GB-mo beyond SKU-included quota; egress Zone-1 pricing applies additionally. SKUs (Basic/Standard/Premium) add their own monthly fee — add yours below.

Storage = unique layers after reuse deduction (80%); actual bills depend on image variance and retention. ACR also carries SKU monthly fees — the SKU input above is your own plan price.

🧅

Image Layer Analyzer

Peel an image: each layer is a filesystem diff. Same layers between images = stored once.

Module 6

registry.example.com/team/payment-api:v2.4.1

logical 59 MBunique 38 MBdedupable 22 MB
base
3 MBshared w/ notifications-worker
base
19 MBshared w/ notifications-worker
deps
24 MBunique
code
12 MBunique
env
1 MBunique
env
0 MBunique

Multi-stage build: node runtime only in final image. Even a tiny app ships the full base — check apk/apt layers & multi-stage=1.

Kubernetes Track Module • Security & Authorization

RBAC & Cluster Hardening Simulator

Design least-privilege Roles and Bindings, trace real authorization decisions through the RBAC chain, attach ServiceAccounts to Pods, apply Pod Security Standards, and harden security contexts — live in the browser.

Module 1 • Authorization Primitives

Role & ClusterRole Builder

Role · namespace ci-system

Namespaced Roles only affect resources in their own namespace.

apiGroups: [""]
resources: [pods, pods/log]
verbs: [get, list, watch, create, delete]
apiGroups: ["apps"]
resources: [deployments, replicasets]
verbs: [get, list, watch, create, update, patch]
rbac.authorization.k8s.io/v1 · Role
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: deployment-manager
  namespace: ci-system
rules:
  - apiGroups: [""]
    resources: ["pods", "pods/log"]
    verbs: ["get", "list", "watch", "create", "delete"]
  - apiGroups: ["apps"]
    resources: ["deployments", "replicasets"]
    verbs: ["get", "list", "watch", "create", "update", "patch"]

💡 get/list/watch is read-only, create/update/patch mutates, delete destroys. Grant the least set the workload actually needs.

Module 2 • Subject Binding

RoleBinding / ClusterRoleBinding Configurator

Subjects: User · Group · ServiceAccount

Lets the CI pipeline manage its own workload objects inside ci-system.

ci-builder@ci-system
⇢
RoleBinding “ci-builder-workloads”
⇢
Role ci-workload
scope: 🗂 ns/ci-system
Generated object
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: ci-builder-workloads
  namespace: ci-system
subjects:
  - kind: ServiceAccount
    name: ci-builder
    namespace: ci-system
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: ci-workload
Module 3 • Live Authorization Engine

Permission Evaluator

RBAC request path simulation
ALLOW2 bindings evaluated

Decision path: ci-builder-workloads

Request: ci-builder@ci-system · create pods · core/v1 · ci-system
✓ ALLOWci-builder-workloads (ns/ci-system)Role ci-workload (ns/ci-system)
rule: apiGroups [""] · resources [pods, pods/log] · verbs [get, list, watch, create, update, delete]
— skipci-builder-node-ro (cluster-wide)ClusterRole node-reader
hint: verb 'create' not granted (this rule grants: get, list, watch)
Effective permissions in request scope (6 entries)
API GroupResourceVerbs
core/v1configmaps
creategetlistupdatewatch
core/v1nodes+pods+services+endpoints
getlistwatch
core/v1pods+pods/log
createdeletegetlistupdatewatch
core/v1services
creategetlistwatch
appsdeployments+replicasets
creategetlistpatchupdatewatch
appsdeployments+statefulsets+daemonsets
getlistwatch
Module 4 • Workload Identity

ServiceAccount & Pod Association

Tokens mount at /var/run/secrets/kubernetes.io/serviceaccount
automountServiceAccountToken
Mount the identity token into every Pod using this SA
imagePullSecrets
Pull images from a private registry
Token files projected by kubelet
tokenca.crtnamespace

The API server authenticates in-cluster calls with this token — which ServiceAccount you bind decides what that Pod may do (Module 3).

ServiceAccount YAML
apiVersion: v1
kind: ServiceAccount
metadata:
  name: ci-builder
  namespace: ci-system
automountServiceAccountToken: true
imagePullSecrets:
  - name: registry-creds
Pod association
apiVersion: v1
kind: Pod
metadata:
  name: ci-builder-pod
  namespace: ci-system
spec:
  serviceAccountName: ci-builder
  automountServiceAccountToken: true
  imagePullSecrets:
    - name: registry-creds
  containers:
    - name: ci-builder-pod
      image: registry.company.io/ci-agent:v2.3
Module 5 • Admission Control

Pod Security Standards

Privileged → Baseline → Restricted

Enforce rejects non-compliant Pods at admission.

privileged: true
container runs with host kernel access
allowPrivilegeEscalation: true
process may gain extra privileges
HostPath volume
mounts a node filesystem path
hostNetwork: true
shares the node network namespace
runs as root (UID 0)
no runAsNonRoot / runAsUser set
seccomp: Unconfined
no seccomp profile applied
extra capabilities (no CAP_ALL drop)
Linux capabilities beyond NET_BIND_SERVICE
✗ POD REJECTEDagainst Restricted policy in ns/payments (enforce)
  • • Privileged containers are prohibited
  • • allowPrivilegeEscalation must be false
  • • hostNetwork is prohibited
  • • HostPath volumes are prohibited
  • • runAsNonRoot: true and a non-zero runAsUser are required
  • • seccompProfile: RuntimeDefault or Localhost is required
  • • capabilities must drop CAP_ALL (NET_BIND_SERVICE may be added)
Controls enforced by each stance
ControlPrivilegedBaselineRestricted
Privileged Containers
privileged: true
✓✗✗
Host Namespaces
hostPID / hostIPC: true
✓✗✗
Host Network & Ports
hostNetwork: true
✓✗✗
HostPath Volumes
mount path from host filesystem
✓✗✗
Privilege Escalation
allowPrivilegeEscalation: false
✓✗✗
Linux Capabilities
Drop ALL, may add NET_BIND_SERVICE only
✓◐◐
Seccomp
RuntimeDefault / Localhost
✓✓✗
runAsNonRoot
runAsNonRoot: true
✓✓✗
runAsUser ≠ 0
non-zero UID at runtime
✓✓✗
SELinux
type container_t only
✓◐◐
Namespace labels · enforce mode
apiVersion: v1
kind: Namespace
metadata:
  name: payments
  labels:
    pod-security.kubernetes.io/enforce: restricted
    pod-security.kubernetes.io/enforce-version: v1.30
Module 6 • Runtime Hardening

Security Context Configurator

Grade A+ · 91/100
privileged: true
host kernel access — never in production
allowPrivilegeEscalation: true
process may gain additional privileges
runAsNonRoot: true
refuse to start as UID 0
readOnlyRootFilesystem: true
root fs read-only, writes go to volumes
drop CAP_ALL capabilities
strip every Linux capability
hostNetwork: true
share the node network namespace
hostPID: true
share host process namespace
fsGroup: 2000
pod-level fsGroup ownership for volumes
Hardening assessment
A+
91/100 hardening score

✓ No hardening issues detected — this Pod matches restricted-level best practice.

Pod
apiVersion: v1
kind: Pod
metadata:
  name: hardened-api
spec:
  securityContext:
    fsGroup: 2000
  containers:
    - name: app
      image: registry:9090/hardened-api:latest
      securityContext:
        privileged: false
        allowPrivilegeEscalation: false
        readOnlyRootFilesystem: true
        runAsNonRoot: true
        runAsUser: 1000
        runAsGroup: 3000
        capabilities:
          drop: ["ALL"]
          add: ["NET_BIND_SERVICE"]
        seccompProfile:
          type: RuntimeDefault
Module 5 • Kubernetes Workload Autoscaling

HPA & VPA Autoscaling Control Room

Configure metrics, tune scale behaviors, then watch replica decisions play out live

autoscaling/v2autoscaling.k8s.io/v1
📈

HPA Metric Configurator

Pick the signal the controller watches

HorizontalPodAutoscaler
Metric Source
Resource
Target Type
Target Utilization (of pod request)
70%
desiredReplicas = ceil( currentReplicas × currentMetric ÷ targetMetric )
$ kubectl get hpa web-api -w
NAME    REFERENCE      TARGETS      MIN  MAX  REPLICAS
web-api  Deployment/web-api  70%  2   10   2
hpa.yaml — autoscaling/v2
apiVersion: autoscaling/v2 kind: HorizontalPodAutoscaler metadata: name: web-api-hpa namespace: production spec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: web-api minReplicas: 2 maxReplicas: 10 metrics: - type: Resource resource: name: cpu target: type: Utilization averageUtilization: 70 behavior: scaleUp: stabilizationWindowSeconds: 0 selectPolicy: Max policies: - type: Percent value: 100 periodSeconds: 15 - type: Pods value: 4 periodSeconds: 15 scaleDown: stabilizationWindowSeconds: 300 selectPolicy: Max policies: - type: Percent value: 100 periodSeconds: 15 - type: Pods value: 2 periodSeconds: 15
🎚️

Scale-Up / Scale-Down Behavior Editor

Stabilization windows + per-event rate policies

▲ Scale-Upwindow 0s · select Max
Max = most aggressive policy wins (fastest growth)
▼ Scale-Downwindow 300s · selectPolicy Max
Disabled = never shrink pods
Pro tip: keep scale-down slower than scale-up — a 300s down window absorbs traffic dips that would otherwise trigger ping-pong scaling.
behavior blockscaleDown.selectPolicy: Max
behavior: scaleUp: stabilizationWindowSeconds: 0 selectPolicy: Max policies: - type: Percent value: 100 periodSeconds: 15 - type: Pods value: 4 periodSeconds: 15 scaleDown: stabilizationWindowSeconds: 300 selectPolicy: Max policies: - type: Percent value: 100 periodSeconds: 15 - type: Pods value: 2 periodSeconds: 15
🧮

Target Utilization → Required Replicas Calculator

ceil( replicas × observed ÷ target ) — the math HPA runs every reconcile loop

Target Utilization
70%
Observed utilization
17%
Required replicas
1
Math applied
3 × 17% ÷ 70%
Action
scale -2
example: 3 pods currently at 17% vs target 70% → desired 1 replicas. HPA clamps this to [min, max] before acting and tolerates ±10% drift.
📊

Vertical Pod Autoscaler — Recommendation Viewer

VPA mines history (histograms) and emits request bounds, not just a target

Mode:
Workload Profiles
⚠ VPA × HPA conflict: never run VPA in Auto on the same resource metric HPA watches — they fight over requests. Use VPA on memory + HPA on CPU, or set VPA to Off.
MetricCurrentObservedLowerRecommendedUncappedUpperΔ vs current
cpu500m318m268m343m366m532m−31%
memory512Mi542Mi484Mi569Mi607Mi910Mi+11%
CPU — current vs recommended vs upper bound343m recommended
Memory — current vs recommended vs upper bound569Mi recommended
showing original requestsAuto mode restarts pods to apply
$ kubectl describe vpa web-vpa
Status: Running · target: web-api
Recommendation: cpu 343m (268m–532m) · memory 569Mi (484Mi–910Mi)
⚙️

Autoscaling Simulator — Live Replica Decisions

Drives the configured metric through HPA math: formula → stabilization → policy → min/max bounds

Replicas
—
Load (mCPU)
—
Status
idle
▲ scale-ups
0
▼ scale-downs
0
Peak replicas
—
Target
70%
Bounds
2–10
📉

Press ▶ Simulate to watch the HPA compute replicas against the traffic spike scenario

replica count target metric demand (mCPU) maxReplicas ceiling
REPLICA COUNT VISUALIZATION — LAST 0 TICKSidle

No samples yet — start the simulation.

SCALING EVENT LOG0 events

— waiting for the first scale event —

✅ Autoscaling Best Practices

  • ▸ Start with CPU utilization (~70–80%), add custom metrics (QPS, queue depth) only once they stabilize.
  • ▸ Set minReplicas ≥ 2 for HA — the floor keeps serving even at zero load.
  • ▸ Scale down gently: stabilizationWindowSeconds ≥ 300s prevents flapping on dips.
  • ▸ HPA scales on per-pod averages; per-pod skew is handled by VPA instead.
  • ▸ VPA on memory + HPA on CPU is battle-tested; avoid both on the same metric in Auto mode.
  • ▸ After VPA changes requests, HPA's utilization denominator changes — re-check targets.
Kubernetes Troubleshooting ModuleMode: Debug Deck

Diagnose Pods Like a Cluster SRE

Walk the Pod lifecycle state machine, decode the four failure archetypes (CrashLoopBackOff, ImagePullBackOff, Pending, OOMKilled), generate exact kubectl fire commands, replay cluster events, and tune liveness/readiness probes until the pod stays green.

5
Visualizations
4
Failure Patterns
Module 1 • Lifecycle & States

Pod Lifecycle State Machine Simulator

Pick a scenario → watch the phase graph light up
Phase Transition Graphclick a phase for details
→
→
→
Branches:
Driving sequencehealthy

Scheduler picks a node, kubelet pulls the image, probes pass, pod goes Ready.

PendingContainerCreatingRunning
$ kubectl get pod
STATUS: Running  READY: 1/1  RESTARTS: 0  AGE: 2m
🖱️

This scenario passes through Pending → ContainerCreating → Running. Click any phase to inspect its meaning, output signals, and the exact kubectl probe.

Module 2 • Failure Patterns

Common Pod Failure Pattern Library

The four states you will debug 95% of the time
Definition

The kubelet detects the container exiting with a non-zero code shortly after start and backs off with exponential delay (10s, 20s, 40s … up to 5m) before each retry. Almost always an app-level bug for freshly written code — rarely an infrastructure problem.

Root causes
  • ▸Application panic / unhandled exception during startup
  • ▸Missing env var, config file, or secret the app hard-crashes on
  • ▸Port already in use inside the container or namespace
  • ▸Wrong architecture: amd64 image scheduled onto an arm64 node
  • ▸Probe fails from t=0 with failureThreshold=1 — the probe kills it before readiness ever passes
First-response fixes
  1. 1Read the last run: kubectl logs <pod> --previous -c app
  2. 2Verify mounts and secrets: kubectl exec <pod> -- env && cat /app/config/*
  3. 3Run the image locally with the same args and observe the exit code
  4. 4Temporarily override the entrypoint to a sleep to debug interactively
  5. 5Give the app room to boot: raise liveness initialDelaySeconds before blaming it
Detection commands
kubectl logs <pod> --previous --tail=50 && kubectl describe pod <pod> | grep -A 8 Events
  • ✓kubectl get pods → STATUS: CrashLoopBackOff, RESTARTS climbing
  • ✓kubectl logs <pod> --previous → application stack trace at startup
  • ✓kubectl describe pod <pod> → Events: Back-off restarting failed container
Witness it in a real terminalsimulated output
$ kubectl get pod api-7d9bbb98-6xz7k NAME READY STATUS RESTARTS AGE api-7d9bbb98-6xz7k 0/1 CrashLoopBackOff 5 3m $ kubectl logs api-7d9bbb98-6xz7k --previous --tail=20 Error: Cannot find module '/app/config/settings.json' at Module._load (node:internal/modules/cjs/loader:121:15) at Module.require (node:internal/modules/cjs/loader:127:19) at Object.<anonymous> (/app/dist/main.js:42:1)
Module 3 • Debug Arsenal

kubectl Debug Command Generator

Compose exact commands, never fumble the flags
Inspect application output; add --previous to see the last crashed container's logs.
Show previous logs
Follow output (-f/-w)
ops-terminal — kubectl
$
kubectl logs web-7b6f9d-4x7k2 -c app -n default --tail=50
Module 4 • Signal Stream

kubectl get events — Live Viewer Simulator

Replay what the Events stream looks like under real failure
Replay:
Time
Object
Type / Reason
Message
$ kubectl get events --sort-by=.lastTimestamp
No events yet — replay a scenario above.
SRE readout: Events are short-lived (the API server aggregates them) — always describe the pod while it is still failing.
Module 5 • Probe Lab

Liveness / Readiness / Startup Probe Debugger

Tune thresholds → see the kubelet fire
Cuts the Service endpoint when the probe fails — the pod stays alive but stops receiving traffic. The right tool for slow boot and dependency health.
Endpoint responds
200 OK — probe will pass
Probe timeline (t = container start)
Press ▶ Run probe simulation to watch the kubelet schedule probe rounds.
Ready — traffic routed
Readiness probe passing; Service endpoints include this pod.
Probe YAML (generated)deployment spec fragment
kind: Deployment spec: template: spec: containers: - name: app image: nginx:1.27 readinessProbe: httpGet: path: /health port: 8080 initialDelaySeconds: 10 periodSeconds: 5 timeoutSeconds: 1 failureThreshold: 3 successThreshold: 1
Docker Track • Container Observability

Observability Stack Lab

Prometheus metric collection, Grafana dashboard composition, Tempo/Jaeger distributed tracing, structured log streams, alert rule authoring, and SLI/SLO budgeting — the full observability loop for containerized workloads.

Scrape · 15s intervalRetention · 15dTargets · 4/5 upTempo · trace.id propagation
Module 1 • Prometheus Metrics

Live Metric Simulator

Gauges sampled every 1300ms — counters stay monotonic, rate() converts them to usable rates.

CPU UsageGauge
Sum of container CPU time over the node, expressed as a percentage of node capacity.
query container_cpu_usage_seconds_total
window 26p · last 1300msnext scrape…
Scrape targetsTSDB state
cadvisor
UP
node-exporter
UP
kube-state-metrics
UP
backend-api
UP
ingress-nginx
DOWN
up = scrape succeeded · labels define each series' cardivality key.
PROMETHEUS:counters always increase — use rate()gauges go up and down (current value) ·histograms power quantiles + SLIs ·alerting uses the same query language as dashboards.
Module 2 • Grafana

Dashboard Builder

Compose panels from the widget palette, size them on the canvas, export provisioning JSON.

Widget palette
Untitled Dashboard · 0 panelsauto refresh 15s · time range 1h
▦

empty canvas — add widgets from the palette

or pick a preset template above

GRAFANA PANELS: each panel renders a PromQL target — the query defines the data, the visualization defines the shape (lines, gauges, heatmaps, tables).
KEY CONCEPTS:dashboards are JSON + provisioned via config ·folders/teams gate dashboard access ·unified alerting = Prometheus rules rendered as panels ·query variables ($namespace) keep dashboards reusable.
Module 3 • Distributed Tracing

Trace Waterfall — Jaeger / Tempo

One user request fans out across services; every span carries start time, duration, status, and its parent link.

Sampling tail-based 1% + errors
Traces (5)
TRACE ID: 7f4a92e1c8b3d5f6a0e1b2c3
propagated via traceparent header · W3C ctx
POST /api/v1/orders · 1240ms total11 spans · service depth 7
0ms310ms620ms930ms1240ms
OK · SERVERcritical path: yes
span: "orders.create"
service: backend-api
start: 86ms
duration: 1140ms
kind: SERVER
trace: 7f4a92e1c8b3
tags: component=http
parent: span@56ms

This span took 1140ms — 92% of the 1240ms trace.

TRACING 101:trace id propagates via traceparent / b3 headers ·span context carries parent→child topology ·tail sampling keeps error traces, drops happy paths ·waterfall duration = wall clock, gaps = queuing/waiting.
Module 4 • Structured Logging

Log Stream Explorer

JSON-structured entries with levels, services, and key=value fields — filter, search, live-tail.

loki / cluster-aggregate
6 of 6 lines
14:02:11.482INFO gatewayrequest started method="POST" path="/api/v1/orders" req_id="req-8f3a"
14:02:11.490INFO backend-apiorder claimed order_id="ord-10421" total=428.9 customer="usr-77"
14:02:11.501WARN inventory-svcstock low for SKU sku="SKU-8821" qty=5 min=10
14:02:11.505INFO payment-svccharge authorized id="ch-77kk" ms=84
14:02:11.512ERRORpayment-gatewayupstream timeout on auth provider="stripe" attempt=2 ms=1004
14:02:11.518INFO gatewayresponse completed req_id="req-8f3a" status=201 ms=38
tailing…
STRUCTURED LOGGING: names + levels only — search by field (e.g. order_id="ord-10*"), correlate to traces via trace_id, and keep INFO for operational context while WARN/ERROR feed the alert path.
Module 5 • Alert Rules

Prometheus Alert Configurator

Author a rule against live simulated values, evaluate it repeatedly, and export the YAML for your cluster.

STATE: INACTIVE
1 · Metric & Condition
expr sum(rate(http_requests_total{status=~"5.."}[5m])) / sum(rate(http_requests_total[5m])) * 100
%
for
2 · Labels & Severity
STATES: value 1.4% > 2 → within budget. for: 5m means the condition must persist through evaluations before FIRING.
Live value vs threshold — simulated feed
1.4%
○ NORMAL
live from module 1 sim
0threshold 2%scale max
eval 1eval 2eval 3 no alert condition met
prometheus-alerts.yml
groups:
  - name: container-observability.alerts
    rules:
      - alert: HTTP_5XX_ERROR_RATE
        expr: sum(rate(http_requests_total{status=~"5.."}[5m])) / sum(rate(http_requests_total[5m])) * 100 > 2
        for: 5m
        labels:
          severity: critical
        team: "platform"
        env: "prod"
        annotations:
          summary: "5xx rate is at {{ $value }}% of traffic"
          description: "Value 1.4% is > 2 for 5m."
ALERTING:expr is a PromQL boolean — true = breach ·for= prevents flapping ·labels route → receiver (severity, team) ·annotations are the human-readable message.
Module 6 • SLI / SLO

Error Budget Calculator

Turn SLIs (good / total) into an SLO, then track how much of the error budget has been consumed.

Window & Target
window (days)
good requests
bad (5xx / timeouts)

SLI = successful requests / total requests — counts "good" events over the window.

DEFINITIONS: SLI = the measured indicator · SLO = the agreed target · error budget = 100% − SLO target, the seconds of allowed failure each month.
Availability (SLI)
99.9500%
Error budget
0.10%
Bad events allowed
4,500
Remaining budget
2,250 events
Error budget consumptionCOMPLIANT
0%85% fast-burn guardrail100%
Good traffic
100.0%
Burn rate (multiplier)
0.50×
Projected exhaustion
60d
Budget left
50.0%
MULTI-WINDOW:burn rate = consumed / elapsed window ·1× burn = steady state ·14.4× = page (fast burn) ·multi-window alerts (1h + 5m) catch both slow leaks and spikes.

Observability Loop Cheat Sheet

METER

Expose /metrics via client libraries; watch label cardinality; scrape on a fixed interval; store with a retention policy.

VISUALIZE

Grafana panels over PromQL; variables + provisioning keep dashboards reproducible; alert from inside the same panels.

TRACE

Correlate one request across services with traceparent. Waterfalls expose latency ownership; sample 1–10% head/tail.

LOG

JSON lines with level, service, and fields. Filter by level and search by field; wire ERROR streams into alert paths.

ALERT

expr boolean + for duration → labels → receivers. Put threshold with slack; PENDING vs FIRING is hysteresis.

SLO

SLI = good/total. Error budget = 100% − target. Multi-window burn alerts page before the budget is gone.