Kubernetes + Helm

Pathrule5 Rules • 2 Memories • 1 Skill

A generated Kubernetes manifest usually deploys and usually survives a demo. What it lacks is everything the cluster needs to treat the workload correctly: requests so the scheduler can place it, three probes with different jobs, a disruption budget so a node drain does not take the service down, a security context so the container is not root, and a shutdown path that stops accepting traffic before it exits. Audits of public Helm charts keep finding the same two gaps, missing disruption budgets and missing resource requests, which is exactly what this bundle refuses to ship without.

Suggested path map

Pathrule places each piece on the matching path, so your assistant only sees it where it belongs. This is the scoping you get on import; you can adjust it in your workspace.

/ workspace root

k8s-production-readiness

k8s/

Every container declares resource requests and limits

Use all three probes, each for its own job

Survive disruption: budgets, spread, and graceful shutdown

Harden the pod: non-root, read-only, no capabilities, pinned image

Configuration, secrets, and how a change reaches a pod

Scaling: autoscalers only work with honest requests

charts/

Keep charts declarative, versioned, and free of secrets

Rules

Every container declares resource requests and limits/k8shighstrictRequests reflect steady-state usage so the scheduler can place the pod, memory limit equals request, and CPU limits are used deliberately.

1	Requests are what the scheduler reserves; limits are what the kernel enforces. A container with neither is scheduled blind and evicted first when the node is under pressure.
2
3	- Set a CPU and memory request on every container, including init and sidecar containers, based on observed steady-state usage rather than a guess.
4	- Set the memory limit equal to the request for anything you care about. Memory is not compressible: exceeding the limit is an OOM kill, and a request far below the limit means the pod is in the Burstable class and gets evicted early.
5	- Be deliberate with CPU limits. A limit throttles the container at the boundary, which shows up as latency spikes rather than load. For latency-sensitive services, prefer a solid request and a generous or absent limit.
6	- Give a namespace a ResourceQuota and LimitRange so a missing request is caught by the cluster, not discovered during an incident.
7	- Revisit the numbers with real data (a vertical autoscaler in recommend mode, or your metrics) instead of copying them between services.

Use all three probes, each for its own job/k8shighstrictStartup probes cover slow boots, readiness gates traffic, liveness restarts a wedged process, and readiness never checks a dependency you do not control.

1	The three probes answer three different questions, and copying one endpoint into all three slots produces restart loops during deploys.
2
3	- `startupProbe` covers initialisation: while it is failing, liveness and readiness are suspended, so a slow-booting app is not killed before it is up. Give it a generous `failureThreshold`.
4	- `readinessProbe` decides whether the pod receives traffic. It should check that this process can serve a request right now (caches warm, migrations applied, worker pool started).
5	- `livenessProbe` decides whether to restart the process. It must only fail when a restart genuinely fixes things: a deadlock, an unrecoverable state. A liveness probe that checks the database restarts every pod during a database blip and turns a partial outage into a total one.
6	- Keep probe endpoints cheap, dependency-light, and unauthenticated within the cluster. No probe should run a query per call.
7	- Tune `periodSeconds`, `timeoutSeconds`, and thresholds so a transient hiccup does not remove capacity, and make readiness fail fast so shutdown drains quickly.

Survive disruption: budgets, spread, and graceful shutdown/k8shighstrictMore than one replica, a PodDisruptionBudget that a drain can still satisfy, spread across nodes and zones, and a preStop hook that drains connections.

1	Nodes are drained for upgrades constantly. A workload that cannot tolerate that is a workload that has an outage on every cluster maintenance window.
2
3	- Run at least two replicas for anything serving traffic, and set a `PodDisruptionBudget` (`minAvailable` or `maxUnavailable`) that keeps enough capacity while still allowing a drain to make progress. A budget that blocks every eviction is worse than none: it stalls upgrades.
4	- Spread replicas with `topologySpreadConstraints` across nodes and, where available, zones, so one node or zone loss is not the whole service.
5	- Set `rollingUpdate` `maxUnavailable` and `maxSurge` intentionally, together with the readiness probe, so a bad rollout stops before it replaces every pod.
6	- Handle SIGTERM: fail readiness immediately, finish in-flight requests, then exit. Add a short `preStop` sleep so the endpoints controller removes the pod from the load balancer before the process stops accepting connections, and set `terminationGracePeriodSeconds` longer than your slowest request.
7	- For batch and queue workers, make shutdown mean "stop taking new work and finish the current item", and make the work idempotent so a kill mid-item is safe.

Harden the pod: non-root, read-only, no capabilities, pinned image/k8shighstrictContainers run as a non-root user with a read-only root filesystem and dropped capabilities, and images are pinned to an immutable tag or digest.

1	A container defaults to more privilege than it needs, and a mutable tag means you cannot say what is running.
2
3	- Set `securityContext` with `runAsNonRoot: true`, an explicit `runAsUser`/`runAsGroup`, `allowPrivilegeEscalation: false`, `readOnlyRootFilesystem: true`, and `capabilities.drop: ["ALL"]`. Mount an `emptyDir` for the paths that genuinely need to be writable.
4	- Never `privileged: true`, never host network, PID, or IPC namespaces, and no host path mounts outside a deliberate, reviewed infrastructure workload.
5	- Pin images to an immutable tag or a digest and set `imagePullPolicy: IfNotPresent` with that pin. `:latest` makes rollbacks meaningless and two replicas can silently run different code.
6	- Give each workload its own ServiceAccount with only the RBAC it needs, and disable the token mount (`automountServiceAccountToken: false`) when the pod does not call the API server.
7	- Restrict traffic with NetworkPolicies: default deny in the namespace, then allow the specific flows. Enforce the whole set with Pod Security Admission or a policy engine so the next manifest cannot skip it.
8
9	See the docker-containers pattern for how the image itself is built, and supply-chain-security for provenance.

Keep charts declarative, versioned, and free of secrets/chartsmediumadvisoryValues carry configuration with sane defaults, secrets come from a secret manager, chart versions bump on every change, and upgrades are atomic.

1	A Helm chart is the deployable contract for a service. Treated casually, it becomes a template nobody can render safely.
2
3	- No secret values in `values.yaml` or in a committed values file. Reference an existing Secret, or generate it from a secret manager (External Secrets, Secrets Store CSI, sealed secrets). Charts are shared and end up in artifact registries.
4	- Give every value a documented default that produces a working, safe deployment, and validate inputs with `values.schema.json` so a typo fails at install rather than at runtime.
5	- Bump the chart `version` on every change and set `appVersion` to the image tag. An unversioned chart makes rollback a guess.
6	- Deploy with `helm upgrade --install --atomic --timeout`, so a failed rollout rolls back instead of leaving half the replicas on new code. Run `helm template` and a policy check (kubeconform, conftest, or your admission policies) in CI.
7	- Do not hand-edit live resources: `kubectl edit` on a chart-managed object is drift that the next upgrade will either revert or fight. Change the chart.
8	- Add a checksum annotation on the pod template for mounted ConfigMaps and Secrets so a config change actually triggers a rollout.

Memories

Configuration, secrets, and how a change reaches a pod/k8sConfigMaps and Secrets are versioned inputs; mounted files update in place, environment variables do not, and neither restarts a pod on its own.

1	Config delivery in Kubernetes is subtle in exactly one way: what happens to a running pod when the config changes.
2
3	- A Secret or ConfigMap consumed as environment variables is read once at start. Changing it does nothing until the pod restarts, so a rollout must be triggered (the standard trick is a checksum annotation on the pod template).
4	- The same object mounted as a volume is updated in place after a short delay, which only helps if the application re-reads the file. Decide which behaviour you want and implement it deliberately.
5	- Keep secret material out of the manifest tree. Pull it from a manager at deploy time or sync it into the cluster, and let RBAC decide who can read the Secret object.
6	- Separate genuinely environment-specific values (replicas, hostnames, resource sizes) from application configuration, so promoting a release does not require editing application settings.
7	- Keep the whole desired state in git and let a controller reconcile it (GitOps) rather than applying from a laptop, so the cluster and the repo cannot disagree silently.
8
9	See /charts for the chart rules and the secrets-env-management pattern for storage and rotation.

Scaling: autoscalers only work with honest requests/k8sThe horizontal autoscaler scales on a signal you choose, requests decide scheduling, and the cluster autoscaler can only help if pods are schedulable.

1	Autoscaling failures are almost always upstream of the autoscaler.
2
3	- Scale on the metric that reflects saturation for the workload: CPU for compute-bound services, queue depth or requests per second for I/O-bound ones (through custom or external metrics). CPU utilisation is measured against the request, so a wrong request makes the target meaningless.
4	- Set `minReplicas` above one for availability, a `maxReplicas` you can actually afford, and stabilisation windows so the autoscaler does not oscillate. Scale up fast, scale down slowly.
5	- The cluster autoscaler adds nodes when pods are pending. Pods pend because of requests, affinity, or topology constraints, so overly strict constraints look like an autoscaler that does not work.
6	- Do not run a horizontal and a vertical autoscaler on the same resource metric; they fight. Use vertical recommendations to set requests, then scale horizontally.
7	- For bursty batch work, prefer a queue plus workers that scale on depth over a fixed deployment sized for the peak, and use priority classes so batch work yields to interactive traffic.
8
9	See /k8s for the requests rule that all of this depends on.

Skills

k8s-production-readiness/rootGate a workload before its first production deploy or before a manifest change ships.

1	---
2	name: k8s-production-readiness
3	description: Production readiness gate for a Kubernetes workload: resources, probes, disruption tolerance, security context, config delivery, and observability. Run before the first production deploy and before merging manifest or chart changes.
4	---
5
6	# Kubernetes production readiness
7
8	## Scheduling
9	- [ ] CPU and memory requests set on every container, from observed usage.
10	- [ ] Memory limit equals request; CPU limit chosen deliberately.
11	- [ ] Namespace has a ResourceQuota and LimitRange.
12
13	## Health
14	- [ ] Startup, readiness, and liveness probes are distinct and cheap.
15	- [ ] Liveness does not check external dependencies.
16	- [ ] Readiness fails immediately on SIGTERM.
17
18	## Disruption
19	- [ ] At least two replicas for serving workloads.
20	- [ ] PodDisruptionBudget allows a drain to complete while keeping capacity.
21	- [ ] `topologySpreadConstraints` spread replicas across nodes and zones.
22	- [ ] `preStop` and `terminationGracePeriodSeconds` drain in-flight requests.
23
24	## Security
25	- [ ] `runAsNonRoot`, explicit user, `readOnlyRootFilesystem`, `capabilities.drop: [ALL]`, no privilege escalation.
26	- [ ] No privileged mode, host namespaces, or host path mounts.
27	- [ ] Image pinned to an immutable tag or digest.
28	- [ ] Dedicated ServiceAccount, minimal RBAC, token mount off if unused.
29	- [ ] NetworkPolicy default-deny plus explicit allows.
30
31	## Configuration
32	- [ ] No secrets in chart values or committed files.
33	- [ ] Config change triggers a rollout (checksum annotation) or is re-read at runtime.
34	- [ ] Chart version bumped; `helm upgrade --atomic` used.
35
36	## Operations
37	- [ ] Logs go to stdout as structured lines.
38	- [ ] Metrics and traces are exported, and dashboards and alerts exist.
39	- [ ] A rollback command is written down and has been tested once.

Why this pattern

AI agents generate Kubernetes manifests with no resource requests, one probe copied into all three slots, a latest image tag, a root container, and no disruption budget or graceful shutdown.

Built for Platform and backend teams running services on Kubernetes with Helm.

Keeps your assistant from:

Deploying a container with no resource requests, so the scheduler cannot place it sanely
Using the same endpoint for liveness and readiness and restarting pods during a slow start
Draining a node and taking every replica of a service with it
Running as root with a writable root filesystem and a mutable latest tag

View as markdown

License: Apache-2.0
Version: 1.0.0
Updated: 2026-08-24