Kubernetes Secrets Security: A Practical Guide

Base64 is not encryption

A Kubernetes Secret stores its data base64-encoded and, by default, writes it to etcd in that form. Base64 is a reversible encoding, not a cipher. Anyone with read access to the Secret — or to an etcd backup — recovers the plaintext instantly:

# This is all it takes to read an "encrypted" Secret
kubectl get secret app-db -o jsonpath='{.data.password}' | base64 -d

So the real question is not “is it a Secret?” but “who can read it, and is it encrypted where it rests?” The rest of this guide answers both.

Step 1: Turn on encryption at rest

Encryption at rest ensures that a stolen etcd snapshot does not hand over every credential in the cluster. Create an EncryptionConfiguration and point the API server at it with --encryption-provider-config:

apiVersion: apiserver.config.k8s.io/v1
kind: EncryptionConfiguration
resources:
  - resources:
      - secrets
    providers:
      # First provider is used to encrypt new writes
      - aescbc:
          keys:
            - name: key1
              secret: <base64-encoded 32-byte key>
      # identity must remain so existing plaintext stays readable
      - identity: {}

After enabling it, re-encrypt everything already stored: kubectl get secrets -A -o json | kubectl replace -f -. For production, prefer a KMS v2provider (GA in Kubernetes 1.29) backed by an external key manager such as AWS KMS, Google Cloud KMS, or Vault — that way the encryption keys live outside the cluster instead of in a file on the API server disk.

Step 2: Lock down RBAC

Encryption at rest does nothing against someone who can simply ask the API for the Secret. The complementary control is least-privilege RBAC: grant get on named Secrets only to the identities that need them, and avoid handing out cluster-wide list or watch on Secrets.

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: app
  name: read-app-db-secret
rules:
  - apiGroups: [""]
    resources: ["secrets"]
    resourceNames: ["app-db"]   # scoped to one Secret
    verbs: ["get"]

One caveat worth knowing: resourceNames restricts get but cannot restrict list, so a role with list access can read every Secret in the namespace. Grant list sparingly.

Step 3: Mount secrets, do not inject them as env vars

Environment variables feel convenient, but they leak. Child processes inherit the whole environment, crash handlers dump it, and application logs routinely print it. A Secret mounted as a volume is backed by in-memory tmpfs, can be made read-only with a restrictive file mode, and can update in place without a container restart.

apiVersion: v1
kind: Pod
metadata:
  name: app
spec:
  automountServiceAccountToken: false
  containers:
    - name: app
      image: myapp:1.4.2
      volumeMounts:
        - name: db-secret
          mountPath: /etc/secrets
          readOnly: true
  volumes:
    - name: db-secret
      secret:
        secretName: app-db
        defaultMode: 0400   # owner read-only

Step 4: Disable unused service account tokens

Every pod historically got a mounted service account token whether it needed to talk to the API or not — a free credential for an attacker who lands in the container. Since Kubernetes 1.24 those tokens are short-lived and audience-bound rather than long-lived Secrets, which is a big improvement, but the best token is the one that is not mounted at all. Set automountServiceAccountToken: false on the pod or service account (as in the manifest above) unless the workload genuinely calls the Kubernetes API.

Step 5: Keep secrets out of Git

GitOps is a huge win for auditability right up until someone commits a plaintext Secret manifest into a repository that a hundred people can clone. Three established patterns keep the real value out of Git:

• Sealed Secrets. A controller in the cluster holds a private key; you commit an encrypted SealedSecret that only it can decrypt into a real Secret.
• SOPS. Encrypts individual fields with a KMS or age key, so the manifest in Git shows structure but not values.
• External Secrets Operator. Stores nothing sensitive in Git at all; it syncs values at runtime from Vault or a cloud secret manager into Kubernetes Secrets.

Managing secrets this way is part of a healthy supply chain overall — see our software supply chain security primer for how it fits with signing and provenance.

Step 6: Rotate and audit

Treat every secret as something that will eventually leak and plan for rotation: prefer short-lived, dynamically issued credentials (a strength of Vault and cloud secret managers) over static long-lived ones, and make rotation a routine operation rather than an incident response. Turn on API server audit logging for Secret access so you can answer “who read this credential, and when?” — and alert on unexpected list operations across namespaces, which are a classic reconnaissance step after a compromise.

Common mistakes to avoid

Most Secret incidents are not exotic — they are the same handful of mistakes repeated. Watch for these:

• Treating base64 as protection. It is encoding. Anyone with read access decodes it in one command.
• Putting secrets in ConfigMaps. ConfigMaps are never encrypted at rest, get no special access treatment, and show up in plain text in kubectl describe. Credentials belong in Secrets, not ConfigMaps.
• Granting broad list/watch on Secrets. A single over-scoped role undoes careful per-Secret get restrictions.
• Committing plaintext manifests to Git.Once pushed, treat the secret as compromised and rotate it — deleting the commit does not scrub every clone and mirror.
• Never rotating. A static credential that has existed for two years has almost certainly been copied into a laptop, a CI log, or a screenshot somewhere.

None of these require sophistication to fix; they require making the safe path the default one, which is exactly what the steps above set up.

Where ScanRook fits

Everything above secures secrets at runtime. There is an earlier failure mode that no amount of etcd encryption or RBAC touches: a credential baked into the container image itself. Keys copied into an image layer persist in the layer history even if a later step deletes them, and they ship to every registry and node that pulls the image. ScanRook is an image scanner, not a Kubernetes secret manager — but because it unpacks and inspects every layer, it can surface embedded keys and vulnerable packages before an image reaches your cluster. That is the honest boundary: manage Secrets with the Kubernetes controls in this guide, and scan images so a hard-coded credential never becomes a runtime problem in the first place. The image side of that gate is covered in our Kubernetes vulnerability scanning guide.

Frequently asked questions