How to Securely Store and Access Secrets in Kubernetes

Introduction

As cloud-native applications evolve, the need to securely manage sensitive configuration data such as passwords, tokens, and API keys becomes increasingly paramount. Kubernetes, the orchestration platform for containerized applications, includes capabilities for managing these pieces of sensitive data through its Secrets resource. In this tutorial, we’ll explore how to securely store and access secrets in Kubernetes, walking through various methods and recommended practices.

Understanding Kubernetes Secrets

Kubernetes Secrets are a resource type which can contain sensitive data such as passwords, OAuth tokens, and SSH keys. They are stored in a temporary filesystem of a pod and can be mounted as data volumes or exposed as environment variables to be used by containers within a pod.

Here is an example of creating a Secret manually:

apiVersion: v1
kind: Secret
metadata:
  name: my-secret
type: Opaque
data:
  username: YWRtaW4=
  password: MWYyZDFlMmU2N2Rm

The data field contains the actual secrets in base64-encoded strings: ‘admin’ for the username and an example password respectively.

Creating Secrets with kubectl

One basic method to create Secrets is by using the kubectl command-line tool. You can create a Secret from a literal value:

kubectl create secret generic my-secret --from-literal=username=admin --from-literal=password='s3cr3tpassw0rd'

Alternatively, if your secrets are in files, you can do:

kubectl create secret generic my-secret-file --from-file=path/to/file

Inspecting our secret:

kubectl get secret my-secret -o yaml

The outputs don’t directly show you the secrets because they’re encoded.

Using Secrets in Pods

Once the secret is created, you can use it in a Pod manifest:

apiVersion: v1
kind: Pod
metadata:
  name: mypod
spec:
  containers:
  - name: mypod
    image: myimage
    env:
      - name: USERNAME
        valueFrom:
          secretKeyRef:
            name: my-secret
            key: username
      - name: PASSWORD
        valueFrom:
          secretKeyRef:
            name: my-secret
            key: password

This snippet sets the USERNAME and PASSWORD environment variables from your secret.

Managed Secrets Solutions

While managing secrets with native Kubernetes tools is viable, enterprise environments often require more robust solutions like HashiCorp’s Vault, AWS Secrets Manager, or Azure Key Vault. These tools offer sophisticated features such as secret rotation, enhanced auditing, and fine-grained access control.

Integrating such systems with Kubernetes typically involves installing an operator or sidecar container which handles authentication and secret retrieval. The secrets are then dynamically injected into your pods, often as environment variables.

For example, integrating with HashiCorp Vault:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: vault-deployment
spec:
  ...

      containers:
      - name: my-container
        image: myimage
        env:
        - name: VAULT_ADDR
          value: 'https://vault.example.com'
        - name: VAULT_TOKEN
          valueFrom:
            secretKeyRef:
              name: vault-token
              key: token

Here we configure a deployment for accessing the Vault by referencing a token stored as a Secret in Kubernetes.

Encrypting etcd

Since even native Kubernetes Secrets are by default only base64 encoded and not encrypted at rest, it’s recommended to encrypt data within etcd, the Kubernetes backing store, with a strong encryption algorithm.

To encrypt secrets at rest in etcd, you apply a secret encryption config like such:

apiVersion: apiserver.config.k8s.io/v1
kind: EncryptionConfiguration
resources:
  - resources:
      - secrets
    providers:
      - aescbc:
          keys:
            - name: key1
              secret: 
      - identity: {}

aescbc specifies using the AES-CBC encryption algorithm with the particular secret key provided.

Best Practices

Here are some best practices to follow:

• Limit secret access to the least privilege necessary.
• Enable auditing to monitor access and modification of secrets.
• Regularly rotate your secrets.
• Use Read-Only Root Filesystem for your containers to prevent secrets from being written to disk.
• Never hard code secrets in your application code; instead, inject them at runtime.

Conclusion

Securing secrets in Kubernetes requires considering a combination of native Kubernetes features and external tools. By leveraging Secrets, integrating with secret management systems, and following best practices, you can significantly enhance the security posture of your Kubernetes-managed applications.