How to Automate Compliance Checks in Kubernetes

Introduction

As Kubernetes (K8s) becomes the de facto standard for container orchestration, ensuring compliance across clusters is a top priority for organizations looking to maintain security and governance standards. Manual compliance checks can be time-consuming and error-prone. Automation is essential for consistency, and it’s critical to implement a process that can repeatedly assess and enforce the desired state of your Kubernetes configurations. In this tutorial, we will explore how to automate compliance checks in Kubernetes using various tools and methodologies. We’ll start with basic examples and advance to more complex scenarios, with code examples and expected outputs to guide you along the way.

Getting Started with Kube-Bench

Kube-Bench is one of the most well-known tools for running Kubernetes security checks. Based on the CIS Kubernetes Benchmark, Kube-Bench audits your clusters against common best practices.

git clone https://github.com/aquasecurity/kube-bench.git
cd kube-bench
kubectl apply -f job.yaml

Once the job is complete, you can view the results by checking the logs of the kube-bench pod:

kubectl logs $(kubectl get pods -l app=kube-bench -o custom-columns=:metadata.name)

This will provide a detailed output of the checks performed, alongside PASS or FAIL notices for each.

Policy-as-Code with Kyverno

Kyverno is a policy engine designed for Kubernetes that allows you to manage, validate, mutate, and generate configurations using policies defined as Kubernetes resources.

helm repo add kyverno https://kyverno.github.io/kyverno/
helm install kyverno kyverno/kyverno --namespace kyverno

Create a Kyverno policy to enforce that all pods have a ‘runAsNonRoot’ security context:

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: require-runasnonroot
spec:
  validationFailureAction: enforce
  rules:
  - name: check-runasnonroot
    match:
      resources:
        kinds:
        - Pod
    validate:
      message: 'SecurityContext `runAsNonRoot` should be set to true.'
      pattern:
        spec:
          containers:
            - =(securityContext):
                =(runAsNonRoot): 'true'

Apply this policy and try to create a pod that does not adhere to it. You should receive an error highlighting non-compliance:

error: failed to create resource: admission webhook "validate.kyverno.svc" denied the request: 

resource Pod/default/podname was blocked due to the following policies

require-runasnonroot:
  check-runasnonroot: 'SecurityContext `runAsNonRoot` should be set to true.'

Continuous Compliance with Open Policy Agent (OPA)

OPA is a general-purpose policy engine that can be used to enforce policies across the Kubernetes API. The Gatekeeper project integrates OPA directly into the Kubernetes admission control process.

kubectl apply -f https://raw.githubusercontent.com/open-policy-agent/gatekeeper/master/deploy/gatekeeper.yaml

Define a ConstraintTemplate and the corresponding Constraint, ensuring that all namespaces must have a specific label:

apiVersion: templates.gatekeeper.sh/v1beta1
kind: ConstraintTemplate
metadata:
  name: k8srequiredlabels
spec:
  crd:
    spec:
      names:
        kind: K8sRequiredLabels
  targets:
    - target: admission.k8s.gatekeeper.sh
      rego: |
        package k8srequiredlabels

        violation[{'msg': msg}] {
          not input.review.object.metadata.labels
          msg := 'You must provide labels'
        }

        violation[{'msg': msg, 'details': {'missing_labels': missing}}] {
          provided := {label | input.review.object.metadata.labels[label]}
          required := {label | label := input.parameters.labels[_]}
          missing := required - provided
          not missing == set()
          msg := 'You must provide all required labels'
        }
  validation:
    openAPIV3Schema:
      properties:
        labels:
          type: array
          items: string

---

apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sRequiredLabels
metadata:
  name: ns-must-have-xyz
spec:
  parameters:
    labels: ["xyz"]

Any attempt to create a namespace without the ‘xyz’ label will result in a denial:

error: failed to create resource: admission webhook "validation.gatekeeper.sh" denied the request: [ns-must-have-xyz] You must provide all required labels

Automating Compliance with Continuous Integration/Continuous Deployment (CI/CD)

Integrating compliance checks into your CI/CD pipeline ensures that non-compliant Kubernetes resources never make it to your clusters. Tools like Terrascan can be hooked into a pipeline to scan Infrastructure as Code configurations.

# Example pipeline step using Terrascan in a GitLab CI/CD pipeline
terrascan_script:
  script:
    - terrascan scan -d kubernetes/ -t k8s
  allow_failure: false

Upon pushing changes to your repository, the CI/CD pipeline will execute Terrascan, scanning for misconfigurations or deviations from policy, and fail the build if it finds any issues.

Conclusion

In this tutorial, we’ve explored various tools and methods for automating compliance checks within your Kubernetes environments. By integrating these practices into your workflows, you will enhance security, streamline operations, and reinforce adherence to regulatory and organizational standards. Remember that compliance is an ongoing effort; continue to review, update, and enforce policies as requirements evolve.