Kubernetes Security using AppArmor: A Case Study

Introduction:-In last few month I got an opportunity to work with Kubernetes Security platform and I found that Open-Source community is continuously enhancing the Kubernetes security by applying various Open Source tools,which provides not only a secure platform to deploy micro services but also provides broad scope to development in terms of security

Scope:-The scope of this document is to provide an introduction of tool called AppArmor. I will also try to explain an use case which depicts the feature of AppArmor over application pod security.

AppArmor:-AppArmor is a Linux kernel security module that supplements the standard Linux user and group based permissions to confine programs to a limited set of resources. AppArmor can be configured for any application to reduce its potential attack surface and provide greater in-depth defense.

It is configured through profiles tuned to allow the access needed by a specific program or container, such as Linux capabilities, network access, file permissions, etc. Each profile can be run in either enforcing mode, which blocks access to disallowed resources, or complain mode, which only reports violation.

Before work with AppArmor we should have some working concepts of some of the Kubernetes objects

DaemonSet:-

The DaemonSet feature is used to ensure that some or all of your pods are scheduled and running on every single available node. This essentially runs a copy of the desired pod across all nodes. When a new node is added to a Kubernetes cluster, a new pod will be added to that newly attached node

As AppArmor will work perform scrutiny on each and every worker node so it need to be deployed as daemonset.

ConfigMap:-

A ConfigMap is an API object used to store non-confidential data in key-value pairs. Pods can consume Config-maps as environment variables, command-line arguments, or as configuration files in a volume

Here config-map is used toad the AppArmor profile to the pod. I am thankful to OpenSource community member to provide a tool to load AppArmor profile

GitHub:- https://github.com/kubernetes/kubernetes/tree/master/test/images/apparmor-loader

AppArmor Workflow

The above picture depicts an use case of AppArmor. Following are the steps how AppArmor are working in this scenario

1. AppArmor Daemon-Set Pods loads the profiles i.e Config-map ( Restriction Rules) into each of the worker node

2. The Pod manifest loads the config-map(k8s-nginx) via annotation

Daemon Set AppDset.yaml :-

ConfigMap(k8s-nginx) AppArCmap.yaml:-

AppArmor Rule:-

  1. Allow communication of tcp/udp,icmp socket
  2. Deny raw socker and packet
  3. Deny writing in /tmp,/etc, /boot mount points of the worker node
  4. Audit checking in / mount points
  5. Allow /var/run/naginxpid file writing
  6. Allow /usr/sbin/nginx file execute

POD Manifest :-

The POD definition is self explanatory , only exception is adding AppArpmor profile through annotation.

AppArmor Installation:-

To install AppArmor Below three commands need to be executed

  1. kubectl create namespace apparmor
  2. kubectl create -f AppArCmap.yaml
  3. kubectl create -f AppDset.yaml

AppArmor Pod Status:-

Deploy POD with AppArmor profile:-

kubectl apply -f SamplePod.yaml

As it can seen by describing the pod the profile is loaded

Let us now try to write something inside workernode /tmp directory though the privilege pod.

As it can seen that as the AppArmor having a rule to deny access inside /tmp directory, the error message shown that permission denied.

So this way we can conclude that even the Pod is having root privileged access , but still one cannot write anything inside worker node /tmp directory as per AppArmor rules ( as we know that container/POD ) share and use host’ s/ workernode’s file system.

Conclusion:-AppArmor is a very good tool and very granular level security can be imposed through it. AppArmor can be installed/use in the same manner irrespective of Managed(GKE,AKS,EKS,ARO) and UnManaged(Openshift,Ranchar) Kubernetes.

13 + years of experience on system integratiin on Linux and 4 years in cloud/devops & ansible LinkedIn:-https://www.linkedin.com/in/indranil-banerjee-894a5016