Eight Ways to Create a Pod

September 20, 2019 Eviatar Gerzi

 

Overview

In our previous research post, “Securing Kubernetes Clusters by Eliminating Risky Permissions”, we covered five examples of risky permissions that can be manipulated by an attacker. One of them was the creation of Pods.

In this blog post, we cover to different RBAC resources that allow the eventual creation of Pods.

An attacker with access to a Kubernetes Pod containing a token that allows the creation of new Pods, could create many crypto-mining containers without you even noticing it. One way to mitigate the risk is to leverage Kubernetes Role-Based access Control (RBAC) to restrict token privileges and deny Pod creation. However, to mitigate the risk, you have to do more than not specify the “Pods” resource in the RBAC policy.

A Little Background on Pods, Workloads Metadata and Controllers

In order to understand the different creation methods for a Pod, we need to understand some of the basics of Kubernetes API.

A Pod is a group of one or more containers that are deployed together on the same host.  A Pod is usually used to run applications with several modules inside different containers. Attackers can utilize Pods to run crypto-mining or phishing containers and can also be used to escalate privileges (see example number 2).

Usually, when assigning a user permission to manage Pods (create, delete, modify, etc.), you need to create an RBAC role containing the “Pods” resource. But this is not the only way to assign permissions related to Pods. There are other resources that allow the creation of Pods using trickery.

As stated in Kubernetes API 1.13, workloads are objects that you use to manage and run your containers on the cluster. Workloads use metadata resources, which are the objects used to configure the behavior of other resources within the cluster.

Workloads will eventually run a container, but to run a container, you will need to run a Pod. It is possible to create a Pod as a standalone object. Kubenretes has controllers that can create and manage multiple Pods, handle replication and rollout and provide self-healing capabilities at cluster scope. For example, if we used a deployment (workload object) controller to create a Pod, if the Pod crashes, the deployment controller will start it again and keep restarting it until it runs without crashing.

All the Ways to Create a Container

In the Kubernetes API, we can find the workload object container. This container object is responsible for container creation but can only be created within the context of a Pod object (different workload object).

The container object can only appear in a PodSpec object. A PodSpec object is the “specification of the desired behavior of the Pod,” which means that it defines the properties of the container. The PodSpec YAML below shows the spec field under which the container object is defined.

Figure 1. PodSpec in Pod object YAML

When checking what objects contain the PodSpec field’s properties, we will find two objects: Pod and PodTemplateSpec. The first is a workload object and creating it will lead to the creation of a container. The latter is a metadata object that describes the Pod that will be created.

As we continue reviewing object available in Kubernetes, we find that there are eight workload objects (see green objects in Figure 2), seven controllers and one Pod object, which can create a container. Each of the controllers are discussed below in more details.

In the chart below (Figure 2), we can also see the spec objects (DeploymentSpec, DaemonSetSpec, etc.). The spec objects describe the desired state for the object (the green objects – the controllers). For example, the DaemonSetSpec object describe the desired state for the DaemonSet object.

The chart (Figure 2) shows that the flow from a controller (green) uses a spec object (blue) to define its state, a pattern which continues all the way to the last spec object (PodSpec) and, eventually, to the creation of container object.

Figure 2. mapping of all the objects that lead to container creation

You can get to this conclusion by reviewing all the workloads in the Kubernetes API. But, as you can see from the chart above, there is also a “special” object, called a PodTemplate object, which might also allow Pod creation.

PodTemplate – What are You?

The PodTemplate (orange) is a metadata object contains the specification (PodTemplateSpec) of a Pod. This specification is included in other objects such as Replication Controllers, Jobs and DaemonSets. Controllers use PodTemplate object to make the actual Pod.

Notice that the chart in Figure 2 shows that this object as an outsider because it is not a controller and doesn’t have the fields necessary to create a container object like the controllers’ objects. At the time of writing this, you cannot use PodTemplate to create Pods directly or by referencing its name in an object. You can only use it to create a Pod template object (see Code snippet 1). If, in the future, it becomes possible to reference the PodTemplate object by its name within objects, for example, ReplicaSet it will allow the creation of a Pod indirectly and then we will have to be aware of it.

apiVersion: v1

kind: PodTemplate

metadata:

name: pod-by-podtemplate

namespace: default

template:

metadata:

name: pod-template

spec:

containers:

- name: container

image: alpine

command: ["/bin/sh"]

args: ["-c", "echo \"malicious code\"; sleep 100"]

Figure 3. PodTemplate YAMLEight Ways to Create a Pod

We now list eight different API objects that, if included in a RBAC policy assigned to a user, will allow the user to create Pods.

1. Pod

The most trivial way to create a Pod is by using the “Pod” object. With this method, the Pod will be created as is and, if it crashes, it won’t start again automatically.

 

kind: Pod

metadata:

name: pod-by-pod

namespace: default

spec:

containers:

- name: container

image: alpine

command: ["/bin/sh"]

args: ["-c", "echo \"malicious code\"; sleep 100"]

Figure 4. Pod YAML

2. ReplicationController

This controller ensures that there are a specific number of Pod replicas running at any time and, if a Pod does crash, the ReplicationController restarts it. This is the original form of replication in Kubernetes and, while it’s being replaced by ReplicaSets, it is still in wide use.

When this controller is created, it will create Pods based on the number of replicas.

apiVersion: v1
kind: ReplicationController
metadata:
  name: pod-by-replicationcontroller
spec:
  replicas: 3
  template:
    metadata:
      name: pod-template
      namespace: default
      labels:
        name: selector
    spec:
      containers:
      - name: container
        image: alpine
        command: ["/bin/sh"]
        args: ["-c", "echo \"malicious code\"; sleep 100"]

Figure 5. ReplicationController YAML

3. ReplicaSet

This controller orchestrates individual Pod lifecycle and updates. It is a sort of hybrid; in some ways ReplicaSets are more powerful (i.e. more options for the selector field) than ReplicationControllers and, in other ways, they are less powerful.

It has the same behavior as the ReplicationController. When this controller is created, it creates Pods based on the number of replicas.

apiVersion: apps/v1
kind: ReplicaSet
metadata:
  name: pod-by-replicaset
spec:
  replicas: 3
  selector:
    matchLabels:
      name: selector
  template:
    metadata:
      name: pod-template
      namespace: default
      labels:
        name: selector
    spec:
      containers:
      - name: container
        image: alpine
        command: ["/bin/sh"]
        args: ["-c", "echo \"malicious code\"; sleep 100"]

Figure 6. ReplicaSet YAML

4. Deployment

This controller manages the state of ReplicaSets and the Pods within it. It is intended to replace ReplicationControllers. It provides the same function (through ReplicaSet) and also the ability to rollout changes and roll them back if necessary.

When it is created, it creates a ReplicaSet, which will create a Pod based on the number of replicas.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: pod-by-deployment
  namespace: default
spec:
  selector:
    matchLabels:
      name: selector
  replicas: 3
  template:
    metadata:
      labels:
        name: selector
    spec:
      containers:
      - name: container
        image: alpine
        command: ["/bin/sh"]
        args: ["-c", "echo \"malicious code\"; sleep 100"]

Figure 7. Deployment YAML

5. DaemonSet

This controller ensures that a single Pod exists on each node in the cluster.

When this controller is created, it creates a Pod on each node in the cluster.

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: pod-by-daemonset
  namespace: default
spec:
  selector:
    matchLabels:
      name: selector
  template:
    metadata:
      labels:
        name: selector
    spec:
      containers:
      - name: container
        image: alpine
        command: ["/bin/sh"]
        args: ["-c", "echo \"malicious code\"; sleep 100"]

Figure 8. DaemonSet YAML

6. Job

This controller creates one or more Pods and ensures that a specified number of them successfully terminate. If a Pod fails, it will be restarted until the number of completions is reached.

When this controller is created, it creates a Pod.

apiVersion: batch/v1
kind: Job
metadata:
  name: pod-by-job
  namespace: default
spec:
  template:
    spec:
      containers:
      - name: container
        image: alpine
        command: ["/bin/sh"]
        args: ["-c", "echo \"malicious code\"; sleep 100"]
      restartPolicy: Never

Figure 9. Job YAML

7. CronJob

This controller creates Jobs on a time-based schedule. It runs a Job written in Cron format periodically on a given schedule.

When this controller is created, it creates a Pod periodically on a given schedule.

apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: pod-by-cronjob
  namespace: default
spec:
  schedule: "*/1 * * * *"
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: container
            image: alpine
            command: ["/bin/sh"]
            args: ["-c", "echo \"malicious code\""]
          restartPolicy: Never

Figure 10. CronJob YAML that creates a Pod every minute

The YAML in Figure 10 above will create new Pod every minute.

Figure 11. Pods created by a CronJob every minute

8. StatefulSet

This controller manages stateful applications, which are programs that save client data from the activities of one session for use in the next session. It manages the deployment and scaling of a set of Pods, considers each Pod as unique and provides order to Pod deployment. StatefulSet ensures that the next Pod will not launch until the current Pod reaches a running and ready state.

When it is created, it creates a Pod.

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: pod-by-statefulset
spec:
  replicas: 3
  serviceName: ""
  selector:
    matchLabels:
      name: selector
  template:
    metadata:
      labels:
        name: selector
    spec:
      containers:
      - name: container
        image: alpine
        command: ["/bin/sh"]
        args: ["-c", "echo \"malicious code\"; sleep 100"]

Figure 12. StatefulSet YAML that creates 3 Pods

Conclusion

We looked at eight different ways to create a Pod:

Figure 13 – Summary of the resources that can create a Pod

The ability to create a Pod is risky and, when you need to provide permissions without allowing the creation and modification of a Pod, you need to make sure that none of the eight resources in Figure 13 above appear in those permissions.

If the cluster you manage and secure is large and you wish to scan it to detect risky permissions, including the eight different resources above, please take a look at the open source tool we created, KubiScan.

Reference

https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.13/

https://www.mirantis.com/blog/kubernetes-replication-controller-replica-set-and-deployments-understanding-replication-options/

 

 

Previous Article
AnsibleFest: CyberArk Presents Ansible Tower Integration
AnsibleFest: CyberArk Presents Ansible Tower Integration

Several members of the CyberArk team recently returned from AnsibleFest 2019, a four-day conference focused...

Next Article
Kubernetes Pentest Methodology Part 2
Kubernetes Pentest Methodology Part 2

Attacking the Cluster Remotely In our previous blog post “Kubernetes Pentest Methodology Part 1”, we wrote ...