Confidential Computing with Kubernetes: SEV Guest Protection for Kata Containers

4 min readJun 14, 2021

Support for AMD SEV guest encryption is now available in Kata Containers, enabling for the first time a method for running confidential containers.

“Box 296” by Chewy734 is licensed under CC BY 2.0

In this post, we introduce our patches for enabling SEV protection in kata-containers 2.x and provide instructions for deploying an end-to-end demo of it with containerd and Kubernetes. At the end of this post, “Future Work”, we discuss additional properties that will be necessary to fully realize a model of confidential containers.

This work is part of a broad open-source collaborative that aims to bring confidential computing to container applications. In particular, the added support for SEV included contributions from our team at IBM Research, AMD, Intel, Red Hat, and the Kata open source community.

Confidential Computing & Containers

High-profile security breaches into public and private computer systems illustrate the requirement for better security and protection of highly sensitive workloads at the lowest levels. New confidential computing hardware, such as AMD SEV and Intel TDx, encrypt the working memory of guest virtual machines (VM) instances, protecting the data within from a compromised host.

In today’s cloud, these VM encryption technologies typically require the customer to create, manage, and deploy entire machine instances for the software they want to protect. What is missing is the ability for the customer to leverage hardware-enforced memory encryption with the convenience of cloud-native application models, containers and pods.

Kata Containers is an open-source project that deploys a Kubernetes pod inside of a dedicated VM instance. Recently, a RFC calling for confidential containers in Kata drew a huge response from the community, and a collaboration was quickly formed to explore its possibility. To this collaboration, our team at IBM Research brings expertise in encrypted virtualization, OVMF firmware, remote attestation, and encrypted OCI container images.

Kata Containers with SEV Guest Protection

The remainder of this post will guide you through the steps to setup an end-to-end demo of an SEV-protected kata container with containerd and Kubernetes

Host Requirements

To support this demo, your host system will need the following requirements:

Hardware: A bare metal AMD SEV machine.
Host OS: Linux kernel that has been built and configured to support SEV guests. You can verify a host is compatible with SEV when following command equals 1: $ cat /sys/module/kvm_amd/parameters/sev
Hypervisor: You must run qemu-system-x86_64version 6.0or greater.

Installing Kata & Containerd

SEV confidential guest support is available on the main branch of the Kata Containers repository, and will be included in the 2.2.0-alpha1 release.

To build and install kata-runtime and containerdfrom source, follow these instructions.

Kata Runtime Configuration

Once the kata-runtime and containerd are installed, the following changes to the kata-runtime configuration file will enable SEV guest protection:

First, under [hypervisor.qemu] enable confidential guests:

confidential_guest = true

Under [hypervisor.qemu] set the kernel, initrd, and machine_type as follows:

kernel = "/user/share/kata-containers/vmlinuz-5.*.*-*-sev"
initrd = "/user/share/kata-containers/kata-containers-initrd.img"
machine_type = "q35"

To build and install a guest kernel with built-in support for SEV (i.e., vmlinuz-5.*.*-*-sev in the above example), use Kata’s build_kernel.sh script under ./tools/packaging/kernel :

./build-kernel.sh -x sev setup 
./build-kernel.sh -x sev build 
./build-kernel.sh -x sev install

Under [hypervisor.qemu] set the firmware path to an OVMF binary
(OVMF binary can be installed through your host OS package manager)

firmware = "/usr/share/ovmf/OVMF.fd"

Finally, under [hypervisor.qemu] set the shared filesystem driver to explicitly use virtio-9p instead virtio-fs (which does not yet support SEV encryption)

shared_fs = "virtio-9p"

Deployment Demo

Once the kata-runtime is configured for it is simple to deploy a SEV-protected "sh” shell container using containerd(direct) or through Kubernetes.

containerd (direct)

Deploy a shell using a busybox container image and the ctr tool:

ctr image pull docker.io/library/busybox:latestctr run —-runtime io.containerd.run.kata.v2 -t --rm   docker.io/library/busybox test-container sh

Once the container shell has started you can confirm SEV protection is enabled by checking the kernel message buffer:

/ # dmesg | grep SEV
[ 0.054321] AMD Memory Encryption Features active: SEV

Kubernetes (kubectl)

Support for Kubernetes pods requires the following:

Note: SEV protection will soon be controllable through the Pod spec (via Annotations), allowing SEV protection to be enabled per-Pod and avoiding a machine-level configuration.

Future Work

Despite it being an important first step, SEV protection alone does not guarantee confidentiality with Kata containers. Towards our team’s research goal of providing completely confidential solution, we outline the four properties of a fully confidential container.

Memory Encryption: The entire working memory of the containerized workload should be encrypted in hardware to protect against unauthorized reads/writes by a malicious host.

Remote Attestation & Secret Injection : All software run inside the protected VM must be attested (i.e., measured and externally verified) before confidential information can be safely exposed. Furthermore, retrieving of guest-owner-provided secrets (e.g., AES keys) must happen early enough to be used during the container launch.

Container Image Offload: To further protect sensitive data, container images can be pulled and unpacked within the secure VM enclave. Furthermore, encrypted layers of the OCI container image can only be safely decrypted within the guest VM using keys the guest-owner provided post-attestation.

Hardened Host/Guest Interfaces: Existing methods of the CRI, OCI, and Kata APIs may be incompatible with a confidential computing environment . Therefore, existing APIs must be hardened to prevent a malicious or untrusted host from accessing sensitive data within a container instance.

Contact Us

Find us on Github for updates and information about our contributions to confidential containers from IBM Research. Also, check out Kata’s confidential containers project board for a look at what’s happening across all contributors.