Hi,

I am thrilled to share that I had the opportunity to speak at the CNCF Québec meetup this week. My presentation focused on the various security vectors that a malicious actor may exploit to target an Azure Kubernetes Cluster deployment.

Throughout my talk, I highlighted several critical topics that I believe are crucial for understanding and preventing security breaches, including:

• The exploitation of Azure Resource Graph for discovery purposes
• The risks associated with malicious admission controllers
• Network attacks and their implications
• The importance of understanding the differences between AKS Service Principal and MSI
• Insider attacks and their impact on security.

Maxime.

Hi!,

As cloud-native applications continue to become more complex, managing the communication between services in a microservices architecture becomes a challenge. Service mesh technology offers a solution to this challenge by providing a layer of abstraction for managing network traffic between services. One of the leading service mesh technologies is Istio, and it is now available as an add-on for Azure Kubernetes Service (AKS).

Istio is an open-source service mesh that provides a uniform way to connect, secure, and manage microservices. It provides features such as traffic management, service discovery, load balancing, and security. With Istio, you can manage the communication between services without modifying the application code or infrastructure. Istio is built on top of Envoy, a high-performance proxy that intercepts and routes network traffic between services.

AKS is a managed Kubernetes service that simplifies the deployment, management, and scaling of containerized applications. AKS provides a secure, reliable, and scalable platform for running microservices. However, managing the communication between services can become complex as the number of services and the traffic between them grows. By adding Istio to AKS, you can simplify the management of microservices communication. Istio provides a set of tools and features that make it easier to manage traffic, secure communication, and monitor services. With Istio, you can:

• Control traffic: Istio provides traffic management features such as load balancing, routing, and fault injection. You can configure Istio to control the traffic between services and ensure that the traffic is routed to the correct version of the service.
• Secure communication: Istio provides security features such as authentication, authorization, and encryption. With Istio, you can secure the communication between services without modifying the application code.
• Monitor services: Istio provides monitoring features such as tracing and metrics. With Istio, you can monitor the performance of services and troubleshoot issues.

To use Istio with AKS, you need to enable the Istio add-on when you create an AKS cluster. You can do this by using the Azure portal, Azure CLI, or Azure PowerShell. Once you have enabled the Istio add-on, Istio is deployed as a set of Kubernetes resources in your AKS cluster.

Deploy Istio addon for a new AKS cluster
az extension update --name aks-preview
az feature register --namespace "Microsoft.ContainerService" --name "AzureServiceMeshPreview"
az group create --name ${RESOURCE_GROUP} --location ${LOCATION}
az aks create \
--resource-group ${RESOURCE_GROUP} \
--name ${CLUSTER} \
--enable-asm

Deploy Istio addon for an existing AKS cluster
az extension update --name aks-preview
az feature register --namespace "Microsoft.ContainerService" --name "AzureServiceMeshPreview"
az aks mesh enable --resource-group ${RESOURCE_GROUP} --name ${CLUSTER}
az aks show --resource-group ${RESOURCE_GROUP} --name ${CLUSTER}  --query 'serviceMeshProfile.mode'

Istio-based service mesh add-on for AKS has the following limitations:

• The add-on currently doesn’t work on AKS clusters using Azure CNI Powered by Cilium.
• The add-on doesn’t work on AKS clusters that are using Open Service Mesh addon for AKS.
• The add-on doesn’t work on AKS clusters that have Istio installed on them already outside the add-on installation.
• Managed lifecycle of mesh on how Istio versions are installed and later made available for upgrades.
• Istio doesn’t support Windows Server containers.
• Customization of mesh based on the following custom resources is blocked for now – EnvoyFilter, ProxyConfig, WorkloadEntry, WorkloadGroup, Telemetry, IstioOperator, WasmPlugin

After Istio is deployed, you can configure it using Istio configuration resources such as VirtualService, DestinationRule, and Gateway. These resources allow you to control the traffic between services, apply security policies, and configure load balancing.

Istio-based service mesh add-on for Azure Kubernetes Service provides a powerful tool for managing microservices communication. With Istio, you can simplify the management of traffic, security, and monitoring in your AKS cluster. The Istio add-on for AKS is easy to use and provides a powerful set of features that can help you manage your microservices architecture with ease.

Reference: https://learn.microsoft.com/en-us/azure/aks/istio-about

Maxime.

Hi!

In a traditional Kubernetes cluster, pods share the same node and therefore have the same level of access to the host system. This can lead to potential security risks, particularly if a malicious actor gains access to a vulnerable pod. Pod sandboxing in AKS addresses this issue by creating a dedicated container for each pod, which is isolated from other pods and the host system.

AKS pod sandboxing achieves this isolation by running each pod in its own container, using the gVisor sandboxing technology. gVisor is an open-source sandboxing solution that provides a lightweight, container-based isolation mechanism for running untrusted workloads. This approach enables AKS to provide a secure runtime environment for each pod, without sacrificing performance or scalability.

AKS pod sandboxing also provides a number of other security features, including encryption at rest for pod volumes, network isolation through virtual networks, and integrated identity and access management through Azure Active Directory. These features help to protect sensitive data and prevent unauthorized access to Kubernetes resources.

To use AKS pod sandboxing, users can simply enable the feature when creating a new AKS cluster. Once enabled, all pods deployed to the cluster will be automatically sandboxed, providing an added layer of security and isolation.

In summary, pod sandboxing is an important technique for securing Kubernetes workloads, particularly in multi-tenant environments. AKS pod sandboxing provides a powerful and easy-to-use solution for isolating pods from one another and from the host system, using the lightweight gVisor sandboxing technology. By enabling AKS pod sandboxing, users can improve the security and reliability of their Kubernetes deployments, while maintaining performance and scalability.

How it works:

To achieve this functionality on AKS, Kata Containers running on Mariner AKS Container Host (MACH) stack delivers hardware-enforced isolation. Pod Sandboxing extends the benefits of hardware isolation such as a separate kernel for each Kata pod. Hardware isolation allocates resources for each pod and doesn’t share them with other Kata Containers or namespace containers running on the same host.

The solution architecture is based on the following components:

• Mariner AKS Container Host
• Microsoft Hyper-V Hypervisor
• Azure-tuned Dom0 Linux Kernel
• Open-source Cloud-Hypervisor Virtual Machine Monitor (VMM)
• Integration with Kata Container framework

To use this feature with a pod, the only difference is to add runtimeClassName kata-mshv-vm-isolation to the pod spec.

Limitations:

• Kata containers may not reach the IOPS performance limits that traditional containers can reach on Azure Files and high performance local SSD.
• Microsoft Defender for Containers doesn’t support assessing Kata runtime pods.
• Container Insights doesn’t support monitoring of Kata runtime pods in the preview release.
• Kata host-network isn’t supported.
• AKS does not support Container Storage Interface drivers and Secrets Store CSI driver in this preview release.

Enable Pod Sandboxing to an existing AKS cluster:

az aks nodepool add --cluster-name myAKSCluster --resource-group myResourceGroup --name nodepool2 --os-sku mariner --workload-runtime KataMshvVmIsolation --node-vm-size Standard_D4s_v3

az aks update --name myAKSCluster --resource-group myResourceGroup

Documentation: https://learn.microsoft.com/en-gb/azure/aks/use-pod-sandboxing

Maxime.