Hi!

In a traditional Kubernetes cluster, pods share the same node and therefore have the same level of access to the host system. This can lead to potential security risks, particularly if a malicious actor gains access to a vulnerable pod. Pod sandboxing in AKS addresses this issue by creating a dedicated container for each pod, which is isolated from other pods and the host system.

AKS pod sandboxing achieves this isolation by running each pod in its own container, using the gVisor sandboxing technology. gVisor is an open-source sandboxing solution that provides a lightweight, container-based isolation mechanism for running untrusted workloads. This approach enables AKS to provide a secure runtime environment for each pod, without sacrificing performance or scalability.

AKS pod sandboxing also provides a number of other security features, including encryption at rest for pod volumes, network isolation through virtual networks, and integrated identity and access management through Azure Active Directory. These features help to protect sensitive data and prevent unauthorized access to Kubernetes resources.

To use AKS pod sandboxing, users can simply enable the feature when creating a new AKS cluster. Once enabled, all pods deployed to the cluster will be automatically sandboxed, providing an added layer of security and isolation.

In summary, pod sandboxing is an important technique for securing Kubernetes workloads, particularly in multi-tenant environments. AKS pod sandboxing provides a powerful and easy-to-use solution for isolating pods from one another and from the host system, using the lightweight gVisor sandboxing technology. By enabling AKS pod sandboxing, users can improve the security and reliability of their Kubernetes deployments, while maintaining performance and scalability.

How it works:

To achieve this functionality on AKS, Kata Containers running on Mariner AKS Container Host (MACH) stack delivers hardware-enforced isolation. Pod Sandboxing extends the benefits of hardware isolation such as a separate kernel for each Kata pod. Hardware isolation allocates resources for each pod and doesn’t share them with other Kata Containers or namespace containers running on the same host.

The solution architecture is based on the following components:

• Mariner AKS Container Host
• Microsoft Hyper-V Hypervisor
• Azure-tuned Dom0 Linux Kernel
• Open-source Cloud-Hypervisor Virtual Machine Monitor (VMM)
• Integration with Kata Container framework

To use this feature with a pod, the only difference is to add runtimeClassName kata-mshv-vm-isolation to the pod spec.

Limitations:

• Kata containers may not reach the IOPS performance limits that traditional containers can reach on Azure Files and high performance local SSD.
• Microsoft Defender for Containers doesn’t support assessing Kata runtime pods.
• Container Insights doesn’t support monitoring of Kata runtime pods in the preview release.
• Kata host-network isn’t supported.
• AKS does not support Container Storage Interface drivers and Secrets Store CSI driver in this preview release.

Enable Pod Sandboxing to an existing AKS cluster:

az aks nodepool add --cluster-name myAKSCluster --resource-group myResourceGroup --name nodepool2 --os-sku mariner --workload-runtime KataMshvVmIsolation --node-vm-size Standard_D4s_v3

az aks update --name myAKSCluster --resource-group myResourceGroup

Documentation: https://learn.microsoft.com/en-gb/azure/aks/use-pod-sandboxing

Maxime.

Hi!

Azure Kubernetes Service (AKS) is now offering two pricing tiers for cluster management: the Free tier and the Standard tier.

	Free tier	Standard tier
When to use	• You want to experiment with AKS at no extra cost • You’re new to AKS and Kubernetes	• You’re running production or mission-critical workloads and need high availability and reliability • You need a financially backed SLA
Supported cluster types	• Development clusters or small scale testing environments • Clusters with fewer than 10 nodes	• Enterprise-grade or production workloads • Clusters with up to 5,000 nodes
Pricing	• Free cluster management • Pay-as-you-go for resources you consume	• Pay-as-you-go for resources you consume
Feature comparison	• Recommended for clusters with fewer than 10 nodes, but can support up to 1,000 nodes • Includes all current AKS features	• Uptime SLA is enabled by default • Greater cluster reliability and resources • Can support up to 5,000 nodes in a cluster • Includes all current AKS features

# Create a new AKS cluster in the Free tier 
az aks create --resource-group myResourceGroup --name myAKSCluster --no-uptime-sla 

# Create a new AKS cluster in the Standard tier 
az aks create --resource-group myResourceGroup --name myAKSCluster --uptime-sla

# Update an existing cluster to the Free tier
az aks update --resource-group myResourceGroup --name myAKSCluster --no-uptime-sla

# Update an existing cluster to the Standard tier
az aks update --resource-group myResourceGroup --name myAKSCluster --uptime-sla

Documentation: https://learn.microsoft.com/en-us/azure/aks/free-standard-pricing-tiers

Maxime.

Hi!

Mariner is an open-source Linux distribution created by Microsoft and is now available for preview as a container host on Azure Kubernetes Service (AKS). Optimized for AKS, the Mariner container host provides reliability and consistency from cloud to edge across the AKS, AKS-HCI, and Arc products. You can deploy Mariner node pools in a new cluster, add Mariner node pools to your existing Ubuntu clusters, or migrate your Ubuntu nodes to Mariner nodes.

Key Capabilities Of CBL-Mariner:

CBL-Mariner provides many of the traditional benefits of using Linux. In addition to that, CBL-Mariner provides hardened security and efficient lifecycle management.

• CBL-Mariner core

Minimal core system that supports a variety of profiles (Azure VM or on bare-metal x64 or ARM64) and allows the customer to build on top of it as needed.

Lightweight footprint: 450MB uncompressed.

• Support & Updates

SLA for vulnerabilities.

Patches automatically available for the customer to update when most convenient for them.

dnf infrastructure used for upgrading packages.

• Security hardened

The kernel and other aspects of the OS are built with an emphasis on security and follow the secure-by-default principle, compliant with Microsoft security standards and industry certifications.

• Federated Builds

Enables teams to innovate on top by allowing the generation and maintenance of packages on top of the CBL-Mariner builds.

With over 6000 packages already built, teams can customize their image easily.

• Robust Testing

Through a robust testing matrix of package, image and kernel tests, we allow for earlier issue detections and mitigations prior to the image being published.

• Virtualization

CBL-Mariner supports a container host image that includes the Kubernetes infrastructure.

• Efficient lifecycle management

CBL-Mariner supports both RPM package and image-based update mechanisms for releases – with an “evergreen” release alongside specific security-patched stable snaps. New releases are made available annually and each release is supported for 18 months.

If you want to replace your existing AKS Ubuntu Node Pool by Mariner, please follow this link: https://microsoft.github.io/CBL-Mariner/docs/#using-cbl-mariner-with-aks

Maxime.