Hi!

As businesses continue to embrace Kubernetes for container orchestration, the need for efficient resource utilization and cost optimization becomes paramount. Enter Karpenter, an open-source node provisioning project tailored specifically for Kubernetes environments. In this article, we’ll explore how Karpenter can be a game-changer for Azure Kubernetes Service (AKS) users, helping them unlock the full potential of their clusters while minimizing operational costs.

This is achieved through a set of core functionalities:

1. Automated Unschedulable Pod Handling: Karpenter actively monitors the Kubernetes scheduler for pods that have been flagged as unschedulable. This ensures that no resources go to waste, and workloads can be efficiently distributed across the cluster.
2. Dynamic Scheduling Constraints Evaluation: The system meticulously evaluates a range of scheduling constraints specified by the pods. These constraints include resource requests, nodeselectors, affinities, tolerations, and topology spread constraints. By taking these factors into consideration, Karpenter ensures optimal node selection for each workload.
3. Precision Node Provisioning: Karpenter excels in the art of resource allocation. It automatically provisions nodes that precisely align with the specific requirements of the pods. This results in a finely tuned infrastructure that maximizes resource utilization.
4. Automated Node Decommissioning: As workloads evolve, the need for certain nodes may diminish. Karpenter is equipped to intelligently identify when nodes are no longer essential and orchestrates their graceful removal from the cluster. This proactive management ensures that resources are allocated efficiently and are not tied up unnecessarily.

The API for AKS Karpenter Provider is currently alpha (v1alpha2).

Documentation: https://github.com/Azure/karpenter

Maxime.

Hi,

SSH is currently enabled by default for AKS provisioned nodes, and it’s essential to manually disable it if desired. This public preview feature grants you the flexibility to toggle SSH on or off, providing you with greater control over cluster security and reducing potential attack vectors.

To disable SSH for your AKS cluster, you can use the following command:

az aks update --name myAKSCluster --resource-group MyResourceGroup --no-ssh-key

Keep in mind that after updating the SSH key, AKS does not automatically apply the changes to your node pool. You have the option to initiate a node pool update operation at your convenience. It’s worth noting that the updated SSH key will only take effect after a node image update has been successfully completed. This ensures a seamless transition to the new SSH configuration.

Documentation: https://learn.microsoft.com/en-us/azure/aks/manage-ssh-node-access#disable-ssh

Maxime.

Hi!
Azure Container Storage is a specialized cloud-based service designed for managing volumes, deployment, and orchestration in containerized environments. It seamlessly integrates with Kubernetes, enabling automatic provisioning of persistent volumes for stateful applications running on Kubernetes clusters.

Utilizing existing Azure Storage offerings for data storage, Azure Container Storage provides a purpose-built solution for container volume management. It supports various backing storage options, allowing you to create a storage pool for your persistent volumes.

Here’s a summary of the supported storage types, recommended workloads, and provisioning models:

1. Azure Elastic SAN Preview
   • Description: Provisioned on demand, fully managed resource.
   • Workloads: General purpose databases, streaming and messaging services, CD/CI environments, and other tier 1/tier 2 workloads.
   • Offerings: Azure Elastic SAN Preview.
   • Provisioning Model: Provisioned on demand per created volume and volume snapshot. Multiple clusters can access a single SAN concurrently, but persistent volumes can only be attached by one consumer at a time.
2. Azure Disks
   • Description: Offers granular control of storage SKUs and configurations.
   • Workloads: Suitable for tier 1 and general purpose databases like MySQL, MongoDB, and PostgreSQL.
   • Offerings: Premium SSD, Premium SSD v2, Standard SSD, Ultra Disk.
   • Provisioning Model: Provisioned per target container storage pool size and maximum volume size.
3. Ephemeral Disk
   • Description: Utilizes local storage resources on AKS nodes.
   • Workloads: Best for applications with no data durability requirement or with built-in data replication support (e.g., Cassandra).
   • Offerings: NVMe only (available on storage optimized VM SKUs).
   • Provisioning Model: Deployed as part of the VMs hosting an AKS cluster. AKS discovers available ephemeral storage on AKS nodes and acquires them for volume deployment.

In the Azure Container Storage Preview, several capabilities have been introduced based on customer feedback, including:

• Improved stateful application availability with multi-zone storage pools and ZRS disks.
• Server-side encryption with customer-managed keys (Azure Disks only).
• Scale up by resizing volumes backed by Azure Disks and NVMe storage pools without downtime.
• Clone persistent volumes within a storage pool.

Azure Container Storage offers several key benefits:

• Rapid scale out of stateful pods: It provides fast attach and detach of persistent volumes, supporting highly resilient, high-scale stateful applications on AKS.
• Improved performance for stateful workloads: Enables superior read performance and near-disk write performance, meeting various container workload requirements.
• Kubernetes-native volume orchestration: Allows for seamless management of volumes using kubectl commands, eliminating the need to switch between different control plane operations.

In summary, Azure Container Storage streamlines volume management, deployment, and orchestration for containerized applications. It integrates seamlessly with Kubernetes, offering a range of storage options to cater to different workload requirements, ultimately enhancing the reliability and performance of stateful applications on AKS.

# Add the AKS-preview extension with upgrade option
az extension add --name aks-preview --upgrade

# Register necessary providers for AKS and Kubernetes Configuration
az provider register --namespace Microsoft.ContainerService --wait 
az provider register --namespace Microsoft.KubernetesConfiguration --wait

# Create a new resource group
az group create --name <resource-group-name> --location <location>

# Create a new AKS cluster with Azure Container Storage
az aks create -n <cluster-name> -g <resource-group-name> \
--node-vm-size Standard_D4s_v3 --node-count 3 \
--enable-azure-container-storage <storage-pool-type>

# Install Azure Container Storage on an existing AKS cluster
az aks update -n <cluster-name> -g <resource-group-name> \
--enable-azure-container-storage <storage-pool-type>

# Install Azure Container Storage on specific node pools
# First, list the available node pools
az aks nodepool list --resource-group <resource-group-name> --cluster-name <cluster-name>

# Then, update the cluster to enable Azure Container Storage on specific node pools
az aks update -n <cluster-name> -g <resource-group-name> \
--enable-azure-container-storage <storage-pool-type> \
--azure-container-storage-nodepools <comma separated values of nodepool names>

Documentation: https://learn.microsoft.com/en-us/azure/storage/container-storage/container-storage-introduction

Maxime.