Configure Amazon VPC CNI

In this lab exercise, we'll start configuring the Amazon VPC CNI.

Let's review the existing VPC and Availability Zone configuration.

~$echo "The secondary subnet in AZ $AZ1 is $SECONDARY_SUBNET_1"

~$echo "The secondary subnet in AZ $AZ2 is $SECONDARY_SUBNET_2"

~$echo "The secondary subnet in AZ $AZ3 is $SECONDARY_SUBNET_3"

Set the AWS_VPC_K8S_CNI_CUSTOM_NETWORK_CFG environment variable to true in the aws-node DaemonSet.

~$kubectl set env daemonset aws-node -n kube-system AWS_VPC_K8S_CNI_CUSTOM_NETWORK_CFG=true

Create an ENIConfig custom resource for each subnet that you want to deploy pods in. Here is the ENIConfig configuration.

/workspace/modules/networking/custom-networking/provision/eniconfigs.yaml

apiVersion: crd.k8s.amazonaws.com/v1alpha1
kind: ENIConfig
metadata:
  name: ${AZ1}
spec:
  securityGroups:
    - ${EKS_CLUSTER_SECURITY_GROUP_ID}
  subnet: ${SECONDARY_SUBNET_1}
---
apiVersion: crd.k8s.amazonaws.com/v1alpha1
kind: ENIConfig
metadata:
  name: ${AZ2}
spec:
  securityGroups:
    - ${EKS_CLUSTER_SECURITY_GROUP_ID}
  subnet: ${SECONDARY_SUBNET_2}
---
apiVersion: crd.k8s.amazonaws.com/v1alpha1
kind: ENIConfig
metadata:
  name: ${AZ3}
spec:
  securityGroups:
    - ${EKS_CLUSTER_SECURITY_GROUP_ID}
  subnet: ${SECONDARY_SUBNET_3}

Let's apply this to our cluster:

~$envsubst < <(cat /workspace/modules/networking/custom-networking/provision/eniconfigs.yaml) | kubectl apply -f -

Confirm that your ENIConfigs were created.

~$kubectl get ENIConfigs

Update your aws-node DaemonSet to automatically apply the ENIConfig for an Availability Zone to any new Amazon EC2 nodes created in your cluster.

~$kubectl set env daemonset aws-node -n kube-system ENI_CONFIG_LABEL_DEF=topology.kubernetes.io/zone