Security Groups for Pods on Amazon EKS

Good morning, everyone. My name is Parth Vyas. First question is what are security groups for pods? Security groups for pods as the name suggest it will assign an elastic network interface to the service accounts, and then via the service account pod will assign a security group. For the security purpose, and this is one of the best practices that we should follow on the EKS, which is the network traffic management. For that purpose, security groups for pods are used. One of the scenarios is we have databases running in the private subnets. Our application requires exposing the pod to the public, and we are running some third-party software inside that pod. As it will have third-party softwares, and we don’t know how someone can deploy, it means the third-party can deploy some malicious software inside it and run it. To protect our infrastructure the best practice is we should assign security groups for each pod group. Only the applications that require the database access, only that pod should assign the separate security group, and only they can access the database. That is one of the scenarios where the security groups for pods are very useful. Why do we want to choose the security groups for pods and not any other tools? Security groups for pods offer AWS native ways to control network traffic from inside and outside the cluster. The same thing if we want to achieve, we have to use other networking plugins for the Kubernetes for example, Cilium, Calico. For the Calico, we have to go with the enterprise version, which comes at a cost. For the small organization where we don’t want the large management head, so for that kind of scenario, security groups for pods are very useful. Where can we find it? I’m going to give a demo on Security Groups For Pods on EKS. This particular feature works for EKS only. It does not work on any other Kubernetes clusters. Let’s get started.

As I said, it is provided in the AWS best practices URL, and it’s also provided in the Amazon blogs. It is very easy to follow.

These are the URLs and now let’s get to the demo. First I will move forward. What are the things needed for this to work? First thing is EKS cluster should be there, and by default EKS comes with the VPC CNI, and the security groups for pods are assigning ENI to the service accounts, and for that, we need to modify the default VPC CNI. With the Helm Chart, we cannot do that because AWS by default deploys it with the Kubernetes manifest file, so it does not have Helm Chart annotation. For that, we have to run this shell script. This shell script finds each and every resource, the name for the VPC CNI, and it will annotate the resources with the AWS node, and it will annotate with the Helm Chart. Later we can manage the VPC CNI with the Helm Chart. These particular annotations and the label will help us to manage it by a Helm Chart. First, we need to run this shell script. Once we run this shell script we can run the Helm Chart, VPC CNI, and I will talk about what exactly we need to apply for it to run. First thing is, the VPC CNI image should be greater than the 1.7.0 version. At the time 1.7.7 was the latest version, so I used it. It is also one of the best practices, we should pin down older versions. It’s not a good thing to use the latest tech everywhere because, with the latest new releases, it will stop working. Another thing is we have to disable the TCP only Linux, and set it [?] and enable the port ENI equal to two. These two things are needed in order to work security groups for pods, and for this purpose, we need to deploy it with a Helm Chart and update the VPC CNI. Here I have created a scaffold configuration and provided the VPC CNI Helm Chart, and currently, it is already deployed. Once you update the VPC CNI, then you have to create this service account which you will assign to a security group. This is the service account example that I have been created with the name and labels. Here I have followed [?] equal to SG-Group-For-Pods and this will create a service account in default namespace. After that, a major thing is we need to create a security group policy. As you can see, it’s a custom resource and it comes from VPC CNI. If you don’t have VPC CNI deployed, it will not work. Here, I have provided the service account selector, our service account name, and here we have to bind the security groups with the service account. Before I have created these pods for the demo, they are a security group just for the Pods, and currently, I have followed the AWS module or input library module, so it will have the shell rule for the inbound and outbound. That way, I can bind this security group to the service account. The last thing is we need to verify that our security group is attached to the Pod or not. Whenever we attach a security group to Pod, the Pod will lose all the networking aspects with the internet. For example, our Pods are running behind the private subnets and behind the NET gateway. By default, it should be able to access the internet and visualize the DNS, but whenever we assign a security group, it will lose its functionality, it will not be able to resolve any DNS names. That thing, I’m going to demonstrate that. First I will verify I am on the correct cluster AES. In this current file, our Pod is utilizing the Security Groups. First, I will remove it, then I will [?] what the difference we are making, and what are the things we need to do in order to work it very well. Currently, I have deployed the Pod without the security group, so it will utilize the security groups from worker nodes. Now if I go inside the Pod, if I do the yum update, it will work. As you can see, we are able to download the packages and update the repositories. I will delete this Pod. I will uncomment this file. Now I will apply it to the security group. As I told earlier, when we assign a security group to a pod, it will lose the DNS resolution and any other connectivity with the internet. The most usable required Pod is the DNS. Without DNS we are not able to download any package or update any repositories as you can see. I’m going to stop it here and apply the security group rules. As you can see, we are getting the error, could not resolve the hostnames. I will go to the management console, copy the security group for the pod, and then I will modify this security group which is EKS worker group and I will apply the DNS rules. In order to work, I need to allow DNS, UDP, and TCP from our pod security group because our Pod will send requests to the worker not about the DNS resolution, so it is required. We have applied the rule and now I will go here, and if I do yum update. As you can see, it starts downloading the package. It is now able to recall the DNS Pods’ names. That’s it for the case demo. Thank you, everyone.