Skip to main content
Contact us
Efficient EKS cluster management with AWS Cloud9
Share on socials

Efficient EKS cluster management with AWS Cloud9

Harold (Tre) King
Harold (Tre) King
20 December 2023
7 min read
Man with Kubernetes clusters
Harold (Tre) King
Harold (Tre) King
20 December 2023
7 min read

The problem

Managing multi-tenant EKS clusters can be a complex task. Teams are often faced with the challenge of reliably and efficiently managing these clusters using AWS-native services while maintaining a high level of security. This is no small task. Ensuring reliable access, efficient operations, and robust security across multiple tenants requires a well-designed and well-implemented solution.

Our solution

To address this challenge, our team turned to AWS Cloud9. AWS Cloud9 is a cloud-based integrated development environment (IDE) that lets you write, run, and debug your code with a browser. We employed AWS Cloud9 as a management environment for our EKS clusters. This environment could be securely shared with other IAM roles or IAM users within our AWS account, facilitating collaborative work while maintaining secure access.

How does it work?

Our focus was on optimising for security. To achieve this, we selected the AWS Cloud9 IDE service. This service exists within our AWS account, which means it inherits the existing access controls of our AWS console. This integration provides a secure environment for managing our EKS clusters.
Within the Cloud9 environment, we deploy and manage our EKS clusters. We've also detailed how to leverage EKS's role-based access control (RBAC) to segment tenants by namespace. This segmentation allows for fine-grained control over access and resources, ensuring each tenant can operate independently and securely.

Customer experience improvement

Our customers have derived several benefits from this solution. It includes an EKS management environment optimised for secure access via the AWS console, providing a safe, controlled environment for managing EKS clusters. Additionally, the solution leverages AWS Cloud9's shared access management capabilities for IAM roles and IAM users within an AWS account, enabling collaborative work while maintaining secure access. Using EKS's unique IAM role-based RBAC capabilities for multi-tenant EKS clusters offers fine-grained access control, guaranteeing secure, independent operations for each tenant. The solution also incorporates reusable Infrastructure as Code (IaC) templates for rapid deployment and adaptation, facilitating quick and effortless setup and modification of EKS clusters. Lastly, an EKS solution optimised for security is a part of the package, ensuring all operations are conducted securely, protecting our customers and their data.

Deployment instructions

Prerequisite:

  1. Deploy VPC via CloudFormation template located in this repo: cfn/vpc-2az.yaml
  2. CloudFormation stack name must be vpc-2az
Note: VPC subnet selected for Cloud9 must be publicly available.
  1. Login AWS Console
  2. Create new Cloud9 environment named "eks-management-env"
  3. Allow Cloud9 environment setup to complete

Update Cloud9 IAM Role permissions to allow AdministratorAccess

  1. Use AWS Console and navigate to IAM -> Role -> AWSCloud9SSMAccessRole
  2. Update existing role AWSCloud9SSMAccessRole by adding IAM permission policy AdministratorAccess

Setup - (CloudFormation)

  1. Open Cloud9 environment named "eks-management-env"
  2. In Cloud9, open Preferences -> AWS Settings. Disable the "AWS Manage Temporary Credentials" toggle
  3. In Cloud9, open a terminal session Clone the GitHub repo for this project:

    Unset
    git clone https://github.com/ha-king/eks-cfn.git
  4. cd eks-cfn/cfn
  5. /bin/sh deploy.sh EKS-DEV us-east-1
  6. Visit CloudFormation service to view the stack status, until status value is CREATE_COMPLETE
  7. cd ../cloud9
  8. /bin/sh install_kubectl.sh
  9. /bin/sh update-kubeconfig.sh EKS-DEV us-east-1
  10. Run the kubectl get all - A command to view all Kubernetes resources

    Clean up:
    Unset
    awscloudformationdelete-stack--stack-nameEKS-DEV

Setup - (Terraform)

  1. Open Cloud9 environment named "eks-management-env"
  2. In Cloud9, open Preferences -> AWS Settings. Disable the "AWS Manage Temporary Credentials" toggle
  3. In Cloud9, open a terminal session Clone the GitHub repo for this project:

    Unset
    git clone https://github.com/ha-king/eks-cfn.git
  4. cd eks-cfn/tf
  5. /bin/sh deploy.sh
  6. cd ../cloud9
  7. /bin/sh install_kubectl.sh
  8. /bin/sh update-kubeconfig.sh EKS-DEV-TF us-east-1
  9. Run the kubectl get all - A command to view all Kubernetes resources

    Clean up:
    Unset
    cd tf
    terraformdestroy--auto-approve

Sharing Cloud9 Environment

  1. To invite an IAM user, enter arn:aws:iam::123456789012:user/MyUser. Replace 123456789012 with your AWS account ID and replace MyUser with the name of the user.
  2. To invite a user with an assumed role or a federated user with an assumed role, enter arn:aws:sts::123456789012:assumed-role/MyAssumedRole/MyAssumedRoleSession.
  3. To invite the AWS account root user, enter arn:aws:iam::123456789012:root. Replace 123456789012 with your AWS account ID.
Example:
Unset
awscloud9create-environment-membership--environment-id1234567890987654321--user-arnarn:aws:iam::123456789098:root--permissionsread-write

Setup - EKS Admin IAM entities

  1. kubectl edit cm/aws-auth -n kube-system
  2. Reference the aws-auth configuration map below:

    Unset
    # Please edit the object below. Lines beginning with a '#' will be ignored,
    # and an empty file will abort the edit. If an error occurs while saving this file will be
    # reopened with the relevant failures.
    #
    apiVersion: v1
    data:
      mapRoles: |
        - groups:
          - system:bootstrappers
          - system:nodes
          rolearn: arn:aws:iam::123456789098:role/eks-dev-EksNodeWorkerRole-123456789
          username: system:node:{{EC2PrivateDNSName}}
        - groups:
          - system:masters
          rolearn: arn:aws:iam::123456789098:role/eks-dev-EksRbacAdminRole-123456789
          username: ec2-admin
        - groups:
          - system:masters
          rolearn: arn:aws:iam::123456789098:role/my-role-name
          username: my-admin
      mapUsers: |
        - groups:
          - system:masters
          userarn: arn:aws:iam::123456789098:user/my-user
          username: my-user
    kind: ConfigMap
    metadata:
      creationTimestamp: "2023-05-25T17:32:16Z"
      name: aws-auth
      namespace: kube-system
      resourceVersion: "21552"
    uid:13e8d1d3-c6a1-4369-aff8-e58e94572ad3

EKS RBAC Setup

Prerequisite: Create an IAM Role for this purpose
Notes: This Cloudformation deployment for EKS cluster also creates an EC2 Instance profile, see the Resources tab of CloudFormation
  1. cd ../cloud9
  2. /bin/sh install_eksctl.sh
  3. cd ../rbac
  4. /bin/sh create-rolebindings.sh NAMESPACE EKSCLUSTER NAMESPACE_ROLE_ARN

EKS RBAC Cleanup

  1. cd rbac
  2. /bin/sh delete-rolebindings.sh NAMESPACE EKSCLUSTER ROLE_YAML BINDING_YAML NAMESPACE_ROLE_ARN

References

  1. https://docs.aws.amazon.com/eks/latest/userguide/service_IAM_role.html#create-service-role
  2. https://docs.aws.amazon.com/prescriptive-guidance/latest/patterns/deploy-an-amazon-eks[…]-cluster-from-aws-cloud9-using-an-ec2-instance-profile.html
  3. https://docs.aws.amazon.com/eks/latest/userguide/install-kubectl.html
  4. https://docs.aws.amazon.com/cloud9/latest/user-guide/share-environment.html#share-environment-admin-user
  5. https://repost.aws/knowledge-center/eks-iam-permissions-namespaces

Don't let the complexity of EKS cluster management slow you down. Contact us today to discuss how our AWS solutions can improve your EKS management experience and bolster your security.

Written by
Harold (Tre) King
Harold (Tre) King
Senior DevOps Consultant
Tre is passionately focused on AWS, security, and automation. He excels in solving problems and building innovative solutions, always driven by a commitment to excellence and efficiency.
DevOps
Cloud
AWS