feat: Multi-cluster architecture to increase resiliency and reduce inter-az data transfer charges #1802

ashoksrirama · 2023-10-23T14:48:30Z

Description

Motivation and Context

Resolves Multi-cluster architecture to increase resiliency and reduce inter-az data transfer charges #1765

How was this change tested?

Yes, I have tested the PR using my local account setup (Provide any test evidence report under Additional Notes)
Yes, I have updated the docs for this feature
Yes, I ran pre-commit run -a with this PR

Additional Notes

bryantbiggs · 2023-11-04T13:37:32Z

patterns/cell-based-eks/0.vpc/main.tf

+data "aws_availability_zones" "available" {}
+
+locals {
+  cluster_name = format("%s-%s", basename(path.cwd), "shared")


Lets follow the current norm from what is used in other patterns:

Suggested change

cluster_name = format("%s-%s", basename(path.cwd), "shared")

name = basename(path.cwd)

bryantbiggs · 2023-11-04T13:37:40Z

patterns/cell-based-eks/0.vpc/main.tf

+  azs      = slice(data.aws_availability_zones.available.names, 0, 3)
+
+  tags = {
+    Blueprint  = local.cluster_name


Suggested change

Blueprint = local.cluster_name

Blueprint = local.name

bryantbiggs · 2023-11-04T13:37:49Z

patterns/cell-based-eks/0.vpc/main.tf

+  source  = "terraform-aws-modules/vpc/aws"
+  version = "~> 5.0"
+
+  name = local.cluster_name


Suggested change

name = local.cluster_name

name = local.name

bryantbiggs · 2023-11-04T13:39:39Z

patterns/cell-based-eks/0.vpc/main.tf

@@ -0,0 +1,47 @@
+provider "aws" {


I don't believe the cluster-per-AZ design requires splitting up the Terraform configurations into multiple directories. We should collapse this back down to a single directory, but have multiple cluster definitions - one for each AZ used. This can be shown with a set of definitions split into multiple files - for example:

az1.tf

az1.yaml

az2.tf

az2.yaml

az3.tf

az3.yaml

Within each of these AZ specific Terraform files we'll have:

EKS cluster definition

Addons definition

Kubernetes and Helm aliased providers scoped to that cluster and addon definition

Then within each of the AZ specific YAML files will be the Karpenter specific manifests for that AZ and cluster within that AZ.

Thoughts?

Done, made changes as suggested. We were following the istio multi-cluster pattern structure before.

bryantbiggs · 2023-11-04T13:40:33Z

patterns/cell-based-eks/1.cell1/README.md

+
+This example shows how to provision a cell based Amazon EKS cluster.
+
+* Deploy EKS Cluster with one managed node group in a VPC and AZ


what is the motivation for mixing Fargate, managed nodegroup, and Karpenter in this design?

It was about showing how to use them in single AZ pattern. Removed the Fargate and using 1 managed node group + Karpenter now.

bryantbiggs · 2023-11-04T13:41:02Z

patterns/cell-based-eks/1.cell1/README.md

+3. [terraform](https://learn.hashicorp.com/tutorials/terraform/install-cli)
+4. [helm](https://helm.sh/docs/helm/helm_install/)
+
+## Deploy


Please see the other pattern readmes for the "standard" README structure

bryantbiggs · 2023-11-04T13:43:06Z

patterns/cell-based-eks/1.cell1/main.tf

+  }
+}
+
+provider "kubectl" {


we are actively moving away from this provider #1675

You can see the start of this in #1819 which also includes updates for Karpenter 0.32

Removed the use of kubectl provider and added instructions in README.md to install the karpenter and sample app.

bryantbiggs · 2023-11-04T13:44:27Z

patterns/cell-based-eks/1.cell1/main.tf

+# Karpenter
+################################################################################
+
+resource "aws_security_group" "karpenter_sg" {


are these Karpenter security group resources required?

Nope, cleaned them up.

bryantbiggs · 2023-11-04T13:44:56Z

patterns/cell-based-eks/1.cell1/main.tf

+
+resource "kubectl_manifest" "karpenter_provisioner" {
+  yaml_body = <<-YAML
+    apiVersion: karpenter.sh/v1alpha5


we'll want to use v0.32 and v1beta1

bryantbiggs · 2023-11-04T13:45:53Z

patterns/cell-based-eks/1.cell1/variables.tf

@@ -0,0 +1,12 @@
+
+variable "name" {


we don't provide variables unless we absolutely require something from users to deploy (i.e. - a domain name or hosted zone) - these are not consumed in place, they are references

Removed the references

github-actions · 2023-12-05T00:10:04Z

This PR has been automatically marked as stale because it has been open 30 days
with no activity. Remove stale label or comment or this PR will be closed in 10 days

ashoksrirama · 2023-12-05T22:20:31Z

I'll work on the changes and get back with an updated PR.

ashoksrirama · 2023-12-27T15:34:08Z

Updates made based on the above feedback.

github-actions · 2024-01-27T00:09:07Z

This PR has been automatically marked as stale because it has been open 30 days
with no activity. Remove stale label or comment or this PR will be closed in 10 days

ashoksrirama · 2024-01-29T13:29:59Z

A gentle reminder to review this.

github-actions · 2024-03-01T00:10:58Z

This PR has been automatically marked as stale because it has been open 30 days
with no activity. Remove stale label or comment or this PR will be closed in 10 days

github-actions · 2024-03-12T00:09:23Z

Pull request closed due to inactivity.

ashoksrirama · 2024-04-11T21:36:20Z

Another gentle reminder to re-open this PR and review the latest changes.

allamand · 2024-04-14T06:28:00Z

Hi @ashoksrirama, so in this pattern we create one EKS cluster per az and you deploy an app in each cluster. I am not sure to understand what new here and what needs another pattern? The title is multi-cluster with objective to reduce inter-az costs, so basically you do it just by having 3 clusters. What about the complexity of managing 3 agains 1 cluster ? What if you deploy microservices that needs to talk each other, how you ensure high availability in case of problem on 1 az ? I think I’m missing something here

ashoksrirama added 3 commits October 19, 2023 13:04

Add Cell based EKS Cluster Pattern

ec607e2

Merge branch 'aws-ia:main' into main

2d06070

Updates to ReadMe

d8c4203

ashoksrirama requested a review from a team as a code owner October 23, 2023 14:48

ashoksrirama had a problem deploying to EKS Blueprints Test October 23, 2023 14:48 — with GitHub Actions Failure

ashoksrirama changed the title ~~[feat] Multi-cluster architecture to increase resiliency and reduce inter-az data transfer charges #1765~~ feat: Multi-cluster architecture to increase resiliency and reduce inter-az data transfer charges #1765 Oct 23, 2023

ashoksrirama changed the title ~~feat: Multi-cluster architecture to increase resiliency and reduce inter-az data transfer charges #1765~~ feat: Multi-cluster architecture to increase resiliency and reduce inter-az data transfer charges Oct 23, 2023

bryantbiggs reviewed Nov 4, 2023

View reviewed changes

github-actions bot added the stale label Dec 5, 2023

github-actions bot removed the stale label Dec 6, 2023

ashoksrirama added 2 commits December 22, 2023 23:12

Merge branch 'aws-ia:main' into main

18de596

Updates based on the PR feedback

03e0ed6

github-actions bot added the stale label Jan 27, 2024

github-actions bot removed the stale label Jan 30, 2024

github-actions bot added the stale label Mar 1, 2024

github-actions bot closed this Mar 12, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Multi-cluster architecture to increase resiliency and reduce inter-az data transfer charges #1802

feat: Multi-cluster architecture to increase resiliency and reduce inter-az data transfer charges #1802

ashoksrirama commented Oct 23, 2023 •

edited

bryantbiggs Nov 4, 2023

ashoksrirama Dec 27, 2023

bryantbiggs Nov 4, 2023

ashoksrirama Dec 27, 2023

bryantbiggs Nov 4, 2023

ashoksrirama Dec 27, 2023

bryantbiggs Nov 4, 2023

ashoksrirama Dec 27, 2023

bryantbiggs Nov 4, 2023

ashoksrirama Dec 27, 2023

bryantbiggs Nov 4, 2023

ashoksrirama Dec 27, 2023

bryantbiggs Nov 4, 2023

ashoksrirama Dec 27, 2023

bryantbiggs Nov 4, 2023

ashoksrirama Dec 27, 2023

bryantbiggs Nov 4, 2023

ashoksrirama Dec 27, 2023

bryantbiggs Nov 4, 2023

ashoksrirama Dec 27, 2023

github-actions bot commented Dec 5, 2023

ashoksrirama commented Dec 5, 2023

ashoksrirama commented Dec 27, 2023

github-actions bot commented Jan 27, 2024

ashoksrirama commented Jan 29, 2024

github-actions bot commented Mar 1, 2024

github-actions bot commented Mar 12, 2024

ashoksrirama commented Apr 11, 2024 •

edited

allamand commented Apr 14, 2024

	cluster_name = format("%s-%s", basename(path.cwd), "shared")
	name = basename(path.cwd)


		This example shows how to provision a cell based Amazon EKS cluster.

		* Deploy EKS Cluster with one managed node group in a VPC and AZ

feat: Multi-cluster architecture to increase resiliency and reduce inter-az data transfer charges #1802

feat: Multi-cluster architecture to increase resiliency and reduce inter-az data transfer charges #1802

Conversation

ashoksrirama commented Oct 23, 2023 • edited

Description

Motivation and Context

How was this change tested?

Additional Notes

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Dec 5, 2023

ashoksrirama commented Dec 5, 2023

ashoksrirama commented Dec 27, 2023

github-actions bot commented Jan 27, 2024

ashoksrirama commented Jan 29, 2024

github-actions bot commented Mar 1, 2024

github-actions bot commented Mar 12, 2024

ashoksrirama commented Apr 11, 2024 • edited

allamand commented Apr 14, 2024

ashoksrirama commented Oct 23, 2023 •

edited

ashoksrirama commented Apr 11, 2024 •

edited