Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[EKS] Add nginx and redis fargate workloads #757

Open
wants to merge 18 commits into
base: main
Choose a base branch
from

Conversation

jennchenn
Copy link
Member

What does this PR do?

Add workloads for testing EKS fargate. These new apps make use of the agent sidecar auto-injection feature.

Which scenarios this will impact?

EKS Fargate

Motivation

Continue building out existing container-integrations e2e test suite.

Additional Notes

Preview:

Previewing update (jennifer-chen-aws-eks):
     Type                                                                 Name                                       Plan       
 +   pulumi:pulumi:Stack                                                  dd-jennifer-chen-aws-eks                   create     
 +   ├─ pulumi:providers:random                                           random                                     create     
 +   ├─ dd:Cluster                                                        aws-eks                                    create     
 +   ├─ pulumi:providers:aws                                              aws                                        create     
 +   ├─ random:index:RandomShuffle                                        aws-rnd-subnet                             create     
 +   ├─ aws:ec2:SecurityGroup                                             aws-eks-sg                                 create     
 +   ├─ aws:iam:Role                                                      aws-eks-cluster-role                       create     
 +   ├─ aws:iam:Role                                                      aws-eks-linux-node-role                    create     
 +   ├─ aws:iam:Role                                                      aws-eks-windows-node-role                  create     
 +   ├─ pulumi:providers:eks                                              eks                                        create     
 +   ├─ eks:index:Cluster                                                 aws-eks                                    create     
 +   │  ├─ eks:index:ServiceRole                                          aws-eks-podExecutionRole                   create     
 +   │  │  ├─ aws:iam:Role                                                aws-eks-podExecutionRole-role              create     
 +   │  │  └─ aws:iam:RolePolicyAttachment                                aws-eks-podExecutionRole-6ad441d9          create     
 +   │  ├─ aws:eks:Cluster                                                aws-eks-eksCluster                         create     
 +   │  ├─ aws:ec2:SecurityGroup                                          aws-eks-nodeSecurityGroup                  create     
 +   │  ├─ pulumi:providers:kubernetes                                    aws-eks-provider                           create     
 +   │  ├─ eks:index:VpcCni                                               aws-eks-vpc-cni                            create     
 +   │  ├─ aws:ec2:SecurityGroupRule                                      aws-eks-eksExtApiServerClusterIngressRule  create     
 +   │  ├─ pulumi:providers:kubernetes                                    aws-eks-eks-k8s                            create     
 +   │  ├─ aws:eks:FargateProfile                                         aws-eks-fargateProfile                     create     
 +   │  ├─ aws:ec2:SecurityGroupRule                                      aws-eks-eksNodeInternetEgressRule          create     
 +   │  ├─ aws:ec2:SecurityGroupRule                                      aws-eks-eksNodeClusterIngressRule          create     
 +   │  ├─ aws:ec2:SecurityGroupRule                                      aws-eks-eksNodeIngressRule                 create     
 +   │  ├─ aws:ec2:SecurityGroupRule                                      aws-eks-eksClusterIngressRule              create     
 +   │  └─ kubernetes:core/v1:ConfigMap                                   aws-eks-nodeAccess                         create     
 +   ├─ eks:index:ManagedNodeGroup                                        aws-bottlerocket                           create     
 +   │  └─ aws:eks:NodeGroup                                              aws-bottlerocket                           create     
 +   ├─ eks:index:ManagedNodeGroup                                        aws-linux                                  create     
 +   │  └─ aws:eks:NodeGroup                                              aws-linux                                  create     
 +   └─ pulumi:providers:kubernetes                                       aws-k8s-provider                           create     
 +      ├─ kubernetes:rbac.authorization.k8s.io/v1:ClusterRoleBinding     datadog-agent                              create     
 +      ├─ dd:apps                                                        workload-tracegen-tracegen                 create     
 +      │  ├─ kubernetes:core/v1:Namespace                                workload-tracegen                          create     
 +      │  ├─ kubernetes:apps/v1:Deployment                               workload-tracegen/tracegen-uds             create     
 +      │  └─ kubernetes:apps/v1:Deployment                               workload-tracegen/tracegen-tcp             create     
 +      ├─ dd:apps                                                        dogstatsd-8128                             create     
 +      │  ├─ kubernetes:core/v1:Namespace                                workload-dogstatsd-standalone              create     
 +      │  ├─ kubernetes:apps/v1:Deployment                               dogstatsd-udp-origin-detection-8128        create     
 +      │  ├─ kubernetes:apps/v1:Deployment                               dogstatsd-uds-8128                         create     
 +      │  └─ kubernetes:apps/v1:Deployment                               dogstatsd-udp-8128                         create     
 +      ├─ dd:apps                                                        redis                                      create     
 +      │  ├─ kubernetes:policy/v1:PodDisruptionBudget                    redis                                      create     
 +      │  ├─ kubernetes:apps/v1:Deployment                               redis                                      create     
 +      │  ├─ kubernetes:apps/v1:Deployment                               redis-query                                create     
 +      │  ├─ kubernetes:core/v1:Namespace                                workload-redis                             create     
 +      │  ├─ kubernetes:core/v1:Service                                  redis                                      create     
 +      │  ├─ kubernetes:datadoghq.com/v1alpha1:DatadogMetric             redis                                      create     
 +      │  └─ kubernetes:autoscaling/v2:HorizontalPodAutoscaler           redis                                      create     
 +      ├─ dd:dogstatsd-standalone                                        dogstatsd                                  create     
 +      │  ├─ kubernetes:core/v1:ServiceAccount                           dogstatsd-standalone                       create     
 +      │  ├─ kubernetes:scheduling.k8s.io/v1:PriorityClass               dogstatsd-standalone                       create     
 +      │  ├─ kubernetes:rbac.authorization.k8s.io/v1:ClusterRole         dogstatsd-standalone                       create     
 +      │  ├─ kubernetes:apps/v1:DaemonSet                                dogstatsd-standalone                       create     
 +      │  ├─ kubernetes:core/v1:Namespace                                dogstatsd-standalone                       create     
 +      │  └─ kubernetes:rbac.authorization.k8s.io/v1:ClusterRoleBinding  dogstatsd-standalone                       create     
 +      ├─ dd:apps                                                        nginx                                      create     
 +      │  ├─ kubernetes:apps/v1:Deployment                               nginx-query                                create     
 +      │  ├─ kubernetes:core/v1:Service                                  nginx                                      create     
 +      │  ├─ kubernetes:policy/v1:PodDisruptionBudget                    nginx                                      create     
 +      │  ├─ kubernetes:core/v1:Namespace                                workload-nginx                             create     
 +      │  ├─ kubernetes:apps/v1:Deployment                               nginx                                      create     
 +      │  ├─ kubernetes:datadoghq.com/v1alpha1:DatadogMetric             nginx                                      create     
 +      │  └─ kubernetes:autoscaling/v2:HorizontalPodAutoscaler           nginx                                      create     
 +      ├─ dd:apps                                                        dogstatsd-8125                             create     
 +      │  ├─ kubernetes:apps/v1:Deployment                               dogstatsd-uds-8125                         create     
 +      │  ├─ kubernetes:core/v1:Namespace                                workload-dogstatsd                         create     
 +      │  ├─ kubernetes:apps/v1:Deployment                               dogstatsd-udp-origin-detection-8125        create     
 +      │  └─ kubernetes:apps/v1:Deployment                               dogstatsd-udp-8125                         create     
 +      ├─ dd:apps                                                        mutated                                    create     
 +      │  └─ kubernetes:core/v1:Namespace                                workload-mutated                           create     
 +      │     └─ kubernetes:apps/v1:Deployment                            mutated                                    create     
 +      ├─ dd:apps                                                        nginx-fargate                              create     
 +      │  ├─ kubernetes:policy/v1:PodDisruptionBudget                    nginx-fargate                              create     
 +      │  ├─ kubernetes:apps/v1:Deployment                               nginx-query-fargate                        create     
 +      │  ├─ kubernetes:core/v1:Service                                  nginx-fargate                              create     
 +      │  └─ kubernetes:apps/v1:Deployment                               nginx-fargate                              create     
 +      ├─ kubernetes:core/v1:ServiceAccount                              datadog-agent                              create     
 +      ├─ dd:apps                                                        prometheus                                 create     
 +      │  ├─ kubernetes:core/v1:Namespace                                workload-prometheus                        create     
 +      │  └─ kubernetes:apps/v1:Deployment                               prometheus                                 create     
 +      ├─ dd:apps                                                        cpustress                                  create     
 +      │  ├─ kubernetes:apps/v1:Deployment                               stress-ng                                  create     
 +      │  └─ kubernetes:core/v1:Namespace                                workload-cpustress                         create     
 +      ├─ random:index:RandomString                                      datadog-cluster-agent-token                create     
 +      ├─ dd:agent                                                       dda                                        create     
 +      │  ├─ kubernetes:core/v1:Secret                                   datadog-credentials                        create     
 +      │  ├─ kubernetes:helm.sh/v3:Release                               dda                                        create     
 +      │  └─ kubernetes:core/v1:Namespace                                datadog                                    create     
 +      ├─ kubernetes:core/v1:Secret                                      datadog-credentials-injection              create     
 +      ├─ dd:apps                                                        redis-fargate                              create     
 +      │  ├─ kubernetes:policy/v1:PodDisruptionBudget                    redis-fargate                              create     
 +      │  ├─ kubernetes:apps/v1:Deployment                               redis-query-fargate                        create     
 +      │  ├─ kubernetes:core/v1:Service                                  redis-fargate                              create     
 +      │  └─ kubernetes:apps/v1:Deployment                               redis-fargate                              create     
 +      ├─ kubernetes:rbac.authorization.k8s.io/v1:ClusterRole            datadog-agent                              create     
 +      └─ kubernetes:core/v1:Namespace                                   fargate                                    create     

Outputs:
    agent-linux-helm-install-name  : output<string>
    agent-linux-helm-install-status: output<string>
    dd-Cluster-aws-eks             : output<string>

Resources:
    + 97 to create

Update:

Updating (jennifer-chen-aws-eks):
     Type                                                                 Name                                       Status     
 +   pulumi:pulumi:Stack                                                  dd-jennifer-chen-aws-eks                   created (12
 +   ├─ dd:Cluster                                                        aws-eks                                    created (74
 +   ├─ pulumi:providers:random                                           random                                     created (0.
 +   ├─ pulumi:providers:aws                                              aws                                        created (0.
 +   ├─ random:index:RandomShuffle                                        aws-rnd-subnet                             created (0.
 +   ├─ aws:ec2:SecurityGroup                                             aws-eks-sg                                 created (2s
 +   ├─ aws:iam:Role                                                      aws-eks-cluster-role                       created (1s
 +   ├─ aws:iam:Role                                                      aws-eks-windows-node-role                  created (1s
 +   ├─ aws:iam:Role                                                      aws-eks-linux-node-role                    created (1s
 +   ├─ pulumi:providers:eks                                              eks                                        created (0.
 +   ├─ eks:index:Cluster                                                 aws-eks                                    created (74
 +   │  ├─ eks:index:ServiceRole                                          aws-eks-podExecutionRole                   created (0.
 +   │  │  ├─ aws:iam:Role                                                aws-eks-podExecutionRole-role              created (0.
 +   │  │  └─ aws:iam:RolePolicyAttachment                                aws-eks-podExecutionRole-6ad441d9          created (0.
 +   │  ├─ aws:eks:Cluster                                                aws-eks-eksCluster                         created (48
 +   │  ├─ aws:ec2:SecurityGroup                                          aws-eks-nodeSecurityGroup                  created (2s
 +   │  ├─ eks:index:VpcCni                                               aws-eks-vpc-cni                            created (4s
 +   │  ├─ pulumi:providers:kubernetes                                    aws-eks-eks-k8s                            created (0.
 +   │  ├─ pulumi:providers:kubernetes                                    aws-eks-provider                           created (0.
 +   │  ├─ kubernetes:core/v1:ConfigMap                                   aws-eks-nodeAccess                         created (0.
 +   │  ├─ aws:ec2:SecurityGroupRule                                      aws-eks-eksNodeClusterIngressRule          created (1s
 +   │  ├─ aws:ec2:SecurityGroupRule                                      aws-eks-eksNodeInternetEgressRule          created (2s
 +   │  ├─ aws:ec2:SecurityGroupRule                                      aws-eks-eksExtApiServerClusterIngressRule  created (1s
 +   │  ├─ aws:ec2:SecurityGroupRule                                      aws-eks-eksClusterIngressRule              created (0.
 +   │  ├─ aws:ec2:SecurityGroupRule                                      aws-eks-eksNodeIngressRule                 created (0.
 +   │  └─ aws:eks:FargateProfile                                         aws-eks-fargateProfile                     created (25
 +   ├─ eks:index:ManagedNodeGroup                                        aws-bottlerocket                           created (13
 +   │  └─ aws:eks:NodeGroup                                              aws-bottlerocket                           created (13
 +   ├─ eks:index:ManagedNodeGroup                                        aws-linux                                  created (12
 +   │  └─ aws:eks:NodeGroup                                              aws-linux                                  created (12
 +   └─ pulumi:providers:kubernetes                                       aws-k8s-provider                           created (0.
 +      ├─ dd:apps                                                        redis                                      created    
 +      │  ├─ kubernetes:core/v1:Namespace                                workload-redis                             created (0.
 +      │  ├─ kubernetes:policy/v1:PodDisruptionBudget                    redis                                      created (0.
 +      │  ├─ kubernetes:core/v1:Service                                  redis                                      created (23
 +      │  ├─ kubernetes:apps/v1:Deployment                               redis-query                                created (4s
 +      │  ├─ kubernetes:apps/v1:Deployment                               redis                                      created (13
 +      │  ├─ kubernetes:datadoghq.com/v1alpha1:DatadogMetric             redis                                      created (0.
 +      │  └─ kubernetes:autoscaling/v2:HorizontalPodAutoscaler           redis                                      created (0.
 +      ├─ dd:apps                                                        cpustress                                  created    
 +      │  ├─ kubernetes:core/v1:Namespace                                workload-cpustress                         created (0.
 +      │  └─ kubernetes:apps/v1:Deployment                               stress-ng                                  created (18
 +      ├─ dd:apps                                                        dogstatsd-8125                             created    
 +      │  ├─ kubernetes:core/v1:Namespace                                workload-dogstatsd                         created (0.
 +      │  ├─ kubernetes:apps/v1:Deployment                               dogstatsd-uds-8125                         created (13
 +      │  ├─ kubernetes:apps/v1:Deployment                               dogstatsd-udp-8125                         created (9s
 +      │  └─ kubernetes:apps/v1:Deployment                               dogstatsd-udp-origin-detection-8125        created (10
 +      ├─ random:index:RandomString                                      datadog-cluster-agent-token                created (0.
 +      ├─ dd:apps                                                        dogstatsd-8128                             created    
 +      │  ├─ kubernetes:core/v1:Namespace                                workload-dogstatsd-standalone              created (0.
 +      │  ├─ kubernetes:apps/v1:Deployment                               dogstatsd-uds-8128                         created (39
 +      │  ├─ kubernetes:apps/v1:Deployment                               dogstatsd-udp-8128                         created (12
 +      │  └─ kubernetes:apps/v1:Deployment                               dogstatsd-udp-origin-detection-8128        created (8s
 +      ├─ dd:agent                                                       dda                                        created (11
 +      │  ├─ kubernetes:core/v1:Namespace                                datadog                                    created (0.
 +      │  ├─ kubernetes:core/v1:Secret                                   datadog-credentials                        created (0.
 +      │  └─ kubernetes:helm.sh/v3:Release                               dda                                        created (11
 +      ├─ dd:apps                                                        mutated                                    created    
 +      │  └─ kubernetes:core/v1:Namespace                                workload-mutated                           created (0.
 +      │     └─ kubernetes:apps/v1:Deployment                            mutated                                    created (19
 +      ├─ dd:dogstatsd-standalone                                        dogstatsd                                  created    
 +      │  ├─ kubernetes:core/v1:Namespace                                dogstatsd-standalone                       created (0.
 +      │  ├─ kubernetes:rbac.authorization.k8s.io/v1:ClusterRole         dogstatsd-standalone                       created (0.
 +      │  ├─ kubernetes:apps/v1:DaemonSet                                dogstatsd-standalone                       created (0.
 +      │  ├─ kubernetes:rbac.authorization.k8s.io/v1:ClusterRoleBinding  dogstatsd-standalone                       created (0.
 +      │  ├─ kubernetes:scheduling.k8s.io/v1:PriorityClass               dogstatsd-standalone                       created (0.
 +      │  └─ kubernetes:core/v1:ServiceAccount                           dogstatsd-standalone                       created (0.
 +      ├─ kubernetes:rbac.authorization.k8s.io/v1:ClusterRole            datadog-agent                              created (0.
 +      ├─ dd:apps                                                        nginx                                      created    
 +      │  ├─ kubernetes:core/v1:Namespace                                workload-nginx                             created (0.
 +      │  ├─ kubernetes:policy/v1:PodDisruptionBudget                    nginx                                      created (0.
 +      │  ├─ kubernetes:apps/v1:Deployment                               nginx                                      created (6s
 +      │  ├─ kubernetes:apps/v1:Deployment                               nginx-query                                created (3s
 +      │  ├─ kubernetes:core/v1:Service                                  nginx                                      created (10
 +      │  ├─ kubernetes:datadoghq.com/v1alpha1:DatadogMetric             nginx                                      created (0.
 +      │  └─ kubernetes:autoscaling/v2:HorizontalPodAutoscaler           nginx                                      created (0.
 +      ├─ kubernetes:core/v1:ServiceAccount                              datadog-agent                              created (0.
 +      ├─ kubernetes:rbac.authorization.k8s.io/v1:ClusterRoleBinding     datadog-agent                              created (0.
 +      ├─ kubernetes:core/v1:Namespace                                   fargate                                    created (0.
 +      ├─ dd:apps                                                        workload-tracegen-tracegen                 created    
 +      │  ├─ kubernetes:core/v1:Namespace                                workload-tracegen                          created (0.
 +      │  ├─ kubernetes:apps/v1:Deployment                               workload-tracegen/tracegen-uds             created (11
 +      │  └─ kubernetes:apps/v1:Deployment                               workload-tracegen/tracegen-tcp             created (11
 +      ├─ dd:apps                                                        prometheus                                 created    
 +      │  ├─ kubernetes:core/v1:Namespace                                workload-prometheus                        created (0.
 +      │  └─ kubernetes:apps/v1:Deployment                               prometheus                                 created (6s
 +      ├─ kubernetes:core/v1:Secret                                      datadog-credentials-injection              created (0.
 +      ├─ dd:apps                                                        redis-fargate                              created    
 +      │  ├─ kubernetes:policy/v1:PodDisruptionBudget                    redis-fargate                              created (0.
 +      │  ├─ kubernetes:core/v1:Service                                  redis-fargate                              created (95
 +      │  ├─ kubernetes:apps/v1:Deployment                               redis-query-fargate                        created (56
 +      │  └─ kubernetes:apps/v1:Deployment                               redis-fargate                              created (85
 +      └─ dd:apps                                                        nginx-fargate                              created    
 +         ├─ kubernetes:apps/v1:Deployment                               nginx-query-fargate                        created (59
 +         ├─ kubernetes:policy/v1:PodDisruptionBudget                    nginx-fargate                              created (0.
 +         ├─ kubernetes:core/v1:Service                                  nginx-fargate                              created (11
 +         └─ kubernetes:apps/v1:Deployment                               nginx-fargate                              created (10
image

[metrics]

@jennchenn jennchenn requested a review from a team as a code owner April 12, 2024 09:55
Comment on lines 227 to 228
Type: pulumi.String("AverageValue"),
AverageValue: pulumi.String("10"),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In fact, I recently discovered (See #721) that using AverageValue was a mistake because the query that is done is already computing an average:

avg:nginx.net.request_per_s{kube_cluster_name:%%%%tag_kube_cluster_name%%%%,kube_namespace:%s,kube_deployment:nginx}.rollup(60)

So, as it is today, the total number of TPS received by all Nginx pods it divided twice by the number of pods:

  • Once on datadog backend side because of the avg: operator in the metrics query.
  • Once by the HPA itself.
Suggested change
Type: pulumi.String("AverageValue"),
AverageValue: pulumi.String("10"),
Type: pulumi.String("Value"),
Value: pulumi.String("10"),

},
},
},
},
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In order to make the HPA more reactive, it could be nice disable the stabilization window as it is done in #721.

Suggested change
},
Behavior: &autoscalingv2beta2.HorizontalPodAutoscalerBehaviorArgs{
ScaleDown: &autoscalingv2beta2.HPAScalingRulesArgs{
StabilizationWindowSeconds: pulumi.IntPtr(0),
},
},

Comment on lines 262 to 263
Type: pulumi.String("AverageValue"),
AverageValue: pulumi.String("10"),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Type: pulumi.String("AverageValue"),
AverageValue: pulumi.String("10"),
Type: pulumi.String("Value"),
Value: pulumi.String("10"),

},
},
)),
},
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
},
Behavior: &autoscalingv2beta2.HorizontalPodAutoscalerBehaviorArgs{
ScaleDown: &autoscalingv2beta2.HPAScalingRulesArgs{
StabilizationWindowSeconds: pulumi.IntPtr(0),
},
},
},

Comment on lines 197 to 198
Type: pulumi.String("AverageValue"),
AverageValue: pulumi.String("10"),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Type: pulumi.String("AverageValue"),
AverageValue: pulumi.String("10"),
Type: pulumi.String("Value"),
Value: pulumi.String("10"),

},
},
},
},
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
},
Behavior: &autoscalingv2beta2.HorizontalPodAutoscalerBehaviorArgs{
ScaleDown: &autoscalingv2beta2.HPAScalingRulesArgs{
StabilizationWindowSeconds: pulumi.IntPtr(0),
},
},
},

Comment on lines 232 to 233
Type: pulumi.String("AverageValue"),
AverageValue: pulumi.String("10"),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Type: pulumi.String("AverageValue"),
AverageValue: pulumi.String("10"),
Type: pulumi.String("Value"),
Value: pulumi.String("10"),

},
},
},
},
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
},
Behavior: &autoscalingv2beta2.HorizontalPodAutoscalerBehaviorArgs{
ScaleDown: &autoscalingv2beta2.HPAScalingRulesArgs{
StabilizationWindowSeconds: pulumi.IntPtr(0),
},
},
},

Comment on lines 268 to 270
if err != nil {
return err
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which error are we checking here ?
In case of non-nil error, shouldn’t have we returned earlier ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry i just forgot to remove this!

@jennchenn jennchenn requested a review from L3n41c May 2, 2024 14:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants