Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add aws support #640

Open
wants to merge 11 commits into
base: master
Choose a base branch
from

Conversation

rabunkosar-dd
Copy link

A proposed skeleton PR to support AWS resource expansion

@rabunkosar-dd
Copy link
Author

@manugarg can you take a look at this if you have time? that would help replace some of our custom code we have internally to supplement cloudprober. Thank you.

@manugarg
Copy link
Contributor

@manugarg can you take a look at this if you have time? that would help replace some of our custom code we have internally to supplement cloudprober. Thank you.

Hello Rabun,

Sorry for the long delay. I've started reviewing. I'll try to finish in a couple of days.

Thanks for submitting this.

@manugarg manugarg self-requested a review December 20, 2023 09:19
@manugarg manugarg added this to the v0.13.3 milestone Dec 21, 2023
@manugarg manugarg added the enhancement New feature or request label Dec 21, 2023
Copy link
Contributor

@manugarg manugarg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks once again for sending this change and your patience. I've not finished reviewing, but sending in the comments so far. Any chance you can split this change it into:

  1. 1 PR to add AWS targets support.
  2. 1 PR to add elasticache and RDS.

cmd/cloudprober.go Outdated Show resolved Hide resolved
internal/rds/aws/elasticache.go Outdated Show resolved Hide resolved
internal/rds/aws/elasticache.go Outdated Show resolved Hide resolved
internal/rds/aws/elasticache.go Outdated Show resolved Hide resolved
internal/rds/aws/elasticache.go Outdated Show resolved Hide resolved
@rabunkosar-dd
Copy link
Author

Thanks once again for sending this change and your patience. I've not finished reviewing, but sending in the comments so far. Any chance you can split this change it into:

  1. 1 PR to add AWS targets support.
  2. 1 PR to add elasticache and RDS.

I have addressed some of your feedback including the splits for AWS APIs. I'll work on splitting the PR into multiple pieces for easy reviews. Thanks for the reviews so far

Copy link
Contributor

@manugarg manugarg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for delay again. Please see my review inline.

internal/rds/client/client.go Outdated Show resolved Hide resolved
targets/endpoint/endpoint.go Outdated Show resolved Hide resolved
internal/rds/client/client.go Outdated Show resolved Hide resolved
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand why this file is included in this change?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's imports tool, placing the internal package in the top section, seems like a good change but I can remove if you want

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, actually flag is in the middle for a reason. It's required for Google's importing tools to work properly. Please drop this file.

internal/rds/aws/testdata/targets.json Outdated Show resolved Hide resolved
internal/rds/aws/testdata/targets2.textpb Outdated Show resolved Hide resolved
internal/rds/aws/testdata/targets1.textpb Outdated Show resolved Hide resolved
Comment on lines +31 to +32
// How often resources should be refreshed.
optional int32 re_eval_sec = 98 [default = 600]; // default 10 mins
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am assuming you'll eventually need names and filters for all these resources too?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes that's correct

Comment on lines 82 to 84
// Amazon Resource Name (ARN) of the load balancer
// if specified, only the corresponding load balancer information is returned.
repeated string name = 1;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As name can be slightly confusing, should we standardize on ARN (arn) for AWS resources?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that was a wrong comment on my end, this was for the name of the LB not ARN, we can ARN later if needed. I plan to add support for LBs as a separate PR and then multiple options (ARN and name etc) can be added as filters if needed

@manugarg
Copy link
Contributor

Hi @rabunkosar-dd, do you want to continue working on it.

Apologies for the initials delays in review, but I think you're pretty close.

(P.S. Bigger PRs require blocking more time but if you break it up, we can get through them more quickly)

@rabunkosar-dd
Copy link
Author

Hi @rabunkosar-dd, do you want to continue working on it.

Apologies for the initials delays in review, but I think you're pretty close.

(P.S. Bigger PRs require blocking more time but if you break it up, we can get through them more quickly)

Hey, I'm planning to resume working on it, got blocked on other projects. Will update the PR soon

Copy link
Contributor

@manugarg manugarg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's getting close. Added some comments about dropping irrelevant file:

  • cmd/cloudprober.go
  • targets/endpoint/endpoint.go
  • rds/client/client.go

Also, it will be great if you can trim down the proto file for now.

Please see my comments inline inline as well.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, actually flag is in the middle for a reason. It's required for Google's importing tools to work properly. Please drop this file.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To make this change more focussed, I'd leave this file too. Typo fix is not relevant to this PR.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please leave out this file too. Fixing of insecure package is an independent thing. Thanks!

Comment on lines +1 to +2
// Copyright 2019 The Cloudprober Authors.
//
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd put the latest year in the copyright headers of new *.go files.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it's not too much trouble, can you please add resource types to protobuf as you actually add their logic. It will be easier to review them together like that. With that you'll have to change the Go code also.

So I'd prefer if this file contains only the following for now:

// AWS provider config.
message ProviderConfig {
  // Profile for the session.
  optional string profile_name = 1;

  // AWS region
  optional string region = 2;

  // TODO: Add resource types in further PRs.
}

@manugarg manugarg removed this from the v0.13.3 milestone Mar 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants