EC2::Client#describe_security_groups undocumented & unchecked options limitation.

[NathanZookCH]

Describe the issue

group_names: option is incompatible with filters: { name: 'vpc-id', } option.

This looks pretty much like a bug, but I expect it is a lot easier to get a documentation fix than a change to the API.

Demonstration from the ruby sdk:

vpc_id = LambdaFunction::WorkerLambda::VPC_ID.to_s
group_name = ec2.describe_security_groups(
  filters: [{name: 'vpc-id', values: [vpc_id]
}]).security_groups[4].group_name

ec2.describe_security_groups(
  filters: [{name: 'vpc-id', values: [vpc_id]}],
  group_names: [group_name]
)

/usr/local/bundle/gems/aws-sdk-core-3.190.2/lib/seahorse/client/plugins/raise_response_errors.rb:17:in `call':
The security group 'security-group' does not exist in default VPC 'vpc-12345678' (Aws::EC2::Errors::InvalidGroupNotFound)

So the options check does not fail. This looks like a bug in the client, but since we're getting an exception anyway...

I also note that the API documentation indicates that group_ids is a required field if you are not in the default vpc. If this were true, it would be a severe bug. Thankfully, the API does not require this, at least for the case I observe.

Links

https://docs.aws.amazon.com/cli/latest/reference/ec2/describe-security-groups.html
https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/EC2/Client.html#describe_security_groups-instance_method

[NathanZookCH]

After some noodling, I realized that there is a reasonable workaround: tag the security group with it's name! That is,

ec2.describe_security_groups(
  filters: [
    {name: 'vpc-id', values: [vpc_id]},
    {name: 'tag:name', values: [group_name]}
  ]
)

Works if you #create_tags(resources: [group_id], tags: [{ key: name, value: group_name}]) first.

[akshat2203]

The error message you provided indicates that there is an issue with using both the group_names and filters options together when calling the describe_security_groups method in the AWS SDK for Ruby.

[Rameshk84]

Describe the issue

group_names: option is incompatible with filters: { name: 'vpc-id', } option.

This looks pretty much like a bug, but I expect it is a lot easier to get a documentation fix than a change to the API.

Demonstration from the ruby sdk:

vpc_id = LambdaFunction::WorkerLambda::VPC_ID.to_s
group_name = ec2.describe_security_groups(
  filters: [{name: 'vpc-id', values: [vpc_id]
}]).security_groups[4].group_name

ec2.describe_security_groups(
  filters: [{name: 'vpc-id', values: [vpc_id]}],
  group_names: [group_name]
)

/usr/local/bundle/gems/aws-sdk-core-3.190.2/lib/seahorse/client/plugins/raise_response_errors.rb:17:in `call':
The security group 'security-group' does not exist in default VPC 'vpc-12345678' (Aws::EC2::Errors::InvalidGroupNotFound)

So the options check does not fail. This looks like a bug in the client, but since we're getting an exception anyway...

I also note that the API documentation indicates that group_ids is a required field if you are not in the default vpc. If this were true, it would be a severe bug. Thankfully, the API does not require this, at least for the case I observe.

Links

https://docs.aws.amazon.com/cli/latest/reference/ec2/describe-security-groups.html https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/EC2/Client.html#describe_security_groups-instance_method

This is my Answer for this documentation i tried this solution using some other Ai tools to leverage my work :

It seems like you're encountering an issue with the AWS SDK for Ruby when using the describe_security_groups method. The error message indicates an incompatibility between the group_names and filters options.

The issue appears to arise when you provide both group_names and filters options in the describe_security_groups call. The error message suggests that the provided security group name (group_name) does not exist in the default VPC.

It's important to note that the AWS API documentation indicates that group_ids is a required field if you are not in the default VPC. However, you've observed that this is not the case in your scenario.

This discrepancy between the documented behavior and the actual behavior you're experiencing could indeed indicate a bug in the AWS SDK for Ruby. It might be worth checking the AWS SDK for Ruby's issue tracker or forums to see if others have reported similar issues. Additionally, reaching out to AWS support could provide further insights or assistance in resolving this issue.

In the meantime, you might consider adjusting your code to either use group_ids instead of group_names or to remove the group_names option altogether if it's not necessary for your use case. This might help to work around the issue until a fix is available.

[RyanFitzSimmonsAK]

Hi @NathanZookCH, thanks for reaching out. I'm having some issues reproducing this behavior. Given that you did your testing in the Ruby SDK, unless you were able to reproduce this in the CLI as well, it might be more helpful for me to transfer this issue to the AWS SDK for Ruby repository.

I did my testing using the following syntax, and had no problems.

aws ec2 describe-security-groups --filters Name=vpc-id,Values=myvpcid Name=group-name,Values=test

Could you verify if you were able to reproduce the behavior in the CLI, or only Ruby? If you were, debug logs might be beneficial. You can get debug logs by adding --debug to your command, and redacting any sensitive information. Thanks!

[NathanZookCH]

I only used the ruby sdk. I believe I posted here because I expected the documentation to be more likely to be updated than the code. Sincerely, Nathan Zook Carrum Health Senior Devops Engineer

…

On Tue, Apr 30, 2024 at 2:02 PM Ryan F. ***@***.***> wrote: Hi @NathanZookCH <https://github.com/NathanZookCH>, thanks for reaching out. I'm having some issues reproducing this behavior. Given that you did your testing in the Ruby SDK, unless you were able to reproduce this in the CLI as well, it might be more helpful for me to transfer this issue to the AWS SDK for Ruby repository. I did my testing using the following syntax, and had no problems. aws ec2 describe-security-groups --filters Name=vpc-id,Values=myvpcid Name=group-name,Values=test Could you verify if you were able to reproduce the behavior in the CLI, or only Ruby? If you were, debug logs might be beneficial. You can get debug logs by adding --debug to your command, and redacting any sensitive information. Thanks! — Reply to this email directly, view it on GitHub <#3018>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AV5ISLD3DFL2WBVWZORTFMDZAABFLAVCNFSM6AAAAABCLWCO62VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOBXGI4TSNRVGE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[RyanFitzSimmonsAK]

Given that it appears to be an SDK for Ruby-exclusive bug, I'm going to transfer it to that repository for triage and reproduction.

[mullermp]

@RyanFitzSimmonsAK The error (Aws::EC2::Errors::InvalidGroupNotFound) is a service API error (parsed response). This is not exclusive to the ruby SDK.

[mullermp]

@RanVaknin from the support team can investigate why this happened and whether the service docs need improvement.

[RanVaknin]

Hi @NathanZookCH ,

Its not clear to me what documentation problem you are having. From looking at your code, you are trying to retrieve a security group named security-group which does not exist in the VPC you are filtering on.

I can raise this exact error by querying a non-existent sg name using the CLI:

$ aws ec2 describe-security-groups --group-names 'nonexistent-sg-name' --filters Name=vpc-id,Values=vpc-REDACTED


An error occurred (InvalidGroup.NotFound) when calling the DescribeSecurityGroups operation: The security group 'nonexistent-sg-name' does not exist in default VPC 'vpc-REDACTED'

Also, I can query a non default vpc using both group_names and vpc-id using the Ruby SDK:

ec2 = Aws::EC2::Client.new(region: 'us-east-1')

response = ec2.describe_security_groups({
  filters: [
    { name: 'vpc-id', values: ['vpc-REDACTED'] },
    { name: 'group-name', values: ['MySecurityGroup2'] }
  ]
})

puts response.security_groups

Results in a valid response:

{:description=>"My security group2", :group_name=>"MySecurityGroup2", :ip_permissions=>[], :owner_id=>"REDACTED", :group_id=>"sg-REDACTED", :ip_permissions_egress=>[{:from_port=>nil, :ip_protocol=>"-1", :ip_ranges=>[{:cidr_ip=>"0.0.0.0/0", :description=>nil}], :ipv_6_ranges=>[], :prefix_list_ids=>[], :to_port=>nil, :user_id_group_pairs=>[]}], :tags=>[], :vpc_id=>"vpc-REDACTED"}

Same with the CLI:

$ aws ec2 describe-security-groups --filters Name=vpc-id,Values=vpc-REDACTED Name=group-name,Values=MySecurityGroup2

{
    "SecurityGroups": [
        {
            "Description": "My security group2",
            "GroupName": "MySecurityGroup2",
            "IpPermissions": [],
            "OwnerId": "REDACTED",
            "GroupId": "sg-REDACTED",
            "IpPermissionsEgress": [
                {
                    "IpProtocol": "-1",
                    "IpRanges": [
                        {
                            "CidrIp": "0.0.0.0/0"
                        }
                    ],
                    "Ipv6Ranges": [],
                    "PrefixListIds": [],
                    "UserIdGroupPairs": []
                }
            ],
            "VpcId": "vpc-REDACTED"
        }
    ]
}

A word on documentation: the client specific SDK docs that you linked , are actually generated from each AWS service's API docs. In this case the source of truth is here. Since the SDK's client docs are code generated, the SDK team cannot change those docs, instead they need to be amended usptream with the service team itself.

You can submit a documentation request yourself by clicking on the Feedback button on the top right corner of each AWS documentation page and it will ensure that the request goes directly to the service team.

Since this seems like a documentation related request rather than a bug, and is not actionable by the SDK team, Im inclined to close this. Please let me know if we misunderstood you, otherwise we can close the issue.

Thanks,
Ran~

[NathanZookCH]

Honestly, I've lost context, and, with my current workload, I cannot afford to pursue this good neighbor action at this time.

I was not attempting to search in the wrong vpc. I was attempting to search in the non-default vpc, and running into apparent options incompatibilities when doing so.

Having said all of that, I have since noted that the group-name filter is what I needed to use instead of the group_names option.