deleteObject ignores isRequesterPays

[jfromme]

Describe the bug

The deleteObject methods in AmazonS3Client do not check or use the request's isRequesterPays value.

Expected Behavior

When a delete object request has isRequesterPays set to true and the target bucket has requester pays enabled then the request should succeed assuming the user has the required permsisions.

Current Behavior

Instead the request fails with an AccessDenied error.

Reproduction Steps

Attempt to delete an object from a requester pays enabled bucket.

Possible Solution

deleteObject should call the populateRequesterPaysHeader as other methods in the client do.

Additional Information/Context

No response

AWS Java SDK version used

1.11.931

JDK version used

8u212

Operating System and version

Alpine Linux 3.9

[debora-ito]

Are you sure the requester has the necessary permissions to access the object? AccessDenied indicates it has not.
Are you providing the requesterPays parameter in DeleteObjectRequest?
Can you share a repro code?

[jfromme]

I believe the permissions are okay. Switching from using deleteObject() to deleteObjects() worked without any modifications to the user or bucket permissions.

Here is a test case which uses MockServer to set expectations on the request being sent.
testDeleteObject() fails with a 404 because the expected request header is absent.
In contrast, testDeleteObjects() passes.
https://github.com/jfromme/delete-object-test

Here is the MockServer log from testDeleteObject():

2022-10-10 16:27:40 5.14.0 INFO 52920 started on port: 52920 
2022-10-10 16:27:41 5.14.0 INFO 52920 creating expectation:

  {
    "httpRequest" : {
      "method" : "DELETE",
      "path" : "/test-bucket/file-key",
      "headers" : {
        "x-amz-request-payer" : [ "requester" ]
      }
    },
    "httpResponse" : {
      "statusCode" : 204
    },
    "id" : "3555c98f-28b1-4216-8d62-68f13223e655",
    "priority" : 0,
    "timeToLive" : {
      "unlimited" : true
    },
    "times" : {
      "unlimited" : true
    }
  }

 with id:

  3555c98f-28b1-4216-8d62-68f13223e655
 
2022-10-10 16:27:41 5.14.0 INFO 52920 received request:

  {
    "method" : "DELETE",
    "path" : "/test-bucket/file-key",
    "headers" : {
      "x-amz-content-sha256" : [ "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" ],
      "content-length" : [ "0" ],
      "content-encoding" : [ ".*" ],
      "amz-sdk-retry" : [ "0/0/500" ],
      "amz-sdk-request" : [ "attempt=1;max=4" ],
      "amz-sdk-invocation-id" : [ "ae9d22c7-80eb-f7f7-99e0-0e6df6702135" ],
      "X-Amz-Date" : [ "20221010T202741Z" ],
      "User-Agent" : [ "aws-sdk-java/1.12.319 Mac_OS_X/10.15.7 OpenJDK_64-Bit_Server_VM/25.312-b07 java/1.8.0_312 vendor/Amazon.com_Inc. cfg/retry-mode/legacy" ],
      "Host" : [ "localhost:52920" ],
      "Content-Type" : [ "application/octet-stream" ],
      "Connection" : [ "Keep-Alive" ],
      "Authorization" : [ "AWS4-HMAC-SHA256 Credential=AKIA5SOSPFND4TX4E4Y7/20221010/us-east-1/s3/aws4_request, SignedHeaders=amz-sdk-invocation-id;amz-sdk-request;amz-sdk-retry;content-type;host;user-agent;x-amz-content-sha256;x-amz-date, Signature=b8d6c98498f9bc24ff46a4022826363e914d595f234aeecfb2d08ab77b800cdc" ]
    },
    "keepAlive" : true,
    "secure" : false,
    "localAddress" : "127.0.0.1:52920",
    "remoteAddress" : "127.0.0.1"
  }
 
2022-10-10 16:27:41 5.14.0 INFO 52920 request:

  {
    "method" : "DELETE",
    "path" : "/test-bucket/file-key",
    "headers" : {
      "x-amz-content-sha256" : [ "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" ],
      "content-length" : [ "0" ],
      "content-encoding" : [ ".*" ],
      "amz-sdk-retry" : [ "0/0/500" ],
      "amz-sdk-request" : [ "attempt=1;max=4" ],
      "amz-sdk-invocation-id" : [ "ae9d22c7-80eb-f7f7-99e0-0e6df6702135" ],
      "X-Amz-Date" : [ "20221010T202741Z" ],
      "User-Agent" : [ "aws-sdk-java/1.12.319 Mac_OS_X/10.15.7 OpenJDK_64-Bit_Server_VM/25.312-b07 java/1.8.0_312 vendor/Amazon.com_Inc. cfg/retry-mode/legacy" ],
      "Host" : [ "localhost:52920" ],
      "Content-Type" : [ "application/octet-stream" ],
      "Connection" : [ "Keep-Alive" ],
      "Authorization" : [ "AWS4-HMAC-SHA256 Credential=AKIA5SOSPFND4TX4E4Y7/20221010/us-east-1/s3/aws4_request, SignedHeaders=amz-sdk-invocation-id;amz-sdk-request;amz-sdk-retry;content-type;host;user-agent;x-amz-content-sha256;x-amz-date, Signature=b8d6c98498f9bc24ff46a4022826363e914d595f234aeecfb2d08ab77b800cdc" ]
    },
    "keepAlive" : true,
    "secure" : false,
    "localAddress" : "127.0.0.1:52920",
    "remoteAddress" : "127.0.0.1"
  }

 didn't match expectation:

  {
    "httpRequest" : {
      "method" : "DELETE",
      "path" : "/test-bucket/file-key",
      "headers" : {
        "x-amz-request-payer" : [ "requester" ]
      }
    },
    "httpResponse" : {
      "statusCode" : 204
    },
    "id" : "3555c98f-28b1-4216-8d62-68f13223e655",
    "priority" : 0,
    "timeToLive" : {
      "unlimited" : true
    },
    "times" : {
      "unlimited" : true
    }
  }

 because:

  method matched
  path matched
  body matched
  headers didn't match
 
2022-10-10 16:27:41 5.14.0 INFO 52920 no expectation for:

  {
    "method" : "DELETE",
    "path" : "/test-bucket/file-key",
    "headers" : {
      "x-amz-content-sha256" : [ "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" ],
      "content-length" : [ "0" ],
      "content-encoding" : [ ".*" ],
      "amz-sdk-retry" : [ "0/0/500" ],
      "amz-sdk-request" : [ "attempt=1;max=4" ],
      "amz-sdk-invocation-id" : [ "ae9d22c7-80eb-f7f7-99e0-0e6df6702135" ],
      "X-Amz-Date" : [ "20221010T202741Z" ],
      "User-Agent" : [ "aws-sdk-java/1.12.319 Mac_OS_X/10.15.7 OpenJDK_64-Bit_Server_VM/25.312-b07 java/1.8.0_312 vendor/Amazon.com_Inc. cfg/retry-mode/legacy" ],
      "Host" : [ "localhost:52920" ],
      "Content-Type" : [ "application/octet-stream" ],
      "Connection" : [ "Keep-Alive" ],
      "Authorization" : [ "AWS4-HMAC-SHA256 Credential=AKIA5SOSPFND4TX4E4Y7/20221010/us-east-1/s3/aws4_request, SignedHeaders=amz-sdk-invocation-id;amz-sdk-request;amz-sdk-retry;content-type;host;user-agent;x-amz-content-sha256;x-amz-date, Signature=b8d6c98498f9bc24ff46a4022826363e914d595f234aeecfb2d08ab77b800cdc" ]
    },
    "keepAlive" : true,
    "secure" : false,
    "localAddress" : "127.0.0.1:52920",
    "remoteAddress" : "127.0.0.1"
  }

 returning response:

  {
    "statusCode" : 404,
    "reasonPhrase" : "Not Found"
  }

[debora-ito]

@jfromme apologies for the long silence.

Yes, I can reproduce, the x-amz-requester-payer is not being added to the request.
The workaround is to use DeleteObjects.