ListPartsPaginator for s3 list_parts operation goes into infinite loop

[AMZN-hgoffin]

Describe the bug

The ListPartsPaginator implementation always goes into an infinite loop.

The S3 ListParts API returns a next token of "0", which is neither an empty string nor an omitted field. The correct test for the next page is the "IsTruncated" boolean field; if IsTruncated is not explicitly true, then the next token should be ignored.

Expected Behavior

Expected: using into_paginator() on the S3 list_parts() operation works correctly.

Current Behavior

Currently: the paginator goes into an infinite loop and never finishes collecting pages.

Reproduction Steps

Call into_paginator().send().collect::<Vec<_>>().await on a list_parts() operation - notice that it will never complete.

Possible Solution

The paginator needs to test IsTruncated before trusting the next token field. This is documented in the API.

Additional Information/Context

The next token field name is "NextPartNumberMarker" and it is modeled as an integer, not a string. Fixing this is probably beyond what is required as it might require a new kind of paginator model for non-string types.

It just so happens that the value is 0 when there are no more parts, and 0 is not a valid part number, but it's not clear if this can behavior can be relied on. The documentation explicitly points to IsTruncated as the indication that there could be more pages of results.

Version

├── aws-config v1.1.1
│   ├── aws-credential-types v1.1.1
│   │   ├── aws-smithy-async v1.1.1
│   │   ├── aws-smithy-runtime-api v1.1.1
│   │   │   ├── aws-smithy-async v1.1.1 (*)
│   │   │   ├── aws-smithy-types v1.1.1
│   │   ├── aws-smithy-types v1.1.1 (*)
│   ├── aws-http v0.60.1
│   │   ├── aws-smithy-runtime-api v1.1.1 (*)
│   │   ├── aws-smithy-types v1.1.1 (*)
│   │   ├── aws-types v1.1.1
│   │   │   ├── aws-credential-types v1.1.1 (*)
│   │   │   ├── aws-smithy-async v1.1.1 (*)
│   │   │   ├── aws-smithy-runtime-api v1.1.1 (*)
│   │   │   ├── aws-smithy-types v1.1.1 (*)
│   ├── aws-runtime v1.1.1
│   │   ├── aws-credential-types v1.1.1 (*)
│   │   ├── aws-http v0.60.1 (*)
│   │   ├── aws-sigv4 v1.1.1
│   │   │   ├── aws-credential-types v1.1.1 (*)
│   │   │   ├── aws-smithy-eventstream v0.60.1
│   │   │   │   ├── aws-smithy-types v1.1.1 (*)
│   │   │   ├── aws-smithy-http v0.60.1
│   │   │   │   ├── aws-smithy-eventstream v0.60.1 (*)
│   │   │   │   ├── aws-smithy-runtime-api v1.1.1 (*)
│   │   │   │   ├── aws-smithy-types v1.1.1 (*)
│   │   │   ├── aws-smithy-runtime-api v1.1.1 (*)
│   │   │   ├── aws-smithy-types v1.1.1 (*)
│   │   ├── aws-smithy-async v1.1.1 (*)
│   │   ├── aws-smithy-eventstream v0.60.1 (*)
│   │   ├── aws-smithy-http v0.60.1 (*)
│   │   ├── aws-smithy-runtime-api v1.1.1 (*)
│   │   ├── aws-smithy-types v1.1.1 (*)
│   │   ├── aws-types v1.1.1 (*)
│   ├── aws-sdk-sso v1.9.0
│   │   ├── aws-credential-types v1.1.1 (*)
│   │   ├── aws-http v0.60.1 (*)
│   │   ├── aws-runtime v1.1.1 (*)
│   │   ├── aws-smithy-async v1.1.1 (*)
│   │   ├── aws-smithy-http v0.60.1 (*)
│   │   ├── aws-smithy-json v0.60.1
│   │   │   └── aws-smithy-types v1.1.1 (*)
│   │   ├── aws-smithy-runtime v1.1.1
│   │   │   ├── aws-smithy-async v1.1.1 (*)
│   │   │   ├── aws-smithy-http v0.60.1 (*)
│   │   │   ├── aws-smithy-protocol-test v0.60.1
│   │   │   │   ├── aws-smithy-runtime-api v1.1.1 (*)
│   │   │   ├── aws-smithy-runtime-api v1.1.1 (*)
│   │   │   ├── aws-smithy-types v1.1.1 (*)
│   │   ├── aws-smithy-runtime-api v1.1.1 (*)
│   │   ├── aws-smithy-types v1.1.1 (*)
│   │   ├── aws-types v1.1.1 (*)
│   ├── aws-sdk-ssooidc v1.9.0
│   │   ├── aws-credential-types v1.1.1 (*)
│   │   ├── aws-http v0.60.1 (*)
│   │   ├── aws-runtime v1.1.1 (*)
│   │   ├── aws-smithy-async v1.1.1 (*)
│   │   ├── aws-smithy-http v0.60.1 (*)
│   │   ├── aws-smithy-json v0.60.1 (*)
│   │   ├── aws-smithy-runtime v1.1.1 (*)
│   │   ├── aws-smithy-runtime-api v1.1.1 (*)
│   │   ├── aws-smithy-types v1.1.1 (*)
│   │   ├── aws-types v1.1.1 (*)
│   ├── aws-sdk-sts v1.9.0
│   │   ├── aws-credential-types v1.1.1 (*)
│   │   ├── aws-http v0.60.1 (*)
│   │   ├── aws-runtime v1.1.1 (*)
│   │   ├── aws-smithy-async v1.1.1 (*)
│   │   ├── aws-smithy-http v0.60.1 (*)
│   │   ├── aws-smithy-json v0.60.1 (*)
│   │   ├── aws-smithy-query v0.60.1
│   │   │   ├── aws-smithy-types v1.1.1 (*)
│   │   ├── aws-smithy-runtime v1.1.1 (*)
│   │   ├── aws-smithy-runtime-api v1.1.1 (*)
│   │   ├── aws-smithy-types v1.1.1 (*)
│   │   ├── aws-smithy-xml v0.60.1
│   │   ├── aws-types v1.1.1 (*)
│   ├── aws-smithy-async v1.1.1 (*)
│   ├── aws-smithy-http v0.60.1 (*)
│   ├── aws-smithy-json v0.60.1 (*)
│   ├── aws-smithy-runtime v1.1.1 (*)
│   ├── aws-smithy-runtime-api v1.1.1 (*)
│   ├── aws-smithy-types v1.1.1 (*)
│   ├── aws-types v1.1.1 (*)
├── aws-sdk-s3 v1.11.0
│   ├── aws-credential-types v1.1.1 (*)
│   ├── aws-http v0.60.1 (*)
│   ├── aws-runtime v1.1.1 (*)
│   ├── aws-sigv4 v1.1.1 (*)
│   ├── aws-smithy-async v1.1.1 (*)
│   ├── aws-smithy-checksums v0.60.1
│   │   ├── aws-smithy-http v0.60.1 (*)
│   │   ├── aws-smithy-types v1.1.1 (*)
│   ├── aws-smithy-eventstream v0.60.1 (*)
│   ├── aws-smithy-http v0.60.1 (*)
│   ├── aws-smithy-json v0.60.1 (*)
│   ├── aws-smithy-runtime v1.1.1 (*)
│   ├── aws-smithy-runtime-api v1.1.1 (*)
│   ├── aws-smithy-types v1.1.1 (*)
│   ├── aws-smithy-xml v0.60.1 (*)
│   ├── aws-types v1.1.1 (*)
├── aws-smithy-runtime v1.1.1 (*)
├── aws-smithy-runtime-api v1.1.1 (*)
├── aws-smithy-types v1.1.1 (*)

Environment details (OS name and version, etc.)

AL2 Lambda environment - not relevant to bug

Logs

No response

[aajtodd]

Thanks I've been able to reproduce this.

The issue is the SDK makes use of the S3 model which uses the paginated traitfor ListParts operation defined as:

                "smithy.api#paginated": {
                    "inputToken": "PartNumberMarker",
                    "outputToken": "NextPartNumberMarker",
                    "items": "Parts",
                    "pageSize": "MaxParts"
                }

It's generating code based off of this.

We'll have to implement a customization to workaround this (Kotlin customization for this).