Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Please provide a robust way to resume the upload #2872

Open
arunj-kp opened this issue Apr 19, 2022 · 0 comments
Open

Please provide a robust way to resume the upload #2872

arunj-kp opened this issue Apr 19, 2022 · 0 comments
Labels
feature-request Request a new feature s3 Issues with the AWS Android SDK for Simple Storage Service (S3).

Comments

@arunj-kp
Copy link
Contributor

arunj-kp commented Apr 19, 2022

Which AWS Services is the feature request for?
S3 TransferUtility under com.amazonaws:aws-android-sdk-s3:<latest_release_version>

Code snippet which I use for upload/resume.

override fun doWork(){
--- // some other code
        val uploadObserver = if (runAttemptCount == 0 || transferRecordId == -1) {
            transferUtility?.upload(uploadKey, mediaFile)
        } else {
            transferUtility?.resume(transferRecordId)
        }
--- // some other code

Is your feature request related to a problem? Please describe.
Yes, multipart upload feature is not effective during a worker restart if the process is recreated.

Currently, TransferUtility expects the client app to pass transferRecordId for resuming the upload process. The problem with this approach is:
(1) The client has to get the transferRecordId by listening to theTransferListener and keep it. There are possibilities that the client app may not receive a transferId from the SDK (or the client might not have handled it properly). In this case, the upload cannot be resumed.
(2) The app has to add the transferId to persistent storage for resuming the upload during a process restart (like an app crash, or a worker restart after a process restart, etc.). It is not the responsibility of the app to maintain the transferId, which is used only by the SDK for tracking upload/pause/resume.

Describe the solution you'd like
The upload and resume should have similar arguments. The SDK can use transferRecordId internally, but the app should not worry about transferRecordId, instead it should pass uploadKey and file (same arguments which the upload() method uses).

Please add a resume method as below:

/**
 * Resumes the transfer task with the upload key. You can resume a transfer in
 * paused, canceled or failed state. If a transfer is in waiting or in progress
 * state but it isn't actually running, this operation will force it to run.
 *
 * @param key  The key in the specified bucket by which to store the new object.
 * @param file The file to upload.
 * @return A TransferObserver used to track upload progress and state
 */
public TransferObserver resume(String key, File file) {
    // Add code for getting the transferId for the upload key
    int id = getTransferIdForUploadKey(key) // A new method which SDK has to implement
    submitTransferJob(TRANSFER_RESUME, id);
    return getTransferById(id);
}

Describe alternatives you've considered
Currently, I am maintaining a session variable for keeping transferRecordId as a work around. But, this won't work during a process restart and the upload starts from the beginning (by ignoring all previously uploaded parts)

Additional context
I have provided all details in this ticket. In case if you need some more inputs, please let me know.

@sdhuka sdhuka added s3 Issues with the AWS Android SDK for Simple Storage Service (S3). feature-request Request a new feature labels Apr 19, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature-request Request a new feature s3 Issues with the AWS Android SDK for Simple Storage Service (S3).
Projects
None yet
Development

No branches or pull requests

2 participants