Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

allow PATCH requests in ResumableUpload.initiate / constructor #388

Open
tswast opened this issue Jul 20, 2023 · 0 comments
Open

allow PATCH requests in ResumableUpload.initiate / constructor #388

tswast opened this issue Jul 20, 2023 · 0 comments
Labels
api: storage Issues related to the googleapis/google-resumable-media-python API. priority: p3 Desirable enhancement or fix. May not be included in next release. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.

Comments

@tswast
Copy link
Contributor

tswast commented Jul 20, 2023

Is your feature request related to a problem? Please describe.

I'd love to be able to use this package to replace files in Google Drive using the following API: https://developers.google.com/drive/api/reference/rest/v3/files/update

Notably, this requires a PATCH request instead of a POST request.

Describe the solution you'd like

Either the ResumableUpload constructor or initiate should accept an optional "method" argument to use PATCH instead of POST.

Describe alternatives you've considered

I was able to get this to work with the following code sample, but note that it monkeypatches a private API to work.

# Copyright 2023 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

"""
Upload latest wheel to Google Drive.

Based on
https://github.com/googleapis/google-resumable-media-python/blob/main/google/resumable_media/requests/__init__.py

Before running, execute the following to make sure you can use the Google Drive API:

gcloud auth application-default login --scopes=openid,https://www.googleapis.com/auth/userinfo.email,https://www.googleapis.com/auth/cloud-platform,https://www.googleapis.com/auth/sqlservice.login,https://www.googleapis.com/auth/drive
"""

import pathlib

import google.auth
import google.auth.transport.requests
import google.resumable_media._upload
import google.resumable_media.requests as resumable_requests

repo_root = pathlib.Path(__file__).parent.parent

# TODO(tswast): This shouldn't be modifying private global constants.
# Use PATCH instead of POST to replace existing files.
google.resumable_media._upload._POST = "PATCH"

credentials, _ = google.auth.default(scopes=["https://www.googleapis.com/auth/drive"])
transport = google.auth.transport.requests.AuthorizedSession(credentials)

wheel_id = "15fZ1DkrFDk4ibMNTzms4akpxmf2pzeAR"
wheel_path = next(iter((repo_root / "dist").glob("bigframes-*.whl")))

pdf_id = "1agYjxmPLrxelsaHI-lc41QHcgnQYemcX"
pdf_path = repo_root / "docs" / "_build" / "latex" / "bigframes-latest.pdf"

uploads = (
    (wheel_id, wheel_path, "application/octet-stream"),
    (pdf_id, pdf_path, "application/pdf"),
)

upload_template = (
    "https://www.googleapis.com/upload/drive/v3/files/{file_id}?uploadType=resumable"
)
chunk_size = 1024 * 1024  # 1MB

for file_id, file_path, content_type in uploads:
    print(f"Uploading {file_path}")
    transport = google.auth.transport.requests.AuthorizedSession(credentials)
    upload = resumable_requests.ResumableUpload(
        upload_template.format(file_id=file_id), chunk_size
    )

    with open(file_path, "rb") as stream:
        response = upload.initiate(
            transport, stream, metadata={}, content_type=content_type
        )
        print(response)
        while not upload.finished:
            response = upload.transmit_next_chunk(transport)
            print(response)

Additional context

It's somewhat surprising not to see this package mentioned in https://developers.google.com/drive/api/guides/manage-uploads#resumable It'd be great to get some code samples there.

@product-auto-label product-auto-label bot added the api: storage Issues related to the googleapis/google-resumable-media-python API. label Jul 20, 2023
@cojenco cojenco added type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. priority: p3 Desirable enhancement or fix. May not be included in next release. labels Jul 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: storage Issues related to the googleapis/google-resumable-media-python API. priority: p3 Desirable enhancement or fix. May not be included in next release. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.
Projects
None yet
Development

No branches or pull requests

2 participants