Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MediaUpload with has_stream() = False and an empty last chunk results in HTTP Error 400 #1801

Open
john0312 opened this issue May 16, 2022 · 0 comments · May be fixed by #1802
Open

MediaUpload with has_stream() = False and an empty last chunk results in HTTP Error 400 #1801

john0312 opened this issue May 16, 2022 · 0 comments · May be fixed by #1802
Assignees
Labels
priority: p2 Moderately-important priority. Fix may not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.

Comments

@john0312
Copy link

Environment details

  • OS type and version: Ubuntu 20.04
  • Python version: Python 3.8.10
  • pip version: pip 20.0.2
  • google-api-python-client version: Version: 2.46.0

Steps to reproduce

  1. Create a class that inherit off MediaUpload, but resumable() = True and has_stream() = False so getbytes() will be used.
  2. Do an upload that is an exact multiples of the chunksize.
  3. The last chunk will be empty, and the header will look like this:
{'Content-Length': '0', 'Content-Range': 'bytes 268435456-268435455/268435456'}
  1. This will yield the HTTP 400 error code.

Code example

Note: Please fill in a folder ID in metadata['parents'] before running.

from __future__ import print_function

import os.path

from google.auth.transport.requests import Request
from google.oauth2.credentials import Credentials
from google_auth_oauthlib.flow import InstalledAppFlow
from googleapiclient.discovery import build
from googleapiclient.errors import HttpError
from googleapiclient.http import MediaUpload

# If modifying these scopes, delete the file token.json.
SCOPES = ["https://www.googleapis.com/auth/drive", "https://www.googleapis.com/auth/drive.file", "https://www.googleapis.com/auth/drive.metadata.readonly"]

class ExampleUpload(MediaUpload):
  def __init__(self):
    self.data = b'\0'*(200*1024*1024)

  def chunksize(self):
    return 100*1024*1024

  def mimetype(self):
    return "application/octet-stream"

  def size(self):
    return None

  def resumable(self):
    return True

  def has_stream(self):
    return False

  def getbytes(self, begin, length):
    length = min(len(self.data)-begin, length)
    return self.data[begin:begin+length]

def main():
    """Shows basic usage of the Drive v3 API.
    Prints the names and ids of the first 10 files the user has access to.
    """
    creds = None
    # The file token.json stores the user's access and refresh tokens, and is
    # created automatically when the authorization flow completes for the first
    # time.
    if os.path.exists('token.json'):
        creds = Credentials.from_authorized_user_file('token.json', SCOPES)
    # If there are no (valid) credentials available, let the user log in.
    if not creds or not creds.valid:
        if creds and creds.expired and creds.refresh_token:
            creds.refresh(Request())
        else:
            flow = InstalledAppFlow.from_client_secrets_file(
                'credentials.json', SCOPES)
            creds = flow.run_local_server(port=0)
        # Save the credentials for the next run
        with open('token.json', 'w') as token:
            token.write(creds.to_json())

    try:
        service = build('drive', 'v3', credentials=creds)

        metadata = {}
        metadata['name'] = 'test_file1'
        metadata['parents'] = <FOLDER ID HERE>
        media_body = ExampleUpload()
        request = service.files().create(body=metadata, media_body=media_body)
        response = None
        while response is None:
          status, response = request.next_chunk()
          print('Got status and response: %s %s'%(status, response))
    except HttpError as error:
        # TODO(developer) - Handle errors from drive API.
        print(f'An error occurred: {error}')


if __name__ == '__main__':
    main()

Stack trace

  File "/venv/lib/python3.8/site-packages/googleapiclient/_helpers.py", line 131, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/venv/lib/python3.8/site-packages/googleapiclient/http.py", line 1097, in next_chunk
    return self._process_response(resp, content)
  File "/venv/lib/python3.8/site-packages/googleapiclient/http.py", line 1128, in _process_response
    raise HttpError(resp, content, uri=self.uri)
googleapiclient.errors.HttpError: <HttpError 400 when requesting https://www.googleapis.com/upload/drive/v3/files?alt=json&uploadType=resumable returned "Bad Request". Details: "Failed to parse Content-Range header.">
@yoshi-automation yoshi-automation added the triage me I really want to be triaged. label May 17, 2022
@parthea parthea added type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns. priority: p1 Important issue which blocks shipping the next release. Will be fixed prior to next release. and removed triage me I really want to be triaged. labels May 17, 2022
@parthea parthea self-assigned this May 17, 2022
@yoshi-automation yoshi-automation added 🚨 This issue needs some love. and removed 🚨 This issue needs some love. labels May 22, 2022
@parthea parthea added priority: p2 Moderately-important priority. Fix may not be included in next release. and removed priority: p1 Important issue which blocks shipping the next release. Will be fixed prior to next release. 🚨 This issue needs some love. labels Jun 27, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
priority: p2 Moderately-important priority. Fix may not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.
Projects
None yet
3 participants