Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failed export googledocs : This file is too large to be exported. #3446

Open
SoniSuciadi opened this issue Feb 28, 2024 · 1 comment
Open
Assignees
Labels
priority: p2 Moderately-important priority. Fix may not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.

Comments

@SoniSuciadi
Copy link

SoniSuciadi commented Feb 28, 2024

Thanks for stopping by to let us know something could be better!

PLEASE READ: If you have a support contract with Google, please create an issue in the support console instead of filing on GitHub. This will ensure a timely response.

  1. Is this a client library issue or a product issue?
    This is the client library for . We will only be able to assist with issues that pertain to the behaviors of this library. If the issue you're experiencing is due to the behavior of the product itself, please visit the Support page to reach the most relevant engineers.

  2. Did someone already solve this?

  1. Do you have a support contract?
    Please create an issue in the support console to ensure a timely response.

If the support paths suggested above still do not result in a resolution, please provide the following details.

Environment details

  • OS: MacOs
  • Node.js version: v18.16.1
  • npm version:9.8.1
  • googleapis version: 133.0.0
import { readFileSync } from 'fs';
import { google } from 'googleapis';
import path from 'path';
import { Readable } from 'stream';

export const bufferToStream = (buffer: Buffer): Readable => {
  const stream = new Readable();
  stream.push(buffer);
  stream.push(null);
  return stream;
};
export const createEmptyStream = (): Readable => {
  const stream = new Readable();
  stream._read = () => {};
  stream.push(null);
  return stream;
};

 export async function downloadPDFDoc(fileId: string): Promise<Buffer> {
  try {
    const KEYFILEPATH = path.join(__dirname, 'happyhomes-360410-4644e17e212c.json');
    const credentials = JSON.parse(readFileSync(KEYFILEPATH, 'utf8'));

    const auth = new google.auth.GoogleAuth({
      credentials,
      scopes: ['https://www.googleapis.com/auth/drive'],
    });
    const drive = google.drive({
      version: 'v3',
      auth,
    });
    const response = await drive.files.export(
      {
        fileId: fileId,
        mimeType: 'application/pdf',
      },
      { responseType: 'stream' },
    );

    return new Promise((resolve, reject) => {
      const chunks: Buffer[] = [];
      response.data.on('data', (chunk: Buffer) => {
        chunks.push(chunk);
      });
      response.data.on('end', () => {
        console.log('File download completed.');
        resolve(Buffer.concat(chunks));
      });
      response.data.on('error', (err: Error) => {
        console.error('Error downloading file:', err);
        reject(err);
      });
    });
  } catch (error) {
    console.error('The API returned an error: ' + error);
    throw error;
  }
}```

The total documents I exported were 18 MB, and when I ran this service, I got this error
```js GaxiosError: {
  "error": {
    "code": 400,
    "message": "This file is too large to be exported.",
    "errors": [
      {
        "message": "This file is too large to be exported.",
        "domain": "global",
        "reason": "badRequest",
        "location": "export.size",
        "locationType": "other"
      }
    ]
  }
}

    at Gaxios._request (/Users/sonisuciadi/Happyhomes/rem-be/node_modules/gaxios/src/gaxios.ts:174:15)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at JWT.requestAsync (/Users/sonisuciadi/Happyhomes/rem-be/node_modules/google-auth-library/build/src/auth/oauth2client.js:405:18)
    at downloadPDFDoc (/Users/sonisuciadi/Happyhomes/rem-be/src/helpers/google-docs/downloadPDFDoc.ts:57:26)
    at publishOfferDocController (/Users/sonisuciadi/Happyhomes/rem-be/src/controllers/project/procurement/procurementDetailOutputJob.controllers.ts:241:32) {
  config: {
    url: 'https://www.googleapis.com/drive/v2/files/1G11FllbKw6qLbYJM1N8TdZpb6G2Q4QazslxsgLw9oO4/export?mimeType=application%2Fpdf',
    method: 'GET',
    responseType: 'stream',
    userAgentDirectives: [ [Object] ],
    paramsSerializer: [Function (anonymous)],
    headers: {
      'x-goog-api-client': 'gdcl/7.0.1 gl-node/20.6.1',
      'Accept-Encoding': 'gzip',
      'User-Agent': 'google-api-nodejs-client/7.0.1 (gzip)',
      Authorization: 'Bearer ya29.c.c0AY_VpZi_y24RI-304tmmNmbUcghDfre0a7qnssZ_nX1xamM7YfmaxWHscd6djR88HBzBzgCLjEQLOqc_GDrGloS3jqaEMa6Pa7_R3InWNRhP8lYdjVKt5_WmFhmfeq7R9rQsLn804hB34_T1o82bW2GecegBaPF0MeDJvoAOMv-xEqb21cOUC4rBdqo-jDWGu0wrwx0otC0UBDfbRbhsrlKV75Yoe204NouUWSaEjJiMRgzOskPb2fmbDMEkTKAgsfk1J2teFrpXhbgGi4M3kStpRgAG2H8cUhTyE9ubcwcT62oIgvG3Rldfa6y0-5A04HyXR1jtHweJsFCL57zJ4jgRNICEOi6A_l1s16EaUYIk3W0CWSXpuaIE384PUel96bBbc4iO3XRimlOpYSXtorulM795V0zauSrp_wrqUozQ2UOstJMF7o8l204ei39jeFFwpI2BM_xlh54ROWt6mbwwY312qeB50aWz0M3wvxddqu3a_20x-zyokwhpMmh0fbm40IIOaFa-Y1B834BY6aUfyQwSRZq6UfO0kUyoWMah1ywmkqMV9kz75ta6ekSnY0QwroMQYbId31dMOWZuaOcaZV9Ok9OdxapZM8B1d8U-QuXORsJv_fa6IdYUgdjwz4rwFkJiM6_zSfkU200urFzOphvcdVkYoppW7mr67rSBznrwqYI0677FtJUqthfnrtJ_ZJpo-t9ougguffaQipJwjFwZmyfoS1fOubfzqatBVnurW87R2B-JlY997ob3bffMZISuQ27MXt9JM5hfYgbyRns39g57m4v6ktVOBvVq7am8w-wqse7kbi0QIa5JjYBoeetzbMu454uev27UZiumv9kIMVFo0e9kp1V-wfXqIymj-74ybm9JyQ8wmqckcI8s9O9q5hXJ2nop_9iR_tj5QqsBmSnZtlBY57MleSsgy_IXW5ktzjez9Rw2SjakQ_i0xl6pi7wZ3q5mX8o2F45vj2Mo7woJB-BhpVcvScq0Fqbv9iavFF7'
    },
    params: { mimeType: 'application/pdf' },
    validateStatus: [Function (anonymous)],
    retry: true,
    errorRedactor: [Function: defaultErrorRedactor],
    retryConfig: {
      currentRetryAttempt: 0,
      retry: 3,
      httpMethodsToRetry: [Array],
      noResponseRetries: 2,
      statusCodesToRetry: [Array]
    }
  },
  response: {
    config: {
      url: 'https://www.googleapis.com/drive/v2/files/1G11FllbKw6qLbYJM1N8TdZpb6G2Q4QazslxsgLw9oO4/export?mimeType=application%2Fpdf',
      method: 'GET',
      responseType: 'stream',
      userAgentDirectives: [Array],
      paramsSerializer: [Function (anonymous)],
      headers: [Object],
      params: [Object],
      validateStatus: [Function (anonymous)],
      retry: true,
      errorRedactor: [Function: defaultErrorRedactor]
    },
    data: '{\n' +
      '  "error": {\n' +
      '    "code": 400,\n' +
      '    "message": "This file is too large to be exported.",\n' +
      '    "errors": [\n' +
      '      {\n' +
      '        "message": "This file is too large to be exported.",\n' +
      '        "domain": "global",\n' +
      '        "reason": "badRequest",\n' +
      '        "location": "export.size",\n' +
      '        "locationType": "other"\n' +
      '      }\n' +
      '    ]\n' +
      '  }\n' +
      '}\n',
    headers: {
      'alt-svc': 'h3=":443"; ma=2592000,h3-29=":443"; ma=2592000',
      'cache-control': 'private, max-age=0',
      'content-length': '321',
      'content-type': 'application/json; charset=UTF-8',
      date: 'Wed, 28 Feb 2024 16:28:46 GMT',
      expires: 'Wed, 28 Feb 2024 16:28:46 GMT',
      server: 'ESF',
      vary: 'Origin, X-Origin',
      'x-content-type-options': 'nosniff',
      'x-frame-options': 'SAMEORIGIN',
      'x-guploader-uploadid': 'ABPtcPqCOaSGjiUlM-J9pa8kUkAtcLeR4NDexI63N2ecZ7dCEkTryxssJKL7HmpIDi5QJWZyPp3eaFaUew',
      'x-xss-protection': '0'
    },
    status: 400,
    statusText: 'Bad Request',
    request: {
      responseURL: 'https://www.googleapis.com/drive/v2/files/1G11FllbKw6qLbYJM1N8TdZpb6G2Q4QazslxsgLw9oO4/export?mimeType=application%2Fpdf'
    }
  },
  error: undefined,
  status: 400,
  [Symbol(gaxios-gaxios-error)]: '6.3.0'
}

Thanks!

@SoniSuciadi SoniSuciadi added priority: p2 Moderately-important priority. Fix may not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns. labels Feb 28, 2024
@danieljbruce danieljbruce self-assigned this Feb 29, 2024
@mn-featurewave
Copy link

Any updates on this? This still looks like there and not able to download files larger than 12MB or so.

@danieljbruce danieljbruce assigned sofisl and unassigned danieljbruce Apr 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
priority: p2 Moderately-important priority. Fix may not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.
Projects
None yet
Development

No branches or pull requests

4 participants