Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(lambda) File uploads #3926

Merged
merged 17 commits into from
Apr 10, 2020
Merged

Conversation

mathix420
Copy link
Contributor

@mathix420 mathix420 commented Mar 28, 2020

Fix #1419, fix #1703 file upload using lambda.
Based on #3676 and #1739

TODO:

  • Update CHANGELOG.md with your change (include reference to issue & this PR)
  • Make sure all of the significant new logic is covered by tests
  • Rebase your changes on master so that they can be merged easily
  • Make sure all tests and linter rules pass

After reading both #3676 and #1739 and seeing no further advancements, I convinced myself to write this PR. I tried to consider all the comments from old PRs and merge best parts of both ideas into this one.
Hoping it can help this feature to move forward 💯

Big thanks to @charleswong28, @k00k and @smschick for having being built all the logic !
Don't hesitate to tell me if there is anything to change.

Closes #3676
Closes #1739

@apollo-cla
Copy link

@mathix420: Thank you for submitting a pull request! Before we can merge it, you'll need to sign the Apollo Contributor License Agreement here: https://contribute.apollographql.com/

@maxgr0
Copy link

maxgr0 commented Mar 28, 2020

Would love to see this merged! Thanks for your work @mathix420

@mathix420
Copy link
Contributor Author

@abernix or @trevor-scheer is it normal that circleci is not passing on my commits ?

@craigjmidwinter
Copy link

Thanks @mathix420! would love to see one of the three PRs that were created to address this issue get merged in

@levimichael
Copy link

Thanks so much @mathix420!

Hope this gets approved soon!

@jordanskole
Copy link

🙏🏼amazing thank you for this! Hopefully gets merged soon!

@abernix abernix self-assigned this Apr 10, 2020
This was effectively `undefined` as it's not exported from the
`apollo-server-integration-testsuite` package.

Node.js 8 wasn't LTS until much later anyhow, so no need to even try to
support this as we never officially support non-LTS versions of Node.js
@abernix abernix changed the base branch from master to release-2.13.0 April 10, 2020 11:37
@abernix abernix added this to the Release 2.13.0 milestone Apr 10, 2020
Copy link
Member

@abernix abernix left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for putting this together! This looks great, and I really appreciate you taking the time to reconcile the concerns from other PRs! I've added a few follow-up commits, but let's get it into an alpha release for testing. I've targeted this PR to the release-2.13.0 branch which will shortly become a release PR.

I realize that some of the progress on the previous PRs seemed uncertain, but there is absolutely harmony to be found in not having diverging patterns between the many integrations. Providing a PR to add functionality is always welcome, but it's often on the maintainers of the project to provide support for it into the future, so I hope there's some understanding as to why the asks were being made. Assuming this works out well in alpha testing, you'll have shown that it is possible to achieve that harmony by the patterns you've adopted within.

I've included co-authorship to the other PR authors in the merge for this, to give contribution credit in the project history. Thank you to all that were involved!

@abernix abernix changed the title Add support for file upload in lambda feat(lambda) File uploads Apr 10, 2020
@abernix abernix merged commit af7f2f0 into apollographql:release-2.13.0 Apr 10, 2020
@jordanskole
Copy link

Sorry @lassesteffen I think I jumped the gun, that didn't fix the issue. I was able to get past the UnhandledPromiseRejectionWarning but I was back to sending an empty object :(

@lassesteffen
Copy link

There, the header is handled case-sensitive so you have to have both content-type and Content-Type until its fixed in this module.

How can I set this header in apollo client or apollo-server-lambda?

@mathix420
Copy link
Contributor Author

Sorry @lassesteffen I think I jumped the gun, that didn't fix the issue. I was able to get past the UnhandledPromiseRejectionWarning but I was back to sending an empty object :(

Which client are you using ?

@jordanskole
Copy link

jordanskole commented Apr 10, 2020

@mathix420 ApolloClient 2.6.8

  const client = new ApolloClient({
    link: ApolloLink.from([
      new RetryLink(),
      onError(({ graphQLErrors, networkError }) => {
        if (graphQLErrors)
          graphQLErrors.forEach(({ message, locations, path }) =>
            console.error(
              `[GraphQL error]: Message: ${message}, Location: ${locations}, Path: ${path}`,
            ),
          );
        if (networkError) console.log(`[Network error]: ${networkError}`);
      }),
      createUploadLink({
        uri: `${REACT_APP_SERVER_URL}/graphql`,
        headers: { Authorization: token }
      })
      // new HttpLink({
      //   uri: `${REACT_APP_SERVER_URL}/graphql`,
      //   headers: { Authorization: token }
      // }),
    ]),
    cache: new InMemoryCache(),
  });

@maxgr0
Copy link

maxgr0 commented Apr 10, 2020

There, the header is handled case-sensitive so you have to have both content-type and Content-Type until its fixed in this module.

How can I set this header in apollo client or apollo-server-lambda?

directly in your handler (before server.createHandler(...))

if(Object.keys(event.headers).includes('Content-Type')){ event.headers['content-type'] = event.headers['Content-Type']; }

@mathix420
Copy link
Contributor Author

@jordanskole I'm not sure if the your client can be the problem but one simple way to know that is by using Altair client.
If you can upload your image and get back your file infos from Altair you'll need to change your client configuration

@jordanskole
Copy link

jordanskole commented Apr 11, 2020

Thanks @mathix420, I tried that and I ran into the same error with Altair 😞so far I have been using serverless-offline. I will try and deploy and see if that fixes the issue, unless there are other ideas.

@maxgr0
Copy link

maxgr0 commented Apr 11, 2020

Thanks @mathix420, I tried that and I ran into the same error with Altair 😞so far I have been using serverless-offline. I will try and deploy and see if that fixes the issue, unless there are other ideas.

Have you tried changing the headers that you have both lowercase and uppercase content-type?

@jordanskole
Copy link

I'm not sure I am following where I can modify them, if not in context? My handler.ts file doesn't have access to event/context?

@maxgr0
Copy link

maxgr0 commented Apr 11, 2020

I'm not sure I am following where I can modify them, if not in context? My handler.ts file doesn't have access to event/context?

Can you share me your handler.ts?

@jordanskole
Copy link

Yeah, it's actually up above

handler.ts

import { ApolloServer, defaultPlaygroundOptions } from 'apollo-server-lambda'
import { schema as typeDefs } from './graphql/schema'
import { resolvers } from './graphql/resolvers';
import { createContext, ContextArguments } from './graphql/context'


const server = new ApolloServer({
  typeDefs,
  resolvers,
  playground: true, // defaultPlaygroundOptions,
  context: ({ event, context }: ContextArguments) => createContext({ event, context }),
});

const graphql = server.createHandler({
  cors: {
    origin: true,
    credentials: true,
  },
});


export { graphql }

graphql/context:

import { Context } from 'aws-lambda';
import Shopify, { IShop } from 'shopify-api-node'
import { getShopDetailsFromToken } from '../lib/jwt'

export type ContextArguments = {
  event: any;
  context: Context;
}

export const createContext = async ({ event, context }: ContextArguments) => {
  try {
    // get the user token from the headers
    const { headers } = event;
    console.log(headers)
    const token: string = event.headers.Authorization || '';
    const shop = token ? await getShopDetailsFromToken(token) : false;
    const shopify = shop ? new Shopify({...shop, autoLimit: true }) : false;
    return { headers, shop, shopify }
  } catch(e) {
    console.log(e)
    return;
  }
}

@maxgr0
Copy link

maxgr0 commented Apr 11, 2020

@jordanskole try this 👋 (the APIGatewayProxyHandler-type needs to be imported from @types/aws-lambda)
The related issue from busboy can be found here: mscdex/busboy#210

export const graphql: APIGatewayProxyHandler = (event, context, callback) => {
  if(Object.keys(event.headers).includes('Content-Type')){
    event.headers['content-type'] = event.headers['Content-Type'];
  }
  const handler = server.createHandler({
    cors: {
      origin: true,
      credentials: true
    }
  });
  return handler(event, context, callback);
};

@jordanskole
Copy link

jordanskole commented Apr 11, 2020

That was it! 🙏🏼feel free to share your paypal email with me I'd love to buy you a beer!

@maxgr0
Copy link

maxgr0 commented Apr 11, 2020

That was it! 🙏🏼feel free to share your email with me I'd love to buy you a beer!

all good, cheers!

@jordanskole
Copy link

Hey @mathix420 , I'm back!

I have my application running fine locally, but I am having trouble actually deploying to AWS, and I think it might be related to this, since the error is getting tossed from the fileUploadHandler function.

I'm currently running alpha.1 and I haven't had any issues running everything locally. This is of course barking about preflight CORS when I get to staging, but I think because preflight is failing since all of the below errors are happing in playground and CORS shouldn't be an issue there -- but perhaps I am wrong?

Should this be a separate issue?

Hitting playground throws the following error (on POST request):

{
    "errorType": "SyntaxError",
    "errorMessage": "Unexpected token e in JSON at position 0",
    "stack": [
        "SyntaxError: Unexpected token e in JSON at position 0",
        "    at JSON.parse (<anonymous>)",
        "    at graphqlHandler (/var/task/node_modules/apollo-server-lambda/dist/lambdaApollo.js:27:26)",
        "    at /var/task/node_modules/apollo-server-lambda/dist/ApolloServer.js:169:16",
        "    at fileUploadHandler (/var/task/node_modules/apollo-server-lambda/dist/ApolloServer.js:163:28)",
        "    at /var/task/node_modules/apollo-server-lambda/dist/ApolloServer.js:166:13",
        "    at Runtime.exports.graphql [as handler] (/var/task/src/handler.js:31:12)",
        "    at Runtime.handleOnce (/var/runtime/Runtime.js:66:25)"
    ]
}

Here is the complete event:

{
  resource: '/graphql',
  path: '/graphql',
  httpMethod: 'POST',
  headers: {
    Accept: '*/*',
    'Accept-Encoding': 'gzip, deflate, br',
    'Accept-Language': 'en-US,en;q=0.9',
    'apollo-query-plan-experimental': '1',
    'CloudFront-Forwarded-Proto': 'https',
    'CloudFront-Is-Desktop-Viewer': 'true',
    'CloudFront-Is-Mobile-Viewer': 'false',
    'CloudFront-Is-SmartTV-Viewer': 'false',
    'CloudFront-Is-Tablet-Viewer': 'false',
    'CloudFront-Viewer-Country': 'US',
    'content-type': 'application/json',
    Host: 'hz59wd6bjh.execute-api.us-east-1.amazonaws.com',
    origin: 'https://hz59wd6bjh.execute-api.us-east-1.amazonaws.com',
    Referer: 'https://hz59wd6bjh.execute-api.us-east-1.amazonaws.com/v1/graphql',
    'sec-fetch-dest': 'empty',
    'sec-fetch-mode': 'cors',
    'sec-fetch-site': 'same-origin',
    'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.122 Safari/537.36',
    Via: '2.0 66ba388c3807ced8474a06fdfcdde4fb.cloudfront.net (CloudFront)',
    'X-Amz-Cf-Id': 'IqNtkUm3dDd-9Jt5PiSnAiMgSi3OIR3IV8qZWHboOOha6x0kKI4eeQ==',
    'X-Amzn-Trace-Id': 'Root=1-5eaecec2-96c15af525480f00a215d6e0',
    'x-apollo-tracing': '1',
    'X-Forwarded-For': '67.149.113.209, 70.132.37.71',
    'X-Forwarded-Port': '443',
    'X-Forwarded-Proto': 'https'
  },
  multiValueHeaders: {
    Accept: [ '*/*' ],
    'Accept-Encoding': [ 'gzip, deflate, br' ],
    'Accept-Language': [ 'en-US,en;q=0.9' ],
    'apollo-query-plan-experimental': [ '1' ],
    'CloudFront-Forwarded-Proto': [ 'https' ],
    'CloudFront-Is-Desktop-Viewer': [ 'true' ],
    'CloudFront-Is-Mobile-Viewer': [ 'false' ],
    'CloudFront-Is-SmartTV-Viewer': [ 'false' ],
    'CloudFront-Is-Tablet-Viewer': [ 'false' ],
    'CloudFront-Viewer-Country': [ 'US' ],
    'content-type': [ 'application/json' ],
    Host: [ 'hz59wd6bjh.execute-api.us-east-1.amazonaws.com' ],
    origin: [ 'https://hz59wd6bjh.execute-api.us-east-1.amazonaws.com' ],
    Referer: [
      'https://hz59wd6bjh.execute-api.us-east-1.amazonaws.com/v1/graphql'
    ],
    'sec-fetch-dest': [ 'empty' ],
    'sec-fetch-mode': [ 'cors' ],
    'sec-fetch-site': [ 'same-origin' ],
    'User-Agent': [
      'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.122 Safari/537.36'
    ],
    Via: [
      '2.0 66ba388c3807ced8474a06fdfcdde4fb.cloudfront.net (CloudFront)'
    ],
    'X-Amz-Cf-Id': [ 'IqNtkUm3dDd-9Jt5PiSnAiMgSi3OIR3IV8qZWHboOOha6x0kKI4eeQ==' ],
    'X-Amzn-Trace-Id': [ 'Root=1-5eaecec2-96c15af525480f00a215d6e0' ],
    'x-apollo-tracing': [ '1' ],
    'X-Forwarded-For': [ '67.149.113.209, 70.132.37.71' ],
    'X-Forwarded-Port': [ '443' ],
    'X-Forwarded-Proto': [ 'https' ]
  },
  queryStringParameters: null,
  multiValueQueryStringParameters: null,
  pathParameters: null,
  stageVariables: null,
  requestContext: {
    resourceId: 'tdveht',
    resourcePath: '/graphql',
    httpMethod: 'POST',
    extendedRequestId: 'L9U-XGDnoAMFtKw=',
    requestTime: '03/May/2020:14:01:38 +0000',
    path: '/v1/graphql',
    accountId: '028260149026',
    protocol: 'HTTP/1.1',
    stage: 'v1',
    domainPrefix: 'hz59wd6bjh',
    requestTimeEpoch: 1588514498259,
    requestId: '0a30768c-e1cb-4020-9e81-d901ae10f448',
    identity: {
      cognitoIdentityPoolId: null,
      accountId: null,
      cognitoIdentityId: null,
      caller: null,
      sourceIp: '67.149.113.209',
      principalOrgId: null,
      accessKey: null,
      cognitoAuthenticationType: null,
      cognitoAuthenticationProvider: null,
      userArn: null,
      userAgent: 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.122 Safari/537.36',
      user: null
    },
    domainName: 'hz59wd6bjh.execute-api.us-east-1.amazonaws.com',
    apiId: 'hz59wd6bjh'
  },
  body: 'eyJvcGVyYXRpb25OYW1lIjoiSW50cm9zcGVjdGlvblF1ZXJ5IiwidmFyaWFibGVzIjp7fSwicXVlcnkiOiJxdWVyeSBJbnRyb3NwZWN0aW9uUXVlcnkge1xuICBfX3NjaGVtYSB7XG4gICAgcXVlcnlUeXBlIHtcbiAgICAgIG5hbWVcbiAgICB9XG4gICAgbXV0YXRpb25UeXBlIHtcbiAgICAgIG5hbWVcbiAgICB9XG4gICAgc3Vic2NyaXB0aW9uVHlwZSB7XG4gICAgICBuYW1lXG4gICAgfVxuICAgIHR5cGVzIHtcbiAgICAgIC4uLkZ1bGxUeXBlXG4gICAgfVxuICAgIGRpcmVjdGl2ZXMge1xuICAgICAgbmFtZVxuICAgICAgZGVzY3JpcHRpb25cbiAgICAgIGxvY2F0aW9uc1xuICAgICAgYXJncyB7XG4gICAgICAgIC4uLklucHV0VmFsdWVcbiAgICAgIH1cbiAgICB9XG4gIH1cbn1cblxuZnJhZ21lbnQgRnVsbFR5cGUgb24gX19UeXBlIHtcbiAga2luZFxuICBuYW1lXG4gIGRlc2NyaXB0aW9uXG4gIGZpZWxkcyhpbmNsdWRlRGVwcmVjYXRlZDogdHJ1ZSkge1xuICAgIG5hbWVcbiAgICBkZXNjcmlwdGlvblxuICAgIGFyZ3Mge1xuICAgICAgLi4uSW5wdXRWYWx1ZVxuICAgIH1cbiAgICB0eXBlIHtcbiAgICAgIC4uLlR5cGVSZWZcbiAgICB9XG4gICAgaXNEZXByZWNhdGVkXG4gICAgZGVwcmVjYXRpb25SZWFzb25cbiAgfVxuICBpbnB1dEZpZWxkcyB7XG4gICAgLi4uSW5wdXRWYWx1ZVxuICB9XG4gIGludGVyZmFjZXMge1xuICAgIC4uLlR5cGVSZWZcbiAgfVxuICBlbnVtVmFsdWVzKGluY2x1ZGVEZXByZWNhdGVkOiB0cnVlKSB7XG4gICAgbmFtZVxuICAgIGRlc2NyaXB0aW9uXG4gICAgaXNEZXByZWNhdGVkXG4gICAgZGVwcmVjYXRpb25SZWFzb25cbiAgfVxuICBwb3NzaWJsZVR5cGVzIHtcbiAgICAuLi5UeXBlUmVmXG4gIH1cbn1cblxuZnJhZ21lbnQgSW5wdXRWYWx1ZSBvbiBfX0lucHV0VmFsdWUge1xuICBuYW1lXG4gIGRlc2NyaXB0aW9uXG4gIHR5cGUge1xuICAgIC4uLlR5cGVSZWZcbiAgfVxuICBkZWZhdWx0VmFsdWVcbn1cblxuZnJhZ21lbnQgVHlwZVJlZiBvbiBfX1R5cGUge1xuICBraW5kXG4gIG5hbWVcbiAgb2ZUeXBlIHtcbiAgICBraW5kXG4gICAgbmFtZVxuICAgIG9mVHlwZSB7XG4gICAgICBraW5kXG4gICAgICBuYW1lXG4gICAgICBvZlR5cGUge1xuICAgICAgICBraW5kXG4gICAgICAgIG5hbWVcbiAgICAgICAgb2ZUeXBlIHtcbiAgICAgICAgICBraW5kXG4gICAgICAgICAgbmFtZVxuICAgICAgICAgIG9mVHlwZSB7XG4gICAgICAgICAgICBraW5kXG4gICAgICAgICAgICBuYW1lXG4gICAgICAgICAgICBvZlR5cGUge1xuICAgICAgICAgICAgICBraW5kXG4gICAgICAgICAgICAgIG5hbWVcbiAgICAgICAgICAgICAgb2ZUeXBlIHtcbiAgICAgICAgICAgICAgICBraW5kXG4gICAgICAgICAgICAgICAgbmFtZVxuICAgICAgICAgICAgICB9XG4gICAgICAgICAgICB9XG4gICAgICAgICAgfVxuICAgICAgICB9XG4gICAgICB9XG4gICAgfVxuICB9XG59XG4ifQ==',
  isBase64Encoded: true
}

and that body base64decoded (just an introspection)

{"operationName":"IntrospectionQuery","variables":{},"query":"query IntrospectionQuery {\n  __schema {\n    queryType {\n      name\n    }\n    mutationType {\n      name\n    }\n    subscriptionType {\n      name\n    }\n    types {\n      ...FullType\n    }\n    directives {\n      name\n      description\n      locations\n      args {\n        ...InputValue\n      }\n    }\n  }\n}\n\nfragment FullType on __Type {\n  kind\n  name\n  description\n  fields(includeDeprecated: true) {\n    name\n    description\n    args {\n      ...InputValue\n    }\n    type {\n      ...TypeRef\n    }\n    isDeprecated\n    deprecationReason\n  }\n  inputFields {\n    ...InputValue\n  }\n  interfaces {\n    ...TypeRef\n  }\n  enumValues(includeDeprecated: true) {\n    name\n    description\n    isDeprecated\n    deprecationReason\n  }\n  possibleTypes {\n    ...TypeRef\n  }\n}\n\nfragment InputValue on __InputValue {\n  name\n  description\n  type {\n    ...TypeRef\n  }\n  defaultValue\n}\n\nfragment TypeRef on __Type {\n  kind\n  name\n  ofType {\n    kind\n    name\n    ofType {\n      kind\n      name\n      ofType {\n        kind\n        name\n        ofType {\n          kind\n          name\n          ofType {\n            kind\n            name\n            ofType {\n              kind\n              name\n              ofType {\n                kind\n                name\n              }\n            }\n          }\n        }\n      }\n    }\n  }\n}\n"}

@mathix420
Copy link
Contributor Author

Hi @jordanskole, hope you're fine

Not sure but it looks like your issue is identical as this one #2599

@jordanskole
Copy link

jordanskole commented May 5, 2020

Hey @mathix420

Thanks for your help! That did fix the first issue I had, but now I am still struggling.

All my other queries and mutations work in my staging deployment after base64 decoding the request, except for my single fileUpload mutation resolver, which attempts to stream data to cloudinary.

Cloudinary is throwing the error which I am investigating, but it appears that my resolver is not building the stream correctly, or is closing the stream, since I see multiple (3+) invocations of my resolver, which makes me think AWS Gateway/Lambda is not handling the multipart request correctly (???)

I think the error may have something to do with the lower-case content-type header again, since API Gateway says it refers to the Content-Type header: "API Gateway will look at the Content-Type and Accept HTTP headers to decide how to handle the body."

image
(from API Gateway settings in aws console)

API Gateway enacts the following restrictions and limitations when handling methods with either Lambda integration or HTTP integration.

  • Header names and query parameters are processed in a case-sensitive way.

~ AWS API Gateway known issues

Both Altair and createUploadLink (from apollo-upload-client) are setting the lower-case content-type header

@mathix420
Copy link
Contributor Author

after base64 decoding the request

Did you skip the decoding when a multipart/form-data is recieved ?
If not have you put event.isBase64Encoded to false ?

@jordanskole
Copy link

jordanskole commented May 5, 2020

@mathix420 amazing! you are my hero again. You are definitely getting a shoutout on this app when it launches. For anybody that finds their way back here, this is what my complete handler looks like now:

export const graphql: APIGatewayProxyHandler | APIGatewayProxyResult = (event, context, callback) => {
  try {

    if(Object.keys(event.headers).includes('Content-Type')){
      event.headers['content-type'] = event.headers['Content-Type'];
    }
    
    if (
      event.isBase64Encoded
      && event.body
      && (
        !event.headers['content-type'].includes('multipart/form-data')
      )
    ) {
       event = {
         ...event,
         body: Buffer.from(event.body, "base64").toString(),
         isBase64Encoded: false
       }
    }
    const handler = server.createHandler({
      cors: {
        origin: true,
        credentials: true
      }
    });
    return handler(event, context, callback);
  } catch (e) {
    console.log(e)
    return callback(e)
  }
};

@lanwen
Copy link

lanwen commented Jul 31, 2020

Thank you for the wrapper, but with aws api gateway v2 and payload format v1 gives error:

{
    "errorType": "Runtime.UnhandledPromiseRejection",
    "errorMessage": "TypeError [ERR_INVALID_ARG_TYPE]: The first argument must be of type string or an instance of Buffer, ArrayBuffer, or Array or an Array-like Object. Received an instance of Object",
    "reason": [
        {
            "errorType": "TypeError",
            "errorMessage": "The first argument must be of type string or an instance of Buffer, ArrayBuffer, or Array or an Array-like Object. Received an instance of Object",
            "message": "The first argument must be of type string or an instance of Buffer, ArrayBuffer, or Array or an Array-like Object. Received an instance of Object",
            "extensions": {
                "code": "INTERNAL_SERVER_ERROR"
            }
        }
    ],
    "promise": {},
    "stack": [
        "Runtime.UnhandledPromiseRejection: TypeError [ERR_INVALID_ARG_TYPE]: The first argument must be of type string or an instance of Buffer, ArrayBuffer, or Array or an Array-like Object. Received an instance of Object",
        "    at process.<anonymous> (/var/runtime/index.js:35:15)",
        "    at process.emit (events.js:315:20)",
        "    at processPromiseRejections (internal/process/promises.js:209:33)",
        "    at processTicksAndRejections (internal/process/task_queues.js:98:32)"
    ]
}

what could be the reason for that?

@lassesteffen
Copy link

I have the same error.

@Umkus
Copy link

Umkus commented Aug 5, 2020

Same error here as well, plus:

...
About to exit with code: 128
Unknown application error occurred

@mathix420
Copy link
Contributor Author

@lanwen could you please give us more context ?

@lanwen
Copy link

lanwen commented Aug 5, 2020

@mathix420 Its a apollo-server-lambda, with one mutation calling apollo-upload

We use aws-cdk (1.55) to deploy the lambda packed with webpack (nothing special there, just handler as in examples above ^^^ as input and output, using ts)

const fn = new lambda.Function(this, id, {
      runtime: lambda.Runtime.NODEJS_12_X,
      handler: "lambda.handler",
      memorySize: 256,
      code: lambda.Code.fromAsset(lambdaAsset),
    });

    const api = new gw.HttpApi(this, "graphql-api", {
      createDefaultStage: false,
    });

    api.addRoutes({
      path: "/graphql",
      methods: [gw.HttpMethod.POST, gw.HttpMethod.GET, gw.HttpMethod.OPTIONS],
      integration: new gw.LambdaProxyIntegration({
        handler: fn,
        // https://github.com/apollographql/apollo-server/issues/3958
        payloadFormatVersion: gw.PayloadFormatVersion.VERSION_1_0,
      }),
    });

    const stage = new gw.HttpStage(this, "DeploymentStage", {
      httpApi: api,
      stageName: environment,
      autoDeploy: true,
    });

Here is an event which comes into handler

{
  version: '1.0',
  resource: '/graphql',
  path: '/staging/graphql',
  httpMethod: 'POST',
  headers: {
    'Content-Length': '5288',
    'Content-Type': 'multipart/form-data; boundary=---------------------------7180963393185668272703501002',
    Host: 'some-api-id.execute-api.region.amazonaws.com',
    'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:79.0) Gecko/20100101 Firefox/79.0',
    'X-Amzn-Trace-Id': 'Root=1-5f282a99-5ad69cbafb4710',
    'X-Forwarded-For': '1.2.3.4',
    'X-Forwarded-Port': '443',
    'X-Forwarded-Proto': 'https',
    accept: '*/*',
    'accept-encoding': 'gzip, deflate, br',
    'accept-language': 'en-GB,en;q=0.5',
    authorization: 'eyJh...HgfdA',
    origin: 'https://something.cloudfront.net',
    referer: 'https://something.cloudfront.net/something/'
  },
  multiValueHeaders: {
    'Content-Length': [ '5288' ],
    'Content-Type': [
      'multipart/form-data; boundary=---------------------------7180963393185668272703501002'
    ],
    Host: [ 'some-api-id.execute-api.region.amazonaws.com' ],
    'User-Agent': [
      'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:79.0) Gecko/20100101 Firefox/79.0'
    ],
    'X-Amzn-Trace-Id': [ 'Root=1-5f282a4710' ],
    'X-Forwarded-For': [ '1.2.3.4' ],
    'X-Forwarded-Port': [ '443' ],
    'X-Forwarded-Proto': [ 'https' ],
    accept: [ '*/*' ],
    'accept-encoding': [ 'gzip, deflate, br' ],
    'accept-language': [ 'en-GB,en;q=0.5' ],
    authorization: [
      'eyJhbGciOiJ...HgfdA'
    ],
    origin: [ 'https://something.cloudfront.net' ],
    referer: [ 'https://something.cloudfront.net/something/' ]
  },
  queryStringParameters: null,
  multiValueQueryStringParameters: null,
  requestContext: {
    accountId: '1',
    apiId: 'some-api-id',
    domainName: 'some-api-id.execute-api.region.amazonaws.com',
    domainPrefix: 'some-api-id',
    extendedRequestId: 'QsBQ=',
    httpMethod: 'POST',
    identity: {
      accessKey: null,
      accountId: null,
      caller: null,
      cognitoAuthenticationProvider: null,
      cognitoAuthenticationType: null,
      cognitoIdentityId: null,
      cognitoIdentityPoolId: null,
      principalOrgId: null,
      sourceIp: '1.2.3.4',
      user: null,
      userAgent: 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:79.0) Gecko/20100101 Firefox/79.0',
      userArn: null
    },
    path: '/staging/graphql',
    protocol: 'HTTP/1.1',
    requestId: 'QQ=',
    requestTime: '03/Aug/2020:15:17:45 +0000',
    requestTimeEpoch: 1596467865307,
    resourceId: 'POST /graphql',
    resourcePath: '/graphql',
    stage: 'staging'
  },
  pathParameters: null,
  stageVariables: null,
  body: 'LS0tLS0tL...(some base64 png)...aA2RgwG...zM5MzE4NTY2ODI3MjcwMzUwMTAwMi0tDQo=',
  isBase64Encoded: true
}

dependencies:

"dependencies": {
    "@types/form-data": "^2.5.0",
    "apollo-datasource-rest": "^0.9.3",
    "apollo-server": "^2.16.0",
    "apollo-server-lambda": "^2.16.0",
    "commander": "^6.0.0",
    "form-data": "^3.0.0",
    "graphql": "^14.0.0",
    "graphql-iso-date": "^3.6.1"
  },

request doesn't come to our resolver and got failed somewhere before. Wasn't able to reproduce with serverless offline locally, didn't try if dependent on the deployment type (like using old rest-api)

@Umkus
Copy link

Umkus commented Aug 5, 2020

Seems like this always happens, unless I replace the event.body with Buffer.from(event.body, 'base64').toString() and set isBase64Encoded to false. But then the images seem broken for me.

@mathix420
Copy link
Contributor Author

mathix420 commented Aug 5, 2020

I think you should check this issue #4430, it seems that this issue is due to recent changes #4311

As far as I can see a simple condition to not decode event.body if contentType.startsWith("multipart/form-data") just here L37 and all should works fine.

@Umkus
Copy link

Umkus commented Aug 5, 2020

This setup worked for me (using 2.16.1):

export function handler(event, context, callback) {
  if (Object.keys(event.headers).includes('Content-Type')) {
    event.headers['content-type'] = event.headers['Content-Type'];
  }

  if (event.isBase64Encoded && event.body && event.headers['content-type'].includes('multipart/form-data')) {
    event.body = Buffer.from(event.body, 'base64').toString('binary');
    event.isBase64Encoded = false;
  }

  return server.createHandler({
    cors: {
      origin: true,
      credentials: true,
    },
  })(event, context, callback);
}

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Apr 21, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[File Upload] - flawed error handling for bad JSON File uploads using apollo-server-lambda
10 participants