Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error: Source and destination must not be the same. When using useServerlessTraceTarget #461

Closed
TuuZzee opened this issue Jun 23, 2020 · 31 comments · Fixed by #758 or #760
Closed

Comments

@TuuZzee
Copy link

TuuZzee commented Jun 23, 2020

This is most likely some misconfiguration on my part but I can't figure out what exactly is wrong. I reported this issue with a comment on #405 but I believe it's more appropriate to create a bug report. Let me know if this isn't the right way :)

Describe the bug
When running serverless with the Inputs useServerlessTraceTarget: true the build fails with the following output:

$->serverless --debug

  DEBUG ─ Resolving the template's static variables.
  DEBUG ─ Collecting components from the template.
  DEBUG ─ Downloading any NPM components found in the template.
  DEBUG ─ Analyzing the template's components dependencies.
  DEBUG ─ Creating the template's components graph.
  DEBUG ─ Syncing template state.
  DEBUG ─ Executing the template's components graph.

  error:
  Error: Source and destination must not be the same.
    at .serverless/components/registry/npm/serverless-next.js@1.15.0-alpha.0/node_modules/@sls-next/lambda-at-edge/node_modules/fs-extra/lib/util/stat.js:39:17
    at cb (util.js:196:39)
    at processTicksAndRejections (internal/process/task_queues.js:81:21)

  100s › webApp › Error: Source and destination must not be the same.

To Reproduce
With the following configuration:

  • package.json:
  "dependencies": {
    "aws-amplify": "^3.0.18",
    "axios": "^0.19.2",
    "axios-retry": "^3.1.8",
    "babel-loader": "^8.1.0",
    "babel-plugin-styled-components": "^1.10.7",
    "bignumber.js": "^9.0.0",
    "bootstrap": "^4.5.0",
    "dotenv": "^8.2.0",
    "dotenv-webpack": "^1.8.0",
    "firebase": "^7.15.3",
    "humps": "^2.0.1",
    "immer": "^6.0.8",
    "install": "^0.13.0",
    "jquery": "^3.5.1",
    "jwt-decode": "^2.2.0",
    "localforage": "^1.7.4",
    "lodash": "^4.17.15",
    "moment": "^2.27.0",
    "next": "^9.4.4",
    "next-images": "^1.4.0",
    "next-redux-wrapper": "^5.0.0",
    "node-sass": "^4.14.1",
    "npm": "^6.14.5",
    "nprogress": "^0.2.0",
    "prop-types": "^15.7.2",
    "raw-loader": "^4.0.1",
    "react": "^16.13.1",
    "react-bootstrap-table": "^4.3.1",
    "react-burger-menu": "^2.6.17",
    "react-circular-progressbar": "^2.0.3",
    "react-countdown": "^2.2.1",
    "react-dom": "^16.13.1",
    "react-intl": "^3.11.0",
    "react-markdown": "^4.3.1",
    "react-redux": "^7.2.0",
    "react-redux-firebase": "^3.6.0",
    "react-redux-toastr": "^7.6.5",
    "react-retina-image": "^2.0.5",
    "reactstrap": "^8.4.1",
    "redux": "^4.0.5",
    "redux-devtools-extension": "^2.13.8",
    "redux-firestore": "^0.13.0",
    "redux-logger": "^3.0.6",
    "redux-thunk": "^2.3.0",
    "reselect": "^4.0.0",
    "sass": "^1.26.8",
    "sass-loader": "^8.0.2",
    "styled-components": "^5.1.1",
    "styled-jsx": "^3.3.0",
    "uuid": "^8.1.0",
    "validator": "^13.1.1"
  },
  "devDependencies": {
    "babel-eslint": "^10.1.0",
    "eslint": "^7.3.1",
    "eslint-config-airbnb": "^18.2.0",
    "eslint-config-prettier": "^6.11.0",
    "eslint-import-resolver-alias": "^1.1.2",
    "eslint-plugin-import": "^2.21.2",
    "eslint-plugin-jsx-a11y": "^6.3.1",
    "eslint-plugin-no-secrets": "^0.6.8",
    "eslint-plugin-prettier": "^3.1.4",
    "eslint-plugin-react": "^7.20.0",
    "eslint-plugin-react-hooks": "^4.0.4",
    "eslint-plugin-security": "^1.4.0",
    "husky": "^4.2.5",
    "lint-staged": "^10.2.11",
    "prettier": "^2.0.5",
    "prettier-eslint": "^10.1.1",
    "prettier-eslint-cli": "^5.0.0",
    "serverless-next.js": "^1.15.0-alpha.0",
    "snazzy": "^8.0.0"
  },
  • serverless.yml:
webApp:
  component: serverless-next.js@1.15.0-alpha.0
  inputs:
    useServerlessTraceTarget: true
    bucketName: "webAppBucket"
    description: "WebApp Client QA NextJS Lambda@Edge for Cloudfront distribution"
    name:
      defaultLambda: webAppLambda
  • next.config.js:
require('dotenv').config();

const withImages = require('next-images');

module.exports = withImages({
  crossOrigin: 'anonymous',
  target: 'experimental-serverless-trace',
});

Desktop (please complete the following information):

  • Node v12.14.1
@danielcondemarin
Copy link
Contributor

@TuuZzee Could you post your directory structure? i.e. Are you in a monorepo and from which directory you are running next build.

@TuuZzee
Copy link
Author

TuuZzee commented Jun 24, 2020

It's a monorepo for this project, here is the current directory and file structure structure:

.
├── .firebase/
├── .github/
├── .serverless/
├── .serverless_nextjs/
├── node_modules/
├── pages/
├── public/
│ ├── static/
├── src/
│ ├── components/
│ ├── context/
│ ├── locale/
│ ├── redux/
│ ├── styles/
│ └── utils/
├── .babelrc
├── .env
├── qa.env
├── prod.env
├── .eslintrc.js
├── .eslintignore
├── .firebaserc
├── .gitignore
├── .prettierignore
├── .prettierrc
├── firebase.json
├── jsconfig.json
├── next-config.js
├── package.json
├── package-lock.json
├── serverless.yml
└── README.md

I am running the next build from the root of the project.
I have the following scripts in the package.json file:

  "scripts": {
    "dev": "next",
    "build": "next build",
    "export": "rm -rf .next out && next build && next export",
    "start": "next start",
    "deploy": "serverless",
  },

@TuuZzee
Copy link
Author

TuuZzee commented Jun 30, 2020

I try to use the useServerlessTraceTarget: true with a basic NextJS project to make to make sure it's coming from my project, it worked fine all fine. So I added the packages and files/folders structure incrementally in hope of finding the root cause. I was able to add all the pkgs and files/folders structure with no issue.
All worked fine until I copied files via the Finder from mac OS. At that point I got the same error again. I tried to delete all the .DS_Store files but it didn't fixed it. Even after reverting the changes to the version that was working last the error still persist. I am not really sure what the problem is.

Any thing I could give a try to ?

@danielcondemarin
Copy link
Contributor

@TuuZzee If you provide a simple reproduction of the problem in a repo I can access that would be greatly appreciated. I'll take a look at it asap.

@TuuZzee
Copy link
Author

TuuZzee commented Jun 30, 2020

I have made a boilerplate here: https://github.com/TuuZzee/boilerplate-NextJS based on this issue. Just need to run the usual npm install and add a .env file. I get the error on it as well, hope this helps.

GitHub
Contribute to TuuZzee/boilerplate-NextJS development by creating an account on GitHub.

@TuuZzee
Copy link
Author

TuuZzee commented Jul 3, 2020

Any luck with the repo I provided ?

I am not sure I understand which Source and destination this error is referring to. It's not building pages under the .serverless_nextjs/default-lambda, I am guessing that could be what this is about.

I tried removing all non essentials files from the project to see if that affect anything(things like editor config files etc...), but it doesn't change the issue.

@danielcondemarin
Copy link
Contributor

Any luck with the repo I provided ?

I am not sure I understand which Source and destination this error is referring to. It's not building pages under the .serverless_nextjs/default-lambda, I am guessing that could be what this is about.

I tried removing all non essentials files from the project to see if that affect anything(things like editor config files etc...), but it doesn't change the issue.

Thanks for the repro! I've not had a had a chance to look into it but I'm hoping to get some time this weekend.

@danielcondemarin
Copy link
Contributor

@TuuZzee I haven’t been able to reproduce the issue using the repro. Are there any specific steps I need to take?

@shoNagai
Copy link

I have the same problem.
I'm also using useServerlessTraceTarget.
When I debugged fs-extra , the file pointed out was package.json.

λ  (Lambda)  server-side renders at runtime (uses getInitialProps or getServerSideProps)
○  (Static)  automatically rendered as static HTML (uses no initial props)
●  (SSG)     automatically generated as static HTML + JSON (uses getStaticProps)

🔥🔥🔥🔥🔥🔥🔥 /Users/nagaishou/projects/Gaudiy/module/gaudiy-react-next-aws/package.json
💀💀💀💀💀💀💀 /Users/nagaishou/projects/Gaudiy/module/gaudiy-react-next-aws/package.json

  error:
  Error: Source and destination must not be the same.
    at /Users/nagaishou/.serverless/components/registry/npm/@sls-next/serverless-component@1.15.2-alpha.0/node_modules/fs-extra/lib/util/stat.js:41:17
    at cb (util.js:207:31)
    at processTicksAndRejections (internal/process/task_queues.js:84:21)

  1283s › gaudiy-react-next2 › Error: Source and destination must not be the same.

Is there a solution such as exclude?

@danielcondemarin
Copy link
Contributor

@shoNagai Could you post your app directory structure? Include as much detail as you can please 🙂

@shoNagai
Copy link

@danielcondemarin Thank you.
The current directory is as follows.

├── Dockerfile
├── README.md
├── cloudbuild.yaml
├── config
│   ├── vercel.develop-tokyo.json
├── dangerfile.ts
├── docker-compose.yml
├── e2e
│   ├── Dockerfile
│   ├── README.md
│   ├── cloudbuild.yaml
│   ├── cypress
│   ├── cypress.json
│   ├── package-lock.json
│   ├── package.json
│   └── tsconfig.json
├── firebase.json
├── firestore.indexes.json
├── firestore.rules
├── index.ts
├── jest.base.config.js
├── jest.config.client.integration.js
├── jest.config.client.js
├── jest.config.snapshot.js
├── next-env.d.ts
├── next.config.js
├── node_modules
├── package.json
├── pages
│   ├── _app.tsx
│   ├── _document.tsx
│   ├── _error.tsx
│   ├── api
│   └── index.tsx
├── public
│   └── static
├── serverless.yml
├── src
│   └── app
├── storybook-static
│   ├── favicon.ico
│   ├── index.html
│   ├── main.0c66b2c8223310d1dcdc.bundle.js
│   ├── preview-head.html
│   ├── runtime~main.0266b089acdf4a16d4c2.bundle.js
│   ├── sb_dll
│   └── vendors~main.a4d6753ad10f6b406738.bundle.js
├── tsconfig.firebase.json
├── tsconfig.json
├── tsconfig.server.json
├── tsconfig.test.json
├── vercel.json
└── yarn.lock

Here is the main script.(package.json)

"build": "NODE_OPTIONS=\"--max_old_space_size=8192 && --gc_interval=100\" NODE_ENV=production next build",
"start": "next start",

@jschimmoeller
Copy link

jschimmoeller commented Aug 4, 2020

I am running into the same issue:
C:\Users\jschi\projects\sr-web>yarn test
yarn run v1.22.4
warning ....\package.json: No license field
$ ncp .test-next.config.js next.config.js && ncp test-serverless.yml serverless.yml
$ serverless

error:
Error: Source and destination must not be the same.
at C:\Users\jschi.serverless\components\registry\npm@sls-next\serverless-component@1.15.1\node_modules@sls-next\serverless-component\node_modules\fs-extra\lib\util\stat.js:39:17
at cb (util.js:207:31)
at processTicksAndRejections (internal/process/task_queues.js:84:21)

81s » sr-web » Error: Source and destination must not be the same.

error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

@jschimmoeller
Copy link

jschimmoeller commented Aug 4, 2020

Here is my dir structure

│ .gitignore
│ .local-next.config.js
│ .prod-next.config.js
│ .test-next.config.js
│ next.config.js
│ next.config.original.1596577128335.js
│ package.json
│ prod-serverless.yml
│ README.md
│ serverless.yml
│ test-serverless.yml
│ yarn-error.log
│ yarn.lock

├───components
│ │ acctAdminGroups.js
│ │ acctArticles.js
│ │ acctClasses.js
│ │ acctNotifications.js
│ │ acctPayment.js
│ │ acctProfile.js
│ │ acctUsers.js
│ │ addCard.js
│ │ addGroup.js
│ │ addLeader.js
│ │ addMember.js
│ │ addon.js
│ │ appContext.js
│ │ appendixCard.js
│ │ article.js
│ │ bugFeature.js
│ │ cancelAccount.js
│ │ careerCard.js
│ │ chapterCard.js
│ │ chatWidget-old.js
│ │ chatWidget.js
│ │ choosePlan.js
│ │ classCard.js
│ │ contactCard.js
│ │ customerTestimonialCard.js
│ │ deleteConfirm.js
│ │ deviceImages.js
│ │ deviceLimit.js
│ │ deviceScreenshots.js
│ │ downloadCard.js
│ │ downloadCertificate.js
│ │ endorsementCard.js
│ │ episodeCard.js
│ │ factCard.js
│ │ faqs.js
│ │ faster.js
│ │ footer.js
│ │ globalStyles.js
│ │ gridBackground.js
│ │ groupCard.js
│ │ header.js
│ │ helpCards.js
│ │ itemQuestion.js
│ │ layout.js
│ │ layoutAccount.js
│ │ ldrConfirm.js
│ │ leftNavLayout.js
│ │ lineChart.js
│ │ login.js
│ │ meetingCard.js
│ │ myEditor.js
│ │ myEpisodeNotes.js
│ │ myGroup.js
│ │ myGroupCard.js
│ │ newJournal.js
│ │ newWordCard.js
│ │ noDatesSet.js
│ │ noDelete.js
│ │ note.js
│ │ noteContainer.js
│ │ onOffSlider.js
│ │ payment.js
│ │ peopleCard.js
│ │ personCard.js
│ │ player.js
│ │ priceCard.js
│ │ promoCode.js
│ │ Question.js
│ │ rangeSlider.js
│ │ readme.md
│ │ reset.js
│ │ resetPassword.js
│ │ resourceCard.js
│ │ SAST.js
│ │ select.js
│ │ seriesCard.js
│ │ seriesCardNew.js
│ │ sgConfirm.js
│ │ share.js
│ │ sharedControl.js
│ │ slate-toolbar.js
│ │ slider.js
│ │ slider2.js
│ │ sliderButton.js
│ │ social.js
│ │ stepCard.js
│ │ studyGuideCard.js
│ │ svgAnimation.js
│ │ svgs.js
│ │ svgsCustomers.js
│ │ tools.js
│ │ topicCards.js
│ │ upgradeModal.js
│ │ usermenu.js
│ │ videoCard.js
│ │ wordCard.js
│ │
│ ├───accountability
│ │ answerCard.js
│ │ esCard.js
│ │ journalCard.js
│ │ nodataCard.js
│ │ scriptureCard.js
│ │ userFeedCard.js
│ │
│ ├───dashboard-cards
│ │ accountabilityCard.js
│ │ browseCoursesCard.js
│ │ classCard.js
│ │ groupsCard.js
│ │ healthCard.js
│ │ journalCard.js
│ │ leaderClassCard.js
│ │ leaderZoneCard.js
│ │ membersCard.js
│ │ nextMeetingCard.js
│ │ profileCheckCard.js
│ │ scriptureCard.js
│ │ upgradeToChurchCard.js
│ │ waitingListCard.js
│ │ watchCard.js
│ │
│ ├───health-cards
│ │ crashCard.js
│ │ emotionalStateCard.js
│ │ freedomCard.js
│ │ groupCard.js
│ │ sastCard.js
│ │ studyGuideCard.js
│ │ userCard.js
│ │ votdCard.js
│ │ weeklyGroupHealth.js
│ │
│ └───super-admin-cards
│ arrCard.js
│ avgGroupSizeCard.js
│ churchValueCard.js
│ churnCard.js
│ countryCard.js
│ dailyRevenueCard.js
│ dateRange.js
│ lifespanValueCard.js
│ mrrCard.js
│ productCard.js
│ sponsorGiftCard.js

├───misc
│ auth.js
│ awsToken.js
│ colorPallet.js
│ fontPallet.js
│ phoneinput.js
│ sast.json
│ timezones.json
│ useInterval.js
│ utils.js

├───pages
│ │ 404.js
│ │ about.js
│ │ account.js
│ │ accountability.js
│ │ buysgchurch.js
│ │ buysgindividual.js
│ │ buysgmember.js
│ │ certificate.js
│ │ classes.js
│ │ confirm.js
│ │ confirmLeader.js
│ │ confirmMember.js
│ │ confirmMemberGroupInvite.js
│ │ contact.js
│ │ contend.js
│ │ courses.js
│ │ crash.js
│ │ dashboard.js
│ │ editCourse.js
│ │ editGroup.js
│ │ editPeople.js
│ │ editStudyGuide.js
│ │ groupHealth.js
│ │ groups.js
│ │ health.js
│ │ help.js
│ │ helpItem.js
│ │ index.js
│ │ journal.js
│ │ leaderzone.js
│ │ login.js
│ │ notes.js
│ │ people.js
│ │ privacy.js
│ │ reactivate.js
│ │ relapse.js
│ │ reset.js
│ │ series.js
│ │ study_guide.js
│ │ svg.js
│ │ terms.js
│ │ toc.js
│ │ understanding.js
│ │ unsubscribe.js
│ │ upgrade.js
│ │ votd.js
│ │ waiting.js
│ │ watching.js
│ │ _app.js
│ │ _document.js
│ │ _error.js
│ │
│ └───api
│ activateUser.js
│ addCard.js
│ addGuestMember.js
│ addToWaitlist.js
│ bibleVerse.js
│ cancelPayment.js
│ canDeleteGroup.js
│ careers.js
│ changeEmail.js
│ checkDuplicateGroupName.js
│ checkDuplicatePeople.js
│ churchStatsData.js
│ churnStatsData.js
│ clearDevices.js
│ completeStudyGuideItem.js
│ contact.js
│ countAdminGroups.js
│ countAdminPeople.js
│ countGroups.js
│ countPeople.js
│ countries.js
│ createGroup.js
│ createLoginAccount.js
│ createPerson.js
│ createShortUrl.js
│ currentPlan.js
│ customerSupportEmail.js
│ customerTestimonials.js
│ dailyRevenueStatsData.js
│ deleteArticle.js
│ deleteCard.js
│ deleteGroup.js
│ deleteItem.js
│ deleteJournal.js
│ deleteNote.js
│ deletePerson.js
│ deleteUser.js
│ deleteVOTD.js
│ emotionalScaleData.js
│ endorsements.js
│ facts.js
│ faqs.js
│ findAccountUsers.js
│ fingerprint.js
│ getAccountabilityUsers.js
│ getArticles.js
│ getCertificateData.js
│ getClasses.js
│ getCourses.js
│ getGroupData.js
│ getGroupHealth.js
│ getGroupNotes.js
│ getGroups.js
│ getGroupUserHealth.js
│ getHealthScore.js
│ getMemberGroupDetails.js
│ getMembers.js
│ getNotes.js
│ getNotifications.js
│ getPeople.js
│ getPersonGroups.js
│ getPersonNotes.js
│ getPlan.js
│ getSeriesVideoPlayedStats.js
│ getSGPC.js
│ getSharedData.js
│ getStreamUserToken.js
│ getStudyGuide.js
│ getStudyGuideRaw.js
│ getTOCData.js
│ getUpcomingMeetings.js
│ getUser.js
│ getUserAccounts.js
│ getUserAccountsDetails.js
│ getUserAdmin.js
│ getUserJournal.js
│ getUserNotes.js
│ getUsers.js
│ getUserSharedData.js
│ getWaitingList.js
│ groupStatsData.js
│ gwl.js
│ helpArticle.js
│ helpArticles.js
│ helpTopics.js
│ incompleteStudyGuideItem.js
│ lastRelapse.js
│ lastSast.js
│ lifespanStatsData.js
│ messageLeader.js
│ plans.js
│ processGift.js
│ processPayment.js
│ productData.js
│ promoCode.js
│ recurringData.js
│ relapseDetails.js
│ reportIt.js
│ resendMemberEmail.js
│ resetPassword.js
│ resetPasswordEmail.js
│ saveArticle.js
│ saveConfirmUserData.js
│ saveCourse.js
│ saveCustomer.js
│ saveEmotionalScale.js
│ saveJournal.js
│ saveLetter.js
│ saveNote.js
│ saveNotification.js
│ saveRaw.js
│ saveTestResults.js
│ saveVOTD.js
│ scriptureData.js
│ searchGroups.js
│ sendAbandonCartEmail.js
│ sendConfirmEmail.js
│ sendEmail.js
│ sendGeneralEmail.js
│ sendGuestInviteEmail.js
│ sendISGMbrEmail.js
│ sendLeaderEmail.js
│ sendMemberEmail.js
│ sendMemberInviteEmail.js
│ sendSastDetailReport.js
│ sendShareEmail.js
│ sendSponsorEmail.js
│ series.js
│ seriessoon.js
│ series_conquer.js
│ series_data.js
│ series_stronger.js
│ series_warpath.js
│ setDefaultCard.js
│ setupDemoData.js
│ sg_conquer.js
│ sg_data.js
│ sg_stronger.js
│ sg_warpath.js
│ signup.js
│ sponsorships.js
│ study_methods.js
│ subscriptionAddons.js
│ updateAddonSubscriptionQuantity.js
│ updateCustomer.js
│ updateDeviceLimit.js
│ updateEmotionalScale.js
│ updateGroup.js
│ updateGroupMemberStatus.js
│ updatePerson.js
│ updatePrimary.js
│ updateSubscription.js
│ updateUserAccountStudyGuideOption.js
│ updateVideoPlayedStat.js
│ uploadLogo.js
│ user.js
│ userHasStudyGuide.js
│ userPaymentInfo.js
│ userSystemMessages.js
│ validateHTML.js
│ validPreCustomer.js
│ validUpgradeSubscription.js
│ verifyAccount.js
│ videos.js
│ votd.js
│ webhookStripe.js

└───public

@jschimmoeller
Copy link

jschimmoeller commented Aug 5, 2020

@danielcondemarin i found the issue that is causing this error for my deployment. I have an api that uses html-validator. In my code if i comment out the require line (below); i no longer get the error.

// version 5.1.14
const validate_html = require("html-validator");

I will be trying other stuff to get around this issue since I do need to validate an html snippet / string passed in via this api call.

UPDATE:
I found out that html-validate has an issue with webpack cli imports; i have downgraded it 5.0.3 and it works.

Webpack If you are using Webpack you might have problems with the 5.1.x series due to html-validate and the cli imports. At the moment the best solution is probably to stick with the 5.0.3 version

thanks
James

@jamiechong
Copy link

Seems like other projects have been hit by this issue: apache/cordova-android#692
culprit? jprichardson/node-fs-extra#657

@dphang
Copy link
Collaborator

dphang commented Aug 12, 2020

I was also facing the same issue (getting Error: Source and destination must not be the same) and spent some time to debug what's happening. It seems to originate from:

copyLambdaHandlerDependencies(
fileList: string[],
reasons: NodeFileTraceReasons,
handlerDirectory: string
): Promise<void>[] {
return fileList
.filter((file) => {
// exclude "initial" files from lambda artefact. These are just the pages themselves
// which are copied over separately
return !reasons[file] || reasons[file].type !== "initial";
})
.map((filePath: string) => {
const resolvedFilePath = path.resolve(filePath);
const dst = normalizeNodeModules(
path.relative(this.serverlessDir, resolvedFilePath)
);
return fse.copy(
resolvedFilePath,
join(this.outputDir, handlerDirectory, dst)
);
});
}
.

I am not too familiar with the code, but it looks like it's copying node_modules dependencies into lambda handlers when serverlessTraceTarget is enabled:

copyTraces = this.copyLambdaHandlerDependencies(
fileList,
reasons,
DEFAULT_LAMBDA_CODE_DIR
);
.

In my case, filelist somehow had my src and pages files in it which are .ts and .tsx files, which I guess shouldn't be copied into the Lambda. This filelist is coming from this line:

const { fileList, reasons } = await nodeFileTrace(apiPages, {
(nodeFileTrace)

As a quick debug, I printed out the values of variables from copyLambdaHandlerDependencies for my tsx file:

outputDir: /Users/daniel/IdeaProjects/web-app/.serverless_nextjs
handlerDirectory: default-lambda
serverlessDir: /Users/daniel/IdeaProjects/web-app/.next/serverless
resolvedFilePath: /Users/daniel/IdeaProjects/web-app/src/mui/theme.tsx
dst: ../../src/mui/theme.tsx
joined dst: /Users/daniel/IdeaProjects/web-app/src/mui/theme.tsx (join(this.outputDir, handlerDirectory, dst))

These files like the above fail due to ../../, since when joining the src and dst becomes the same, failing the copy and finally the serverless command. So in this case, it's not a problem with node-fs-extra.

It works for files in node_modules since the ../../node_modules becomes node_modules due to normalization to remove ../../ in: https://github.com/serverless-nextjs/serverless-next.js/blob/d0a630035cab10e6ff70c5706f16c8564227630c/packages/libs/lambda-at-edge/src/lib/normalizeNodeModules.ts

Example values

outputDir: /Users/daniel/IdeaProjects/web-app/.serverless_nextjs
handlerDirectory: default-lambda
serverlessDir: /Users/daniel/IdeaProjects/web-app/.next/serverless
resolvedFilePath: /Users/daniel/IdeaProjects/web-app/node_modules/yallist/yallist.js
dst: node_modules/yallist/yallist.js
joined dst: /Users/daniel/IdeaProjects/web-app/.serverless_nextjs/default-lambda/node_modules/yallist/yallist.js

Quick Solution

I quickly updated:

.filter((file) => {
// exclude "initial" files from lambda artefact. These are just the pages themselves
// which are copied over separately
return !reasons[file] || reasons[file].type !== "initial";
})
but in the compiled build.js directly to filter filepaths that didn't contain node_modules, and the serverless command succeeds.

return fileList
            .filter((file) => {
            if (!file.includes('node_modules')) {
            	return false;
            }
            return !reasons[file] || reasons[file].type !== "initial";
        }

The above works (and is a quick fix if you're just deploying from a local machine, for CI/CD you'd have to find the build.js file and do this replacement using sed or similar) but not sure if this is a proper fix that should be a PR? I should figure out why filelist has my ts files, I did set baseUrl: src in my tsconfig.json, maybe that's why?

@danielcondemarin will also try to get a minimal repro when I get time (can't share above code since it's private), but hopefully this is helpful as a start.

FYI, I am running this on Mac OS 10.15.5 / Macbook Pro 16

@jamiechong
Copy link

Great work @dphang. I had traced the problem to build.js also, but didn't get much further. I can confirm that your solution fixes my build also. For my cases, the error was happening for files in the public folder:
The error thrown from fs-extra happens when:
src: /Users/chongjam/Sites/catalog-pages/app/public/locales/en/pdp.json
dest: /Users/chongjam/Sites/catalog-pages/app/public/locales/en/pdp.json

On an unrelated note: Now that my build deploys, I need to figure out why my lambda is 503ing...

@zoozalp
Copy link
Contributor

zoozalp commented Aug 18, 2020

I got a workaround.

#236 (comment)

zoozalp added a commit to asyasoft/serverless-next.js that referenced this issue Aug 18, 2020
@jaypeng2015
Copy link

@zoozalp I've tested this with 1.17.0-alpha.12 and seems still getting this problem, as described in #236 (comment).

@zoozalp
Copy link
Contributor

zoozalp commented Sep 11, 2020

Check your build size and make sure useServerlessTraceTarget is enabled.

@TuuZzee
Copy link
Author

TuuZzee commented Oct 26, 2020

This has happened to me again recently, while upgrading some packages to their latest version. It was only happening in my staging environment for some reasons, production was fine.

During this process I got hit by the following error in my CI/CD coming from NextJS:

You are using a non-standard "NODE_ENV" value in your environment. This creates inconsistencies in the project and is strongly advised against. Read more: https://err.sh/next.js/non-standard-node-env

In my case I was using qa so I changed it to staging. This still didn't do the trick.
What fixed it for me was to set NODE_ENV and ENV to production in my .env file. I added a new APP_ENV to handle in app env stuff like NextJS is recommending, since I can live with the rest being set to production.

I am not sure if that is the actual root cause of considering other contributions from this tread. Anyway hope it helps!

@Enalmada
Copy link
Contributor

Enalmada commented Nov 2, 2020

I am using typescript in my project and just got this error. Adding this to build.js like was called out above made it work:

            if (!file.includes('node_modules')) {
            	return false;
           }

Any reason this wouldn't be appropriate to turn into a permanent fix?

@dphang
Copy link
Collaborator

dphang commented Nov 2, 2020

@Enalmada I think that can be a fix but just wanted to make sure with @danielcondemarin it won't break other cases?

Edit: I think that is not quite the right fix, let me take a closer look.

Edit2: I believe I have fixed this for TypeScript projects, if still encountering issues for other types of projects, please open a new issue with details of the error messages. Thanks!

@jonahallibone
Copy link

jonahallibone commented Nov 3, 2020

I am suffering from this problem as well. I am using mikro-orm in my project, which uses TypeScript. If I build outside of the serverless environment, or on vercel, everything works as expected. If I try and build locally using the serverless command in my terminal, it errors out. Output from serverless --debug below

This is my config:

tabgrab:
  component: "@sls-next/serverless-component@1.18.0"
  inputs:
    domain: <domain>
    useServerlessTraceTarget: true

λ  (Lambda)  server-side renders at runtime (uses getInitialProps or getServerSideProps)
○  (Static)  automatically rendered as static HTML (uses no initial props)
●  (SSG)     automatically generated as static HTML + JSON (uses getStaticProps)
   (ISR)     incremental static regeneration (uses revalidate in getStaticProps)


  error:
  Error: Source and destination must not be the same.

EDIT: I tested on a vanilla create-next-js project without a problem, so like mentioned above, it makes me think that typescript is the problem. The code for this project can be found here if you need to see folder structure.

@dphang dphang reopened this Nov 3, 2020
@dphang
Copy link
Collaborator

dphang commented Nov 3, 2020

@jonahallibone thanks for that. Can you please try to use the latest alpha version which has the fix for TypeScript and see if that works. If not, you can also try to build this component from source and try adding some logging statements here to figure out why it is failing, e.g you can print out dst or other variables: https://github.com/serverless-nextjs/serverless-next.js/blob/master/packages/libs/lambda-at-edge/src/build.ts#L147-L155.

But for other project structures, I'm thinking we should just modify the fse.copy(resolvedFilePath, join(this.outputDir, handlerDirectory, dst)); line instead return a no-op promise when source and destination are the same.

@jonahallibone
Copy link

jonahallibone commented Nov 3, 2020

@dphang with the latest alpha I still get

jonah@MacBook-Pro-2 tabgrab % serverless

  error:
  Error: Source and destination must not be the same.
    at /Users/jonah/.serverless/components/registry/npm/@sls-next/serverless-component@1.18.1-alpha.1/node_modules/fs-extra/lib/util/stat.js:39:17
    at cb (util.js:196:39)
    at processTicksAndRejections (internal/process/task_queues.js:80:21)

  27s › tabgrab › Error: Source and destination must not be the same.

@dphang
Copy link
Collaborator

dphang commented Nov 3, 2020

@jonahallibone Yeah, probably the error message is not helpful as it's from fse.copy() failing due to source and destination being the same. I added a PR which should at least prevent those copies from happening - and let the build proceed, but not sure if it will be a correct build, but at least will be a better developer experience.

@jonahallibone
Copy link

@dphang I am more than happy to build from source and test it, but tbh I am a little unsure how to do that locally. If you have any pointers, I'd love to help test.

@dphang
Copy link
Collaborator

dphang commented Nov 3, 2020

@jonahallibone thanks. You can follow https://github.com/serverless-nextjs/serverless-next.js/blob/master/CONTRIBUTING.md to setup. Basically it involves:

  1. Updating code in packages/libs/lambda-at-edge (where I pointed to earlier). For example, you can add logging statements.
  2. Run yarn at top level to build all changes
  3. Point your serverless.yml file's component to nextjs-component directory where you built.

You could also pull the #760 branch and see if it helps. It at least should not fail, though I'm not completely if it will copy all sources correctly for different project structures. I will probably just merge it in a bit as it is safe and still better DX than before.

@jonahallibone
Copy link

jonahallibone commented Nov 3, 2020

@dphang I built locally and ran it, seems to have successfully deployed. Will update when the deployment finishes on AWS

EDIT: @dphang All is working as expected with the latest alpha. Thanks for the quick fix 👍

@dphang
Copy link
Collaborator

dphang commented Nov 5, 2020

Cool, thanks for the feedback.

No one else should be getting this exact issue anymore with the latest alpha version, but if you have related issues (e.g assets not copied to right place), please let us know by filing a new issue. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
10 participants