Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support mono-repos in deployment #653

Open
laurenzlong opened this issue Jan 31, 2018 · 172 comments
Open

Support mono-repos in deployment #653

laurenzlong opened this issue Jan 31, 2018 · 172 comments
Assignees

Comments

@laurenzlong
Copy link
Contributor

See: firebase/firebase-functions#172

Version info

3.17.3

Steps to reproduce

Expected behavior

Actual behavior

@dinvlad
Copy link

dinvlad commented Feb 22, 2018

This seems to work in my setup, i.e. deploy picks up packages from the root node_modules, even though package.json for that is located under the api/ workspace (I've used a different folder name instead of functions/). Is there anything else that needs fixing here?

EDIT: Moreover, I copy package.json into api/dist to be used by deploy.

// firebase.json
  ...
  "functions": {
    "source": "api/dist"
  },
  ...

So, 2 levels of nesting still resolve the root node_modules successfully.

@orouz
Copy link

orouz commented Apr 23, 2018

@dinvlad could you share a repo?

@dinvlad
Copy link

dinvlad commented Apr 23, 2018

@orouz unfortunately not yet, it's closed source for now.

@audkar
Copy link

audkar commented Jun 17, 2019

Does anyone managed to tackle this problem? Sharing simple example project would be very useful.

@jthegedus
Copy link
Contributor

@audkar Currently I just use lerna.js.org and it's run command to execute an npm script in each subfolder with this folder structure:

- service1/
|  - .firebaserc
|  - firebase.json
- service2/
|  - .firebaserc
|  - firebase.json
- app1/
|  - .firebaserc
|  - firebase.json
- app2/
|  - .firebaserc
|  - firebase.json
- firestore/
|  - firestore.rules
|  - firestore.indexes.json
- etc...

Ensuring the firebase.json files for each service don't stomp on one another is left up to the user. Simple conventions of using function groups and multi-site name targeting means this is solved for Cloud Functions and Hosting. Still haven't got a solution for Firestore/GCS rules yet, though splitting them up may not be ideal...

discussed here previously - #1116

@audkar
Copy link

audkar commented Jun 18, 2019

@jthegedus thank you for your reply. But I think issue of this ticket is different. I am trying to use yarn workspaces. And seems that firebase tools doesn't pickup symlink dependencies when uploading functions

@jthegedus
Copy link
Contributor

Ah fair enough, I've avoided that rabbit hole myself

@dinvlad
Copy link

dinvlad commented Jun 18, 2019

Could you elaborate what the issue is? As mentioned above, I just use bare Yarn with api and app workspaces in it, and I build them using yarn workspace api build && yarn workspace app build (with build script specific to each workspace). The build scripts

  1. compile TS code with outDir into api/dist and app/dist respectively
  2. copy the corresponding package.json files into dist directories
  3. copy yarn.lock from the root folder, into dist directories

Then I just run yarn firebase deploy from the root folder, and it picks up both api/dist and app/dist without any hiccups. My firebase.json looks like

  "functions": {
    "source": "api/dist"
  },
  "hosting": {
    "public": "app/dist",

Unfortunately, I still can’t share the full code, but this setup is all that matters, afaik.

@dinvlad
Copy link

dinvlad commented Jun 18, 2019

Also, I might be wrong but I think the firebase deploy script doesn’t actually use your node_modules directory. I think it just picks up the code, package.json, and yarn.lock from the dist directories, and does the rest.

@samtstern
Copy link
Contributor

samtstern commented Jun 18, 2019 via email

@jthegedus
Copy link
Contributor

@dinvlad Yes, it requires the package.json and whichever lock file you use as it installs deps in the cloud post deployment.

I believe the scenario originally outlined in the other issue was using a shared package within the workspace and some issues with scope-hoisting. As I was not using yarn this way I can only speculate from what I have read there.

@dinvlad
Copy link

dinvlad commented Jun 18, 2019

@samtstern @jthegedus thanks, good to know!

@audkar
Copy link

audkar commented Jun 18, 2019

Seems we all talk about different problems. I will try to describe yarn workspaces problem.

Problematic project

project layout

- utilities/
|  - package.json
- functions/
|  - package.json
- package.json

./package.json

{
  "private": true,
  "workspaces": ["functions", "utilities"]
}

functions/package.json

{
  <...>
  "dependencies": {
    "utilities": "1.0.0",
    <...>
  }
}

Problem

Error during function deployment:

Deployment error.
Build failed: {"error": {"canonicalCode": "INVALID_ARGUMENT", "errorMessage": "`gen_package_lock` had stderr output:\nnpm WARN deprecated left-pad@1.3.0: use String.prototype.padStart()\nnpm ERR! code E404\nnpm ERR! 404 Not Found: utilities@1.0.0\n\nnpm ERR! A complete log of this run can be found in:\nnpm ERR!     /builder/home/.npm/_logs/2019-06-18T07_10_42_472Z-debug.log\n\nerror: `gen_package_lock` returned code: 1", "errorType": "InternalError", "errorId": "1971BEF9"}}

Functions works fine locally on emulator.

Solutions tried

Uploading node_modules (using functions.ignore in firebase.json). Result is same.

My guess that it is because utilities is created as syslink in node-modules node_modules/utilities -> ../../utilities

Could it be that firebase-tools doesn't include content of symlink'ed modules when uploading (no dereferencing)?

@dinvlad
Copy link

dinvlad commented Jun 18, 2019

Sorry, could you clarify which folder your firebase.json lives in (and show its configuration section for functions)?

@audkar
Copy link

audkar commented Jun 18, 2019

firebase.json was in root folder. Configuration was standard. Smth like this:

  "functions": {
    "predeploy": [
      "yarn --cwd \"$RESOURCE_DIR\" run lint",
      "yarn --cwd \"$RESOURCE_DIR\" run build"
    ],
    "source": "functions",
    "ignore": []
  },
  <...>

everything was deployed as expected (including node_modules) except node_modules/utilities which is symlink.


I manage to workaround this issue by writing few scripts which:

  • create packages for each workspace (yarn pack). e.g. this creates utilities.tgz.
  • moving all output to some specific dir.
  • modifying package.json to use tgz files for workspace dependencies. e.g. dependencies { "utilities": "1.0.0" -> dependencies { "utilities": "file:./utilities.tgz"
  • deploying that dir to firebase

output dir content before upload:

- dist
|  - lib
|  | -index.js
|  - utilities.tgz
|  - package.json <---------- This is modified to use *.tgz for workspaces

@0x80
Copy link

0x80 commented Jun 28, 2019

@audkar Today I ran into the same issue as you.

I am new to both Lerna and Yarn workspaces. As I understand it you can also just use Lerna. Would that help in any way?

Your workaround seems a bit complicated for me 🤔

Also wondering, what is `--cwd "$RESOURCE_DIR" for?

@audkar
Copy link

audkar commented Jun 28, 2019

--cwd stands for "current working directory" and $RESOURCE_DIR holds value for source dir (functions in this case). Adding this flag will make yarn to be executed in functions dir instead of root

@0x80
Copy link

0x80 commented Jun 28, 2019

@audkar Ah I see. So you could do the same with yarn workspace functions lint and yarn workspace functions build

@0x80
Copy link

0x80 commented Jun 28, 2019

@dinvlad It is unclear to me why you are targeting the dist folder and copying things over there. If you build to dist, but leave the package.json where it is and point main to dist/index.js then things should work the same no? You should then set source to api instead of api/dist.

@0x80
Copy link

0x80 commented Jun 28, 2019

@dinvlad I learned the yarn workspace command from your comments, but can't seem to make it work for some reason. See here. Any idea?

Sorry for going a bit off-topic here. Maybe comment in SO, to minimize the noise.

@dinvlad
Copy link

dinvlad commented Jun 28, 2019

@0x80 I copy package.jsonto api/dist and point firebase.json to api/dist so only the "built" files are packaged inside the cloud function. I'm not sure what will happen if I point firebase.json to api - perhaps it will still be smart enough to only package what's inside api/dist (based on main attribute in package.json). But I thought it was cleaner to just point to api/dist.

Re yarn workspace, I responded on SO ;)

@0x80
Copy link

0x80 commented Jul 1, 2019

@dinvlad it will bundle the root of what you point it to, but you can put everything that you don't want included in the firebase.json ignore list.

I've now used a similar workaround to @audkar.

{
  "functions": {
    "source": "packages/cloud-functions",
    "predeploy": ["./scripts/pre-deploy-cloud-functions"],
    "ignore": [
      "src",
	  "node_modules"
    ]
  }
}

Then the pre-deploy-cloud-functions script is:

#!/usr/bin/env bash

set -e

yarn workspace @gemini/common lint
yarn workspace @gemini/common build

cd packages/common
yarn pack --filename gemini-common.tgz
mv gemini-common.tgz ../cloud-functions/
cd -

cp yarn.lock packages/cloud-functions/

yarn workspace @gemini/cloud-functions lint
yarn workspace @gemini/cloud-functions build

And packages/cloud-functions has an extra gitignore file:

yarn.lock
*.tgz

@kaminskypavel
Copy link

kaminskypavel commented Aug 1, 2019

here's what worked for me

- root/
|  - .firebaserc
|  - firebase.json
- packages/
  | - package1/
  | - functions/
    | - dist/
    | - src/
    | packages.json

and in the root/firebase.json :

{
	"functions": {
		"predeploy": "npm --prefix \"$RESOURCE_DIR\" run build",
		"source": "packages/functions"
	}
}

@0x80
Copy link

0x80 commented Aug 1, 2019

@kaminskypavel is your packages/functions depending on packages/package1 (or some other sibling package)?

@kaminskypavel
Copy link

@0x80 positive.

@0x80
Copy link

0x80 commented Aug 15, 2019

I think there was something fundamental I misunderstood about monorepos. I assumed you can share a package and deploy an app using that package without actually publishing the shared package to NPM.

It seems that this is not possible, because deployments like Firebase or Now.sh will usually upload the code and then in the cloud do an install and build. Am I correct?

@kaminskypavel I tried your approach and it works, but only after publishing my package to NPM first. Because in my case the package is private I initially got a "not found" error, so I also had to add my .npmrc file to the root of the cloud functions package as described here

@audkar Are you publishing your common package to NPM, or are you like me trying to deploy with shared code which is not published?

@StephenHaney
Copy link

StephenHaney commented Aug 30, 2019

@0x80 I'm with you on this understanding - I think Firebase Function deployments are just (erroneously) assuming that all packages named in package.json will be available on npm, in the name of speeding up deployments.

As yarn workspace setups are becoming more popular, I imagine more folks are going to be surprised that they can't use symlinked packages in Firebase Functions – especially since they work fine until you deploy.

@angelcervera
Copy link

@Stradivario You've convinced me. I will try (but not today).

Thanks!

@0x80
Copy link

0x80 commented Aug 11, 2023

@angelcervera Turbo is used as an example, but there is no link. The isolate command is executed so you can deploy the result with firebase-tools (or any other deployment environment that requires an isolated package). It depends on neither Turbo nor Firebase.

@angelcervera
Copy link

@0x80 Maybe the problem is that I used the example as reference. I will find time to try it again. I will do the same with the @Stradivario firebasetool replacement.

@Stf-F
Copy link

Stf-F commented Aug 16, 2023

Adding more coal to the stove here after trying a few things. I came back from using the private registry solution suggested in the docs as this would have led me to update all the other build/deployment pipelines of my repo that look after the packages that depend on the same shared dependency as my functions, as all of them would have required authentication to install the deps.
In some cases (containerised apps), this would have increased the complexity significantly, and overall, it simply did not make sense to make a rule for the exception.
Inspired by the numerous really good examples available in this post, I went down the custom script route, as my preference is generally to avoid new packages / dependencies when I can.
The script below sits at the root of my project, as using workspace references from within the functions folder did not work.

/
- deploy.js
- firebase.json
- apps/
  - functions/
- packages/
  - sharedstuff/
const fs = require("fs");
const path = require("path");
const { exec } = require("child_process");

const argv = process.argv;
const deploymentStage = process.argv[2];

if (!deploymentStage) {
  console.error("Error: Deployment stage not specified.");
  process.exit(1);
}

let resourceDir;

// /!\ you won't need this 👇 Update the resourceDir path accordingly /!\

if (argv.length === 4 && argv[3] === "api") {
  resourceDir = path.join(process.cwd(), "apps", "api");
} else if (argv.length === 4 && argv[3] === "functions") {
  resourceDir = path.join(process.cwd(), "apps", "functions");
} else {
  console.error("Error: Invalid target specified.");
  process.exit(1);
}

if (deploymentStage === "pre-deploy") {
  console.log("Packaging internal dependencies...");

  const tempFolderPath = path.join(resourceDir, "temp");
  console.log("tempFolderPath", tempFolderPath);
  if (fs.existsSync(tempFolderPath)) {
    console.log("existsSync");
    fs.rmSync(tempFolderPath, { recursive: true });
  }
  fs.mkdirSync(tempFolderPath);

  const packageJson = require(path.join(resourceDir, "package.json"));
  const dependencies = packageJson.dependencies || {};
  const devDependencies = packageJson.devDependencies || {};
  const internalDependencies = packageJson.internalDependencies || {};

  Object.keys(internalDependencies).forEach((packageName) => {
    const tarBallPackageName = `${packageName
      .replace("@", "")
      .replace(/\//g, "-")}-${internalDependencies[packageName]}.tgz`;

    // Build and pack the dependency
    exec(`npm pack -w=${packageName} --pack-destination ${tempFolderPath}`);

    // Update the package.json
    if (dependencies[packageName]) {
      const packageJsonPath = path.join(resourceDir, "package.json");
      const packageJson = JSON.parse(fs.readFileSync(packageJsonPath, "utf8"));
      packageJson.dependencies[
        packageName
      ] = `file:./temp/${tarBallPackageName}`;
      fs.writeFileSync(packageJsonPath, JSON.stringify(packageJson, null, 2));
    } else if (devDependencies[packageName]) {
      const packageJsonPath = path.join(resourceDir, "package.json");
      const packageJson = JSON.parse(fs.readFileSync(packageJsonPath, "utf8"));

      packageJson.devDependencies[
        packageName
      ] = `file:./temp/${tarBallPackageName}`;

      fs.writeFileSync(packageJsonPath, JSON.stringify(packageJson, null, 2));
    }
  });

  console.log("Finished packaging internal packages!");
} else if (deploymentStage === "post-deploy") {
  console.log("Cleaning up internal package code...");
  const tempFolderPath = path.join(resourceDir, "temp");
  if (fs.existsSync(tempFolderPath)) {
    fs.rmSync(tempFolderPath, { recursive: true });
  }

  const packageJson = require(path.join(resourceDir, "package.json"));
  const dependencies = packageJson.dependencies || {};
  const devDependencies = packageJson.devDependencies || {};
  const internalDependencies = packageJson.internalDependencies || {};

  Object.keys(internalDependencies).forEach((packageName) => {
    const version = internalDependencies[packageName];
    if (dependencies[packageName]) {
      const packageJsonPath = path.join(resourceDir, "package.json");
      const packageJson = JSON.parse(fs.readFileSync(packageJsonPath, "utf8"));

      packageJson.dependencies[packageName] = version;

      fs.writeFileSync(packageJsonPath, JSON.stringify(packageJson, null, 2));
    } else if (devDependencies[packageName]) {
      const packageJsonPath = path.join(resourceDir, "package.json");
      const packageJson = JSON.parse(fs.readFileSync(packageJsonPath, "utf8"));

      packageJson.devDependencies[packageName] = version;

      fs.writeFileSync(packageJsonPath, JSON.stringify(packageJson, null, 2));
    }
  });

  console.log("Finished cleaning up internal package code!");
} else {
  console.error(
    "Not a valid parameter. Must be one of: 'pre-deploy'|'post-deploy'."
  );
  process.exit(1);
}

The script is triggered via the pre/postdeploy hooks like so:

 "predeploy": [
        "node \"$PROJECT_DIR/deploy.js\" pre-deploy functions",
        "npm --prefix \"$RESOURCE_DIR\" run build:deploy"
      ],

And expects that shared packages should be referenced in an internalDependencies key in the package.json.

The path to trigger the script will need to be adjusted on a case by case basis depending on where your firebase.json lives.

@gabyshev
Copy link

Do I get it right that there are no plans to support monorepos in deployment for firebase functions?

@bmcbarron
Copy link

@gabyshev It works fine for me when I use isolate-package. The minimum changes I needed to make were:

  1. Add isolate to functions.predeploy in firebase.json in the directory <root>/packages/client1/functions.
  2. Create an isolate.config.json config file in the same directory.
{
  "workspaceRoot": "../../.."
}

... and then things worked as described earlier in this thread and in isolate-package.

My motivation for posting is to give folks confidence that I, as a random independent third party, found that this solved my problem, was straightforward to understand (which is important to me), and required only a small investment of time to integrate.

Should firebase-tools incorporate this directly? That would be nice, but I wouldn't say that monorepos aren't supported, especially since there is no standard for them (yet).

@gabyshev
Copy link

@bmcbarron wow, sounds great. I'll give it a try. Thanks!

@kollein
Copy link

kollein commented Aug 20, 2023

@bmcbarron nice, i have same issue. As mentioned, it would be great to make a standard.

@kollein
Copy link

kollein commented Aug 20, 2023

Thanks bro! @0x80 the isolate-package works like a charm for me.

@BartSpangenberg
Copy link

BartSpangenberg commented Sep 22, 2023

Solution: Similar to this solutions: #653 (comment) from @johanneskares

I believe it is quite elegant and it works fine. Just create a postdeploy.js and predeploy.js file and modify firebase.json

Npm workspaces are used in the project (but will work with any workspace).


File structure

/root
--/functions/package.json
--/functions/predeploy+postdeploy.js
--src/packages/packageNames
--firebase.json

file: /firebase.json

  "functions": {
    "predeploy": ["npm --prefix \"$RESOURCE_DIR\" run build", "node \"$RESOURCE_DIR/predeploy.js\""],
    "postdeploy": ["node \"$RESOURCE_DIR/postdeploy.js\""],
    "source": "functions"
  },


file: /functions/predeploy.js

const { execSync } = require('child_process');
const path = require('path');
const fs = require('fs');

// The tarball files are the building blocks of npm packages
const createTarballs = (packagePath) => {
    // Navigate to the folder in which the package is located
    process.chdir(path.join(__dirname, packagePath));

    // Compile the package into JS and then pack it
    // This iwill create a .tgz (tarball) file inside the package folder
    execSync('npm run build && npm pack', { stdio: 'inherit' });

    // Move the .tgz file to the funcitons folder
    const tgzFile = fs.readdirSync('./').find(file => file.endsWith('.tgz'));
    if (tgzFile) {
      fs.renameSync(tgzFile, path.join(__dirname, '/', tgzFile));
    }
};

  let packagePaths = ["../src/packages/entities", "../src/packages/utility"]

  packagePaths.forEach(p => createTarballs(p));

  // Navigate back to the functions directory to install packages
  process.chdir(__dirname);

  // Install the .tgz packages using npm
  const tgzFiles = fs.readdirSync('./').filter(file => file.endsWith('.tgz'));
  for (const tgzFile of tgzFiles) {
    execSync(`npm install ./${tgzFile}`, { stdio: 'inherit' });
  }


file: /functions/postdeploy.js

const fs = require('fs');
const path = require('path');

// Define the path to the package.json file in the functions folder
const packageJsonPath = path.join(__dirname, 'package.json');

// Define the names of the packages you want to remove
const packagesToRemove = ['@org/packagename', '@org/packagename2'];

  // Read and parse the package.json file
  const packageJsonData = JSON.parse(fs.readFileSync(packageJsonPath, 'utf-8'));

  // Remove the specified packages from dependencies
  packagesToRemove.forEach((pkg) => {
    if (packageJsonData.dependencies && packageJsonData.dependencies[pkg]) {
      delete packageJsonData.dependencies[pkg];
    }
  });

  // Write the modified package.json back to disk
  fs.writeFileSync(packageJsonPath, JSON.stringify(packageJsonData, null, 2));

  // Delete the .tgz files from the functions folder
  fs.readdirSync(__dirname).forEach((file) => {
    if (file.endsWith('.tgz')) {
      fs.unlinkSync(path.join(__dirname, file));
    }
  });

@Psycarlo
Copy link

+1. Would be awesome to support turborepo + pnpm

@0x80
Copy link

0x80 commented Nov 28, 2023

I have been busy lately trying to overcome the last major hurdles and I have some exciting news to share!

  • isolate-package now generates a compatible PNPM lockfile for deterministic deployments. The isolate process is now also exported as a function to embed in other code.
  • I have forked firebase-tools to integrate isolate-package as part of the deploy command. As it then only isolates during deploy, it preserves live code updates when running the local emulators.
  • I have worked on a monorepo example that includes multiple Firebase deployments using the isolate process. It also showcases how to use different internal package strategies including linking directly to TS code. This allows for live code updates from your internal packages without running build watch tasks.

I hope to publish an article about it later this week, but the monorepo readme and code should already contain all the information you need.

The main thing left to do is to also generate lockfiles for NPM and Yarn. For NPM there is arborist, and for Yarn I think there is some user-land code... I tried arborist briefly but NPM was giving me issues, and my focus was on PNPM because it is my preferred package manager.

I plan to give arborist another try later this week.

The integration with the firebase-tools could be improved, and I would love for the Firebase team to help me out on this, because it seems that some refactoring is needed to get all the log messages to display 100% correct information. But I think this works fine for now and because I don't expect it to be part of firebase-tools any time soon, I made the fork available on NPM.

Enjoy! 🌈

@gabyshev
Copy link

@BartSpangenberg have you seen this error? I tried to use your setup in monorepo (pnpm + turbo).

✔  functions: apps/api folder uploaded successfully
i  functions: updating Node.js 18 (1st Gen) function api:myFunc(region1)...
Build failed: npm ERR! code EUNSUPPORTEDPROTOCOL
npm ERR! Unsupported URL Type "workspace:": workspace:*

npm ERR! A complete log of this run can be found in:

However, in api/package.json i saw that script changed a reference from workspace:* to file:...tgz

@0x80
Copy link

0x80 commented Nov 28, 2023

🔥 I have now added support for NPM lockfiles in
isolate-package@1.6.0 and firebase-tools-with-isolate@12.9.1-10.

It turned out to be much easier than figuring this out for PNPM. It's also much slower than PNPM though, but that's maybe not surprising.

The latter is released as a pre-release number because I did not want to mess up the version match with the original firebase-tools. When 12.9.2 comes out I will likely release an updated version 12.9.2 of firebase-tools-with-isolate, and so on, to keep it in sync.

It would obviously be nice to also support Yarn lockfiles, but honestly, for me, it's not a priority at all, so I might wait for someone to show me how to do it or make a PR.

Have fun!

@0x80
Copy link

0x80 commented Nov 28, 2023

Sorry, I seem to have messed something up... Please hold off with using NPM lockfiles until further notice 🙏 I think it's bedtime for me.

NPM lockfiles are deploying again with isolate-package@1.6.2 and firebase-tools-with-isolate@12.9.1-11

@0x80
Copy link

0x80 commented Dec 17, 2023

Some updates:

isolate-package

  • NPM lockfiles are now correct (it wasn't before) and fast to generate
  • Lockfiles for Yarn v1 are now supported

firebase-tools-with-isolate has been synced with upstream firebase-tools to match to latest version 13.0.2

The mono-ts boilerplate, showcasing both, now has two alternative branches, one for NPM and one for Yarn classic.

I haven't figured out how to generate lockfiles for modern Yarn versions yet, but as a fallback it could just generate an NPM lockfile if you use a package manager that is not supported. I don't think it matters what package manager is used in your cloud deployment, as long as the isolated lockfile versions match your original lockfile.

@0x80
Copy link

0x80 commented Dec 18, 2023

🔥 More happy news from version 1.9.x

  • Lockfile output is now available for all package managers, incl Yarn v4
  • Modern versions of Yarn fall back to generating an NPM lockfile, which matches the Yarn lockfile versions.
  • With forceNpm you can opt to always output in NPM format. So if you work with PNPM or Yarn for your monorepo setup, but you want to deploy your isolate output to an environment that only supports NPM, now you can do so.
  • Spaces in monorepo paths are now handled correctly
  • There are configuration options to pick or omit scripts from the manifest
  • All outstanding issues have been resolved ✨

The mono-ts boilerplate now contains 4 parallel branches:

  • main (using pnpm)
  • use-npm (using npm)
  • use-yarn-classic (using yarn v1)
  • use-yarn-modern (using yarn v4)

Enjoy!

@Crocsx
Copy link

Crocsx commented Jan 3, 2024

I bumped into this thread after long research because I am unable to deploy a nextJs app using Nx monorepo.

Might be my bad for not searching priorly the tools properly, but supporting a monorepo nowadays should be a must for such a big service that is firebase. I was planning to go full firebase/firestore/firebase-auth but I am stuck at this step that I thought would be the easiest ^^'

@SvenBudak
Copy link

Are there any updates here? My product owner/client is slowly forcing me to host elsewhere... but I don't want to. ^^"

@0x80
Copy link

0x80 commented Feb 12, 2024

@SvenBudak Sounds dramatic :) Are the known workarounds and solutions not sufficient for your client?

The isolate-package approach seems to be solid. Some people have also been using it for other types of deployments like google cloud functions (outside of firebase) and cloud run.

The only caveat is that using NPM is problematic if you want to deploy with pruned lockfiles to multiple platforms in parallel, because the lockfile pruning (based on npmcli Arborist) currently can not execute in parallel To get around it you could deploy in series, use PNPM or Yarn instead of NPM, or omit the lockfile (not recommended)

@0x80
Copy link

0x80 commented Feb 12, 2024

Has anyone tried the Turborepo prune command for Firebase deployments? It seems very similar to what isolate-package is doing.

Also, Nx has an option to generate a pruned lockfile. I remember looking at Nx and somehow concluding that it was not what I wanted, but I can't recall why. So I'm also curious to hear if anyone tried that.

@0x80
Copy link

0x80 commented Mar 31, 2024

Hi all,

Just a small update to tell you I have released isolate-package@1.13.0 and firebase-tools-with-isolate@13.6.2 and this includes support for Rush monorepos. It has only been tested with PNPM but I am fairly confident that NPM and Yarn will work too.

Notable recent changes:

  • Add Rush monorepo support (zero-config, only tested with PNPM)
  • Adopt PNPM overrides from root package manifest
  • Prune PNPM lockfile packages list (this didn't affect installs, but bloated lockfile size)

@AndyOooh
Copy link

AndyOooh commented May 6, 2024

Hi @0x80

I just gave firebase-tools-with-isolate a go and I'm impressed.
It Took me 5-10 mins to switch my Turborepo project from yarn to pnpm (I was planning to anyway), install the package and get it up and running.
So far everything works:

  • Emulators with live changes (using plain tsc --watch in a separate terminal).
  • Emulators with vscode debugger.
  • Deployments with local package (internal) dependency.

Thanks for you efforts on this!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests