pnpm deploy doesn't rewrite workspace:* dependencies in package.json

[rijkvanzanten]

pnpm version:

7.30.0

Code to reproduce the issue:

Run pnpm -F <project> deploy

Expected behavior:

When the target directory is created, I expected the package.json to be "updated" to include the workspace packages with their respective version numbers similar to pnpm pack or pnpm publish.

Actual behavior:

workspace:* versions remain in package.json. This in turn means consumers of the output directory can't rely on npm/yarn or any other package manager. A temporary workaround would be to pnpm pack the target directory, then pull the "clean" package.json from the tarball, and put that back into the target directory of the deploy.

Additional information:

• node -v prints: v18.12.1
• Windows, macOS, or Linux?: any

[pkerschbaum]

Had the same problem, and solved it with a script which replaces the workspace: protocol of workspace dependencies by the respective version number.
Maybe it helps you as a workaround.

// replace-workspace-protocol-by-version.mjs
import fs from 'node:fs';
import path from 'node:path';
import url from 'node:url';
import { Command } from '@commander-js/extra-typings';
import { findWorkspacePackagesNoCheck } from '@pnpm/find-workspace-packages';

const program = new Command().argument(
	'<pathtopackagejson>',
	'the path to the package.json of which the dependencies using the "workspace:*" protocol should be replaced by the versions present in the monorepo (e.g. "./deployed-package/package.json")'
);

program.parse();

const __dirname = url.fileURLToPath(new URL('.', import.meta.url));

const projects = await findWorkspacePackagesNoCheck(path.join(__dirname, '..', '..', '..'));
// first elem of returned array is the root package.json, the monorepo itself --> discard that
const [_rootProject, ...workspaceProjects] = projects;

const absolutePath = path.join(process.cwd(), program.args[0]);

const packageJsonContent = JSON.parse(await fs.promises.readFile(absolutePath, 'utf-8'));

/**
 * @param {{[key: string]: string}} dependencies
 */
function replaceWorkspaceProtocolBySpecificVersion(dependencies) {
	if (typeof dependencies !== 'object' || dependencies === null) {
		return;
	}

	for (const [pkgName, pkgVersion] of Object.entries(dependencies)) {
		if (pkgVersion.startsWith('workspace:')) {
			const matchingWorkspaceProject = workspaceProjects.find(
				project => project.manifest.name === pkgName
			);
			if (!matchingWorkspaceProject) {
				throw new Error(
					`could not replace the workspace protocol of a package, reason: package not found! pkgName=${pkgName}`
				);
			}
			if (!matchingWorkspaceProject.manifest.version) {
				throw new Error(
					`could not replace the workspace protocol of a package, reason: package has no version! pkgName=${pkgName}`
				);
			}

			dependencies[pkgName] = `^${matchingWorkspaceProject.manifest.version}`;
		}
	}
}

replaceWorkspaceProtocolBySpecificVersion(packageJsonContent.dependencies);
replaceWorkspaceProtocolBySpecificVersion(packageJsonContent.devDependencies);
replaceWorkspaceProtocolBySpecificVersion(packageJsonContent.peerDependencies);
replaceWorkspaceProtocolBySpecificVersion(packageJsonContent.optionalDependencies);

await fs.promises.writeFile(
	absolutePath,
	JSON.stringify(packageJsonContent, undefined, 2),
	'utf-8'
);

# run it against the package.json of the deployed package
node ./replace-workspace-protocol-by-version.mjs ./deployed-package/package.json

[rijkvanzanten]

Thanks @pkerschbaum! We ended up working around this by relying on the output of pack and pulling the "correct" package.json from that (in a Dockerfile):

RUN pnpm --recursive run build \
	&& pnpm --filter <package> deploy --prod pruned \
	&& cd pruned \
	&& pnpm pack \
	&& tar -zxvf *.tgz package/package.json \
	&& rm package.json \
	&& mv package/package.json package.json \
	&& rm *.tgz

[alvis]

It appears that it's a careless mistake which still remains in version 8.0.0-rc1. Nevertheless, it is not a significant issue since the code is capable of running without the need for package.json.

[rijkvanzanten]

Nevertheless, it is not a significant issue since the code is capable of running without the need for package.json.

This really depends on your actual use case. In my case, it's a major blocker that requires the workaround 🤷🏻

[alvis]

This really depends on your actual use case. In my case, it's a major blocker that requires the workaround 🤷🏻

@rijkvanzanten Just curious. If you need to read the version of your dependencies at runtime, you probably want to get it like

import libPackageJson from 'lib_name/package.json';
const libVersion = libPackageJson.version;

Alternatively you can get the versions from the lock file.

[rijkvanzanten]

@alvis It's not about reading the package version (also you can't import json like that in ESM) 👍🏻 It's about making sure that the output package.json format is a "valid" package.json regardless of package manager used. As of right now, pnpm pack creates a "npm valid" package.json, whereas pnpm deploy doesn't

[alvis]

@rijkvanzanten Right, I guess I got you. Like using read-pkg-up which would try to normalize the content of package.json and as it's not a valid one it throw an error?

[rijkvanzanten]

I'm distributing an app through a Docker image that relies on pnpm deploy to generate the final "dist" that's shipped. To keep the image lightweight, it's "just" node18-alpine and our dist folder, that in turn also means that the user may or may not have to interact with the packages through npm instead of pnpm. Even though it's recommended to enable pnpm with corepack in that case, that's not always followed or feasible. When trying to use any npm command against the package (like npx to run the bin), it'll fail as the package.json file is "invalid" in that context 👍🏻

[alvis]

@rijkvanzanten I think your use case example makes a perfect sense to get it sorted asap.

[JacobLey]

I believe this is addressed by #6943

Which is not released yet, but hopefully will be soon

[rijkvanzanten]

Perfect! Thanks Jacob 🙂

[dgellow]

Hmm, unfortunately #6943 has been reverted by #7058. Is there another solution? I'm also facing this problem in the context of docker images.

[kael-shipman]

I'm experiencing something similar which indicates the broader issue is actually that pnpm deploy doesn't apply pnpm's special publishing process at all when it exports the workspace deps. In my case, it's also not applying any of the items listed in publishConfig, which results in dependencies that won't import anymore because "main": "src/index.ts" is not correctly replaced with "main": "dist/index.js". And since this affects imports in the deployed app, it does require a work-around.

See this tag for a reproducible example. (This is a POC monorepo, so should be fairly minimal.)

In my case, I have several workspace libraries that need to be "published" into the pruned deploy. None of these libraries are correctly processed and so none of them import when the program is executed. @rijkvanzanten's solution is a good start, but unfortunately it doesn't work out of the box for the libraries that are copied into the pruned folder because you can't run pnpm pack successfully from inside a subfolder of node_modules (because it doesn't resolve its own dependencies correctly).

Since my use-case is also docker, I have a little more leeway to do messy or destructive things, so my work-around is somewhat brutish:

RUN \
  pnpm --filter my-app build && \
  pnpm --filter my-app --prod deploy /pruned/my-app && \
  sed -ri 's#"main": "src/(.+)\.ts"#"main": "dist/\1.js"#' /pruned/my-app/node_modules/@my-namespace/*/package.json

[flevi29]

It might be somewhat related, in my case since deploy doesn't create a flat node_modules, the docker image cannot find packages that would be in the workspace root package.json dependencies. But strangely this only happens in docker, on local deploy seems to work fine, it's probably more nuanced.

Point is all of this added complexity and headache could be entirely avoided if pnpm deploy created something npm compatible. That would be really nice.