Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: download fewer metadata from npm registry #436

Merged
merged 1 commit into from Mar 26, 2024
Merged

fix: download fewer metadata from npm registry #436

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
20 changes: 6 additions & 14 deletions sources/npmRegistryUtils.ts
View file
Open in desktop
Expand Up @@ -9,7 +9,7 @@ export const DEFAULT_HEADERS: Record<string, string> = {
};
export const DEFAULT_NPM_REGISTRY_URL = `https://registry.npmjs.org`;

export async function fetchAsJson(packageName: string) {
export async function fetchAsJson(packageName: string, version?: string) {
const npmRegistryUrl = process.env.COREPACK_NPM_REGISTRY || DEFAULT_NPM_REGISTRY_URL;

if (process.env.COREPACK_ENABLE_NETWORK === `0`)
Expand All @@ -25,18 +25,14 @@ export async function fetchAsJson(packageName: string) {
headers.authorization = `Basic ${encodedCreds}`;
}

return httpUtils.fetchAsJson(`${npmRegistryUrl}/${packageName}`, {headers});
return httpUtils.fetchAsJson(`${npmRegistryUrl}/${packageName}${version ? `/${version}` : ``}`, {headers});
Copy link

@PayBas PayBas May 3, 2024 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aduh95 @arcanis
This completely breaks COREPACK_NPM_REGISTRY in combination with Sonatype Nexus repository manager.

ARG YARN_VERSION
ARG NPM_REGISTRY_URL="https://nexus.megacorp.com/repository/npmjs-proxy/"
ENV COREPACK_NPM_REGISTRY $NPM_REGISTRY_URL

RUN  npm config set registry $NPM_REGISTRY_URL \
  && npm install --global corepack@latest \
  && corepack enable \
  && corepack install --global yarn@${YARN_VERSION} \
  && yarn config set --home npmRegistryServer $NPM_REGISTRY_URL

Results in:

Installing yarn@4.2.1...
Internal Error: Server answered with HTTP 400 when performing the request to https://nexus.megacorp.com/repository/npmjs-proxy//@yarnpkg/cli-dist/4.2.1; for troubleshooting help, see https://github.com/nodejs/corepack#troubleshooting
    at fetch (/home/jenkins/.npm-global/lib/node_modules/corepack/dist/lib/corepack.cjs:22769:11)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async fetchAsJson (/home/jenkins/.npm-global/lib/node_modules/corepack/dist/lib/corepack.cjs:22776:20)
    at async fetchTarballURLAndSignature (/home/jenkins/.npm-global/lib/node_modules/corepack/dist/lib/corepack.cjs:22724:27)
    at async installVersion (/home/jenkins/.npm-global/lib/node_modules/corepack/dist/lib/corepack.cjs:22987:52)
    at async Engine.ensurePackageManager (/home/jenkins/.npm-global/lib/node_modules/corepack/dist/lib/corepack.cjs:23449:32)
    at async InstallGlobalCommand.installFromDescriptor (/home/jenkins/.npm-global/lib/node_modules/corepack/dist/lib/corepack.cjs:23846:5)
    at async InstallGlobalCommand.execute (/home/jenkins/.npm-global/lib/node_modules/corepack/dist/lib/corepack.cjs:23828:9)
    at async InstallGlobalCommand.validateAndExecute (/home/jenkins/.npm-global/lib/node_modules/corepack/dist/lib/corepack.cjs:20954:22)
    at async _Cli.run (/home/jenkins/.npm-global/lib/node_modules/corepack/dist/lib/corepack.cjs:21929:18)

Nexus doesn't provide metadata at the ${npmRegistryUrl}/${packageName}/${version} url.
I believe it only serves metadata at the ${npmRegistryUrl}/${packageName} url.

So this change breaks corepack for Nexus and perhaps Artifactory as well.

Had to revert to corepack 0.26.0

Update

I've found a public Nexus instance to show what I mean:
Web view: https://nexus3.onap.org/#browse/browse:npm:%40yarnpkg%2Fcli-dist
Artifact: https://nexus3.onap.org/repository/npm/%40yarnpkg/cli-dist/-/cli-dist-4.2.1.tgz
Metadata: https://nexus3.onap.org/repository/npm/%40yarnpkg/cli-dist

There is no metadata available at https://nexus3.onap.org/repository/npm/%40yarnpkg/cli-dist/4.2.1 !

}

export async function fetchLatestStableVersion(packageName: string) {
const metadata = await fetchAsJson(packageName);

const {latest} = metadata[`dist-tags`];
if (latest === undefined)
throw new Error(`${packageName} does not have a "latest" tag.`);
const metadata = await fetchAsJson(packageName, `latest`);

const {shasum} = metadata.versions[latest].dist;
return `${latest}+sha1.${shasum}`;
const {shasum} = metadata.dist;
return `${metadata.version}+sha1.${shasum}`;
}

export async function fetchAvailableTags(packageName: string) {
Expand All @@ -50,11 +46,7 @@ export async function fetchAvailableVersions(packageName: string) {
}

export async function fetchTarballUrl(packageName: string, version: string) {
const metadata = await fetchAsJson(packageName);
const versionMetadata = metadata.versions?.[version];
if (versionMetadata === undefined)
throw new Error(`${packageName}@${version} does not exist.`);

const versionMetadata = await fetchAsJson(packageName, version);
const {tarball} = versionMetadata.dist;
if (tarball === undefined || !tarball.startsWith(`http`))
throw new Error(`${packageName}@${version} does not have a valid tarball.`);
Expand Down
Binary file modified tests/nocks.db
View file
Open in desktop
Binary file not shown.