Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(manager/pip-compile): Allow security updates for transitive dependencies #27561

Merged
merged 9 commits into from Mar 30, 2024
23 changes: 23 additions & 0 deletions lib/modules/manager/pip-compile/extract.spec.ts
Expand Up @@ -366,6 +366,29 @@ describe('modules/manager/pip-compile/extract', () => {
);
});

it('adds transitive dependency to deps in package file', async () => {
fs.readLocalFile.mockResolvedValueOnce(
getSimpleRequirementsFile(
'pip-compile --output-file=requirements.txt requirements.in',
['friendly-bard==1.0.1', 'bards-friend==1.0.0'],
),
);
fs.readLocalFile.mockResolvedValueOnce('FrIeNdLy-._.-bArD>=1.0.0');

const lockFiles = ['requirements.txt'];
const packageFiles = await extractAllPackageFiles({}, lockFiles);
expect(packageFiles).toBeDefined();
const packageFile = packageFiles!.pop();
expect(packageFile!.deps).toHaveLength(2);
expect(packageFile!.deps[1]).toEqual({
datasource: 'pypi',
depType: 'indirect',
depName: 'bards-friend',
lockedVersion: '1.0.0',
enabled: false,
});
});

it('handles -r reference to another input file', async () => {
fs.readLocalFile.mockImplementation((name): any => {
if (name === '1.in') {
Expand Down
53 changes: 52 additions & 1 deletion lib/modules/manager/pip-compile/extract.ts
Expand Up @@ -5,7 +5,12 @@ import { ensureLocalPath } from '../../../util/fs/util';
import { normalizeDepName } from '../../datasource/pypi/common';
import { extractPackageFile as extractRequirementsFile } from '../pip_requirements/extract';
import { extractPackageFile as extractSetupPyFile } from '../pip_setup';
import type { ExtractConfig, PackageFile, PackageFileContent } from '../types';
import type {
ExtractConfig,
PackageDependency,
PackageFile,
PackageFileContent,
} from '../types';
import { extractHeaderCommand } from './common';
import type {
DependencyBetweenFiles,
Expand Down Expand Up @@ -135,6 +140,7 @@ export async function extractAllPackageFiles(
);
const existingPackageFile = packageFiles.get(packageFile)!;
existingPackageFile.lockFiles!.push(fileMatch);
extendWithIndirectDeps(existingPackageFile, lockedDeps);
lockFileSources.set(fileMatch, existingPackageFile);
continue;
}
Expand Down Expand Up @@ -183,6 +189,7 @@ export async function extractAllPackageFiles(
);
}
}
extendWithIndirectDeps(packageFileContent, lockedDeps);
const newPackageFile: PackageFile = {
...packageFileContent,
lockFiles: [fileMatch],
Expand Down Expand Up @@ -230,3 +237,47 @@ export async function extractAllPackageFiles(
);
return result;
}

function extendWithIndirectDeps(
packageFileContent: PackageFileContent,
lockedDeps: PackageDependency[],
): void {
for (const lockedDep of lockedDeps) {
if (
!packageFileContent.deps.find(
(dep) =>
normalizeDepName(lockedDep.depName!) ===
normalizeDepName(dep.depName!),
)
) {
packageFileContent.deps.push(indirectDep(lockedDep));
}
}
}

/**
* As indirect dependecies don't exist in the package file, we need to
* create them from the lock file.
*
* By removing currentValue and currentVersion, we ensure that they
* are handled like unconstrained dependencies with locked version.
* Such packages are updated when their update strategy
* is set to 'update-lockfile',
* see: lib/workers/repository/process/lookup/index.ts.
*
* By disabling them by default, we won't create noise by updating them.
* Unless they have vulnerability alert, then they are forced to be updated.
* @param dep dependency extracted from lock file (requirements.txt)
* @returns unconstrained dependency with locked version
*/
function indirectDep(dep: PackageDependency): PackageDependency {
const result = {
...dep,
lockedVersion: dep.currentVersion,
depType: 'indirect',
enabled: false,
};
delete result.currentValue;
delete result.currentVersion;
not7cd marked this conversation as resolved.
Show resolved Hide resolved
return result;
}
6 changes: 6 additions & 0 deletions lib/modules/manager/pip-compile/readme.md
Expand Up @@ -83,3 +83,9 @@ Renovate reads the `requirements.txt` file and extracts these `pip-compile` argu
- `--output-file`

All other allowed `pip-compile` arguments will be passed over without modification.

### Transitive / indirect dependencies

This manager detects dependencies that only appear in lock files.
They are disabled by default but can be forced to enable by vulnerability alerts.
They will be upgraded with `--upgrade-package` option.