Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(argocd): implement manager #9771

Merged
merged 8 commits into from Apr 29, 2021
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 2 additions & 0 deletions lib/manager/api.ts
@@ -1,5 +1,6 @@
import * as ansible from './ansible';
import * as ansibleGalaxy from './ansible-galaxy';
import * as argoCD from './argocd';
import * as azurePipelines from './azure-pipelines';
import * as batect from './batect';
import * as batectWrapper from './batect-wrapper';
Expand Down Expand Up @@ -65,6 +66,7 @@ export default api;

api.set('ansible', ansible);
api.set('ansible-galaxy', ansibleGalaxy);
api.set('argocd', argoCD);
api.set('azure-pipelines', azurePipelines);
api.set('batect', batect);
api.set('batect-wrapper', batectWrapper);
Expand Down
11 changes: 11 additions & 0 deletions lib/manager/argocd/__fixtures__/malformedApplications.yml
@@ -0,0 +1,11 @@
---
# malformed application as the source section is missing
apiVersion: argoproj.io/v1alpha1
kind: Application
spec:
target:
namespace: testing
---
# malformed application as the source section is missing
apiVersion: argoproj.io/v1alpha1
kind: Application
21 changes: 21 additions & 0 deletions lib/manager/argocd/__fixtures__/randomManifest.yml
@@ -0,0 +1,21 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
labels:
app: nginx
spec:
replicas: 3
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.14.2
ports:
- containerPort: 80
36 changes: 36 additions & 0 deletions lib/manager/argocd/__fixtures__/validApplication.yml
@@ -0,0 +1,36 @@
---
apiVersion: argoproj.io/v1alpha1
kind: Application
spec:
source:
chart: kube-state-metrics
repoURL: https://prometheus-community.github.io/helm-charts
targetRevision: 2.4.1
---
apiVersion: argoproj.io/v1alpha1
kind: Application
spec:
source:
chart: traefik
helm:
values: |
traefik:
service:
spec:
loadBalancerIP: 1.2.3.4
repoURL: gs://helm-charts-internal
targetRevision: 0.0.2
---
apiVersion: argoproj.io/v1alpha1
kind: Application
spec:
source:
repoURL: https://git.example.com/foo/bar.git
targetRevision: v1.2.0
---
# malformed application as the source section is missing
apiVersion: argoproj.io/v1alpha1
kind: Application
spec:
target:
namespace: testing
27 changes: 27 additions & 0 deletions lib/manager/argocd/__snapshots__/extract.spec.ts.snap
@@ -0,0 +1,27 @@
// Jest Snapshot v1, https://goo.gl/fbAQLP

exports[`manager/argocd/extract extractPackageFile() full test 1`] = `
Array [
Object {
"currentValue": "2.4.1",
"datasource": "helm",
"depName": "kube-state-metrics",
"registryUrls": Array [
"https://prometheus-community.github.io/helm-charts",
],
},
Object {
"currentValue": "0.0.2",
"datasource": "helm",
"depName": "traefik",
"registryUrls": Array [
"gs://helm-charts-internal",
],
},
Object {
"currentValue": "v1.2.0",
"datasource": "git-tags",
"depName": "https://git.example.com/foo/bar.git",
},
]
`;
34 changes: 34 additions & 0 deletions lib/manager/argocd/extract.spec.ts
@@ -0,0 +1,34 @@
import { getName, loadFixture } from '../../../test/util';
import { extractPackageFile } from './extract';

const validApplication = loadFixture('validApplication.yml');
const malformedApplication = loadFixture('malformedApplications.yml');
const randomManifest = loadFixture('randomManifest.yml');

describe(getName(), () => {
describe('extractPackageFile()', () => {
it('returns null for empty', () => {
expect(extractPackageFile('nothing here', 'applications.yml')).toBeNull();
});

it('return null for kubernetes manifest', () => {
const result = extractPackageFile(randomManifest, 'applications.yml');
expect(result).toBeNull();
});

it('return null if deps array would be empty', () => {
const result = extractPackageFile(
malformedApplication,
'applications.yml'
);
expect(result).toBeNull();
});

it('full test', () => {
const result = extractPackageFile(validApplication, 'applications.yml');
expect(result).not.toBeNull();
expect(result.deps).toBeArrayOfSize(3);
expect(result.deps).toMatchSnapshot();
});
});
});
50 changes: 50 additions & 0 deletions lib/manager/argocd/extract.ts
@@ -0,0 +1,50 @@
import { safeLoadAll } from 'js-yaml';
import * as gitTags from '../../datasource/git-tags';
import * as helm from '../../datasource/helm';
import type { ExtractConfig, PackageDependency, PackageFile } from '../types';
import type { ApplicationDefinition } from './types';
import { fileTestRegex } from './util';

function createDependency(
definition: ApplicationDefinition
): PackageDependency {
const source = definition.spec?.source;

if (source == null) {
return null;
}

// a chart variable is defined this is helm declaration
if (source.chart) {
return {
depName: source.chart,
registryUrls: [source.repoURL],
currentValue: source.targetRevision,
datasource: helm.id,
};
}
return {
depName: source.repoURL,
currentValue: source.targetRevision,
datasource: gitTags.id,
};
}

export function extractPackageFile(
content: string,
fileName: string,
config?: ExtractConfig
): PackageFile | null {
// check for argo reference. API version for the kind attribute is used
if (fileTestRegex.test(content) === false) {
secustor marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, so we can save parsing phase if we know this is no argocd document. 👍

@rarkins I think we can enable it by default for all yaml files as this regex will ignore files which do not have argocd content.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My main concern is that repos may have hundreds of YAML files while only a very small fraction would be argo or even use argo.

Is there any naming convention which would cover maybe 80%+ of the files e.g. /argo.*\.yaml$/i ?

If so then I'd prefer that as the default. But if not, it could be ok to default to all .yaml

Copy link
Collaborator Author

@secustor secustor Apr 29, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, there aren't such conventions.
Basically this is updating Kubernetes manifests which contain ArgoCD CR manifests. Because of that fact every .yaml could contain an ArgoCD manifest.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, let's see how it goes then. I think we may read detected files into memory in parallel, so it could increase memory use by a lot if there are large YAML files.

return null;
}

const definitions: ApplicationDefinition[] = safeLoadAll(content);

const deps = definitions
.map((definition) => createDependency(definition))
.filter(Boolean);

return deps.length ? { deps } : null;
}
5 changes: 5 additions & 0 deletions lib/manager/argocd/index.ts
@@ -0,0 +1,5 @@
export { extractPackageFile } from './extract';

export const defaultConfig = {
fileMatch: [],
};
33 changes: 33 additions & 0 deletions lib/manager/argocd/readme.md
@@ -0,0 +1,33 @@
The `argocd` manager has no `fileMatch` default patterns, so it won't match any files until you configure it with a pattern. This is because there is no commonly accepted file/directory naming convention for argocd YAML files and we don't want to check every single `*.yaml` file in repositories just in case any of them contain ArgoCD definitions.

If most `.yaml` files in your repository are argocd ones, then you could add this to your config:

```json
{
"argocd": {
"fileMatch": ["\\.yaml$"]
}
}
```

If instead you have them all inside a `argocd/` directory, you would add this:

```json
{
"argocd": {
"fileMatch": ["argocd/.+\\.yaml$"]
}
}
```

Or if it's just a single file then something like this:

```json
{
"argocd": {
"fileMatch": ["^config/applications\\.yaml$"]
}
}
```

If you need to change the versioning format, read the [versioning](https://docs.renovatebot.com/modules/versioning/) documentation to learn more.
9 changes: 9 additions & 0 deletions lib/manager/argocd/types.ts
@@ -0,0 +1,9 @@
export interface ApplicationDefinition {
spec: {
source: {
chart?: string;
repoURL: string;
targetRevision: string;
};
};
}
3 changes: 3 additions & 0 deletions lib/manager/argocd/util.ts
@@ -0,0 +1,3 @@
export const keyValueExtractionRegex = /^\s*(?<key>[^\s]+):\s+"?(?<value>[^"\s]+)"?\s*$/;
// looks for `apiVersion: argoproj.io/
export const fileTestRegex = /\s*apiVersion:\s*argoproj.io\/\s*/;