Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: initial implementation of bazel-module manager #21893

Merged
merged 95 commits into from May 20, 2023
Merged
Show file tree
Hide file tree
Changes from 92 commits
Commits
Show all changes
95 commits
Select commit Hold shift + click to select a range
5c9822a
Initial manager files
cgrindel Apr 20, 2023
11ef4b1
Add placeholder spec file
cgrindel Apr 21, 2023
a8d8374
Save work
cgrindel Apr 21, 2023
b1b7c2e
Use new datasource name
cgrindel Apr 25, 2023
bfde79f
Have current return undefined instead of null
cgrindel Apr 25, 2023
d521f72
Initial parse with type coercion.
cgrindel Apr 26, 2023
3068800
Refactor to use processStack
cgrindel Apr 26, 2023
e76e671
Add complex Ctx test
cgrindel Apr 26, 2023
d5a59e2
Add lifecycle tests for fragments
cgrindel Apr 26, 2023
a4d3e85
Add tests for Fragments functions.
cgrindel Apr 27, 2023
f8c6cf9
Test name change
cgrindel Apr 27, 2023
b1cd662
Rename types to stack.
cgrindel Apr 27, 2023
cd61684
Add missing test for Stack.
cgrindel Apr 27, 2023
8cf6d5c
Add missing tests for context
cgrindel Apr 27, 2023
a0dd10e
Add isRule to RecordFragment
cgrindel Apr 27, 2023
818203d
Add instanceExists
cgrindel Apr 27, 2023
711b1cd
Add StarlarkBoolean
cgrindel Apr 27, 2023
326f293
Save work. Need to fix extractPackageFile tests
cgrindel Apr 27, 2023
25c3cd2
Save work.
cgrindel Apr 27, 2023
93e30db
Tests are green
cgrindel Apr 27, 2023
0edc936
Do not export BazelDepRecord
cgrindel Apr 27, 2023
d0a91f2
Refactor extract. Add Fragments.safeAsAttribute.
cgrindel Apr 28, 2023
36b6422
Add schemas for fragments
cgrindel Apr 28, 2023
743a626
Change extract to use zod parse/transform to filter output
cgrindel Apr 28, 2023
6c04e9b
Fix typeError bug
cgrindel Apr 28, 2023
49c2a76
Add readme.md
cgrindel Apr 28, 2023
4af1143
Add test for invalid string to new BooleanFragment.
cgrindel Apr 28, 2023
7908cfc
Remove TODO.
cgrindel Apr 29, 2023
024e2a3
Move as functions to their respective class.
cgrindel Apr 29, 2023
f4dcc59
Refactor popStack
cgrindel Apr 29, 2023
5ad6079
Rename Ctx.from to Ctx.as.
cgrindel Apr 29, 2023
70633c9
Disable the manager
cgrindel Apr 30, 2023
fdf91c7
Merge branch 'main' into feat/13658_bazel_module_manager_new1
cgrindel Apr 30, 2023
cf883b4
Update WIP comment with link to renovate tracking issue.
cgrindel May 2, 2023
6b1f68c
Update the readme that the manager is WIP.
cgrindel May 2, 2023
277427f
Merge branch 'main' into feat/13658_bazel_module_manager_new1
cgrindel May 2, 2023
0a27498
Merge branch 'main' into feat/13658_bazel_module_manager_new1
cgrindel May 4, 2023
4f6efd6
Apply formatting to readme.md based upon review feedback.
cgrindel May 4, 2023
d40cb76
Reformat last line in readme.md.
cgrindel May 4, 2023
a0544a9
Merge branch 'main' into feat/13658_bazel_module_manager_new1
cgrindel May 4, 2023
d5dc0cb
Remove it from test descriptions
cgrindel May 4, 2023
9502078
Clarify test description about adding an attribute without a parent.
cgrindel May 4, 2023
5224ff1
Update the readme.md to clarify that it is disabled and that it only
cgrindel May 4, 2023
83be3b9
Merge branch 'main' into feat/13658_bazel_module_manager_new1
cgrindel May 5, 2023
cd5a628
Rename filters
cgrindel May 5, 2023
2630b71
Remove StarlarkBoolean
cgrindel May 5, 2023
e9e1359
Merge branch 'main' into feat/13658_bazel_module_manager_new1
cgrindel May 8, 2023
a777e53
Migrate test to it.each
cgrindel May 8, 2023
958d30f
Add test cases to extract.spec.ts
cgrindel May 8, 2023
8037482
Use is.plainObject instead of exists
cgrindel May 8, 2023
eb77be1
Switch to is.boolean
cgrindel May 8, 2023
9907ff0
Add mock test for extractPackageFile
cgrindel May 8, 2023
d5ee2a5
Return null instead of undefined
cgrindel May 8, 2023
4fda799
Remove extraneous Ctx.as()
cgrindel May 8, 2023
8b5aaae
Switch extract mock test to use spyOn
cgrindel May 8, 2023
a499e87
Change wording for readme.md.
cgrindel May 9, 2023
653af45
Merge branch 'main' into feat/13658_bazel_module_manager_new1
cgrindel May 9, 2023
c152124
Rework fragments to use discriminatedUnion
cgrindel May 9, 2023
a2c879f
-m
cgrindel May 9, 2023
0695214
Test coverage to 100%
cgrindel May 9, 2023
5a46618
Merge branch 'main' into feat/13658_bazel_module_manager_new1
cgrindel May 10, 2023
26b9c5c
Clarify error message
cgrindel May 10, 2023
9376437
Add filters back
cgrindel May 10, 2023
9e7b243
Merge branch 'main' into feat/13658_bazel_module_manager_new1
cgrindel May 10, 2023
2a92466
Update toPackageDependency to resemble later implementation
cgrindel May 10, 2023
12cfe5f
Reimplement Ctx.as.
cgrindel May 10, 2023
712c98f
Update import
cgrindel May 10, 2023
67bffa7
Merge branch 'main' into feat/13658_bazel_module_manager_new1
cgrindel May 11, 2023
2174087
Use isNotNullOrUndefined from lib/util/array
cgrindel May 11, 2023
d97f05c
Rework parse and extract to avoid null value until the end of
cgrindel May 11, 2023
9b3f194
Rename kwParams to kvParams
cgrindel May 11, 2023
0da45f5
Remove try-catch in parser as it appears to be not needed
cgrindel May 11, 2023
7a10ec4
Fix check for undefined in Stack.current()
cgrindel May 11, 2023
c7a5421
Merge branch 'main' into feat/13658_bazel_module_manager_new1
cgrindel May 12, 2023
978f0eb
Remove unused parameter from parse signature
cgrindel May 12, 2023
03b15bd
Fix parser.spec
cgrindel May 12, 2023
39cee4b
Add log message if a package file does not provide any package
cgrindel May 12, 2023
ff894e5
Tests are green after reworking fragments removing classes
cgrindel May 12, 2023
416178b
Refactor context processing to remove istanbul comment
cgrindel May 12, 2023
bf8a435
Remove toPackageDependency
cgrindel May 12, 2023
4aedab1
Clean up
cgrindel May 12, 2023
842411a
Merge branch 'main' into feat/13658_bazel_module_manager_new1
cgrindel May 15, 2023
7a38968
Remove Stack
cgrindel May 15, 2023
d6f85c2
Add try-catch in extractPackageFile
cgrindel May 15, 2023
c4e2d06
Add tests for Ctx.current
cgrindel May 15, 2023
951b14d
Add test for try-catch
cgrindel May 15, 2023
9908d3e
Hide current and safeCurrent
cgrindel May 15, 2023
f37aa4d
Rename packageFile to filename
cgrindel May 15, 2023
36ba437
Change to debug log
cgrindel May 15, 2023
dd814dc
Merge branch 'main' into feat/13658_bazel_module_manager_new1
cgrindel May 15, 2023
2ab5671
Merge branch 'main' into feat/13658_bazel_module_manager_new1
cgrindel May 15, 2023
738f769
Remove Ctx.as
cgrindel May 15, 2023
ddf1348
Merge branch 'main' into feat/13658_bazel_module_manager_new1
cgrindel May 18, 2023
f29f956
Use LooseArray and LooseRecord
cgrindel May 18, 2023
100ccbf
Use arrow functions in parser and use conditional return
cgrindel May 18, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 2 additions & 0 deletions lib/modules/manager/api.ts
Expand Up @@ -6,6 +6,7 @@ import * as azurePipelines from './azure-pipelines';
import * as batect from './batect';
import * as batectWrapper from './batect-wrapper';
import * as bazel from './bazel';
import * as bazelModule from './bazel-module';
import * as bazelisk from './bazelisk';
import * as bicep from './bicep';
import * as bitbucketPipelines from './bitbucket-pipelines';
Expand Down Expand Up @@ -96,6 +97,7 @@ api.set('azure-pipelines', azurePipelines);
api.set('batect', batect);
api.set('batect-wrapper', batectWrapper);
api.set('bazel', bazel);
api.set('bazel-module', bazelModule);
api.set('bazelisk', bazelisk);
api.set('bicep', bicep);
api.set('bitbucket-pipelines', bitbucketPipelines);
Expand Down
23 changes: 23 additions & 0 deletions lib/modules/manager/bazel-module/bazel-dep.spec.ts
@@ -0,0 +1,23 @@
import { BazelDatasource } from '../../datasource/bazel';
import { ToBazelDep } from './bazel-dep';
import * as fragments from './fragments';

describe('modules/manager/bazel-module/bazel-dep', () => {
describe('ToBazelDep', () => {
it('transforms a record fragment', () => {
const record = fragments.record({
rule: fragments.string('bazel_dep'),
name: fragments.string('rules_foo'),
version: fragments.string('1.2.3'),
dev_dependency: fragments.boolean(true),
});
const result = ToBazelDep.parse(record);
expect(result).toEqual({
datasource: BazelDatasource.id,
depType: 'bazel_dep',
depName: 'rules_foo',
currentValue: '1.2.3',
});
});
});
});
28 changes: 28 additions & 0 deletions lib/modules/manager/bazel-module/bazel-dep.ts
@@ -0,0 +1,28 @@
import { z } from 'zod';
import { BazelDatasource } from '../../datasource/bazel';
import type { PackageDependency } from '../types';
import {
BooleanFragmentSchema,
RecordFragmentSchema,
StringFragmentSchema,
} from './fragments';

const BazelDepSchema = RecordFragmentSchema.extend({
children: z.object({
rule: StringFragmentSchema.extend({
value: z.literal('bazel_dep'),
}),
name: StringFragmentSchema,
version: StringFragmentSchema,
dev_dependency: BooleanFragmentSchema.optional(),
}),
});

export const ToBazelDep = BazelDepSchema.transform(
({ children: { rule, name, version } }): PackageDependency => ({
datasource: BazelDatasource.id,
depType: rule.value,
depName: name.value,
currentValue: version.value,
})
);
98 changes: 98 additions & 0 deletions lib/modules/manager/bazel-module/context.spec.ts
@@ -0,0 +1,98 @@
import { Ctx, CtxProcessingError } from './context';
import * as fragments from './fragments';

describe('modules/manager/bazel-module/context', () => {
describe('Ctx', () => {
it('construct simple bazel_dep', () => {
const ctx = new Ctx()
.startRule('bazel_dep')
.startAttribute('name')
.addString('rules_foo')
.startAttribute('version')
.addString('1.2.3')
.endRule();

expect(ctx.results).toEqual([
fragments.record(
{
rule: fragments.string('bazel_dep'),
name: fragments.string('rules_foo'),
version: fragments.string('1.2.3'),
},
true
),
]);
});

it('construct a rule with array arg', () => {
const ctx = new Ctx()
.startRule('foo_library')
.startAttribute('name')
.addString('my_library')
.startAttribute('srcs')
.startArray()
.addString('first')
.addString('second')
.endArray()
.endRule();

expect(ctx.results).toEqual([
fragments.record(
{
rule: fragments.string('foo_library'),
name: fragments.string('my_library'),
srcs: fragments.array(
[fragments.string('first'), fragments.string('second')],
true
),
},
true
),
]);
});

describe('.currentRecord', () => {
it('returns the record fragment if it is current', () => {
const ctx = new Ctx().startRecord();
expect(ctx.currentRecord).toEqual(fragments.record());
});

it('throws if there is no current', () => {
const ctx = new Ctx();
expect(() => ctx.currentRecord).toThrow(
new Error('Requested current, but no value.')
);
});

it('throws if the current is not a record fragment', () => {
const ctx = new Ctx().startArray();
expect(() => ctx.currentRecord).toThrow(
new Error('Requested current record, but does not exist.')
);
});
});

describe('.currentArray', () => {
it('returns the array fragment if it is current', () => {
const ctx = new Ctx().startArray();
expect(ctx.currentArray).toEqual(fragments.array());
});

it('throws if the current is not a record fragment', () => {
const ctx = new Ctx().startRecord();
expect(() => ctx.currentArray).toThrow(
new Error('Requested current array, but does not exist.')
);
});
});

it('throws if add an attribute without a parent', () => {
const ctx = new Ctx().startAttribute('name');
expect(() => ctx.addString('chicken')).toThrow(
new CtxProcessingError(
fragments.attribute('name', fragments.string('chicken'))
)
);
});
});
});
154 changes: 154 additions & 0 deletions lib/modules/manager/bazel-module/context.ts
@@ -0,0 +1,154 @@
import type {
AllFragments,
ArrayFragment,
ChildFragments,
RecordFragment,
} from './fragments';
import * as fragments from './fragments';

// Represents the fields that the context must have.
export interface CtxCompatible {
results: RecordFragment[];
stack: AllFragments[];
}

export class CtxProcessingError extends Error {
readonly current: AllFragments;
readonly parent?: AllFragments;
constructor(current: AllFragments, parent?: AllFragments) {
const msg = `Invalid context state. current: ${current.type}, parent: ${
parent?.type ?? 'none'
}`;
super(msg);
this.name = 'CtxProcessingError';
this.current = current;
this.parent = parent;
}
}

export class Ctx implements CtxCompatible {
results: RecordFragment[];
stack: AllFragments[];

constructor(results: RecordFragment[] = [], stack: AllFragments[] = []) {
viceice marked this conversation as resolved.
Show resolved Hide resolved
this.results = results;
this.stack = stack;
}

private get safeCurrent(): AllFragments | undefined {
return this.stack.at(-1);
}

private get current(): AllFragments {
const c = this.safeCurrent;
if (c === undefined) {
throw new Error('Requested current, but no value.');
}
return c;
}
get currentRecord(): RecordFragment {
const current = this.current;
if (current.type === 'record') {
return current;
}
throw new Error('Requested current record, but does not exist.');
}

get currentArray(): ArrayFragment {
const current = this.current;
if (current.type === 'array') {
return current;
}
throw new Error('Requested current array, but does not exist.');
}

private popStack(): boolean {
const current = this.stack.pop();
if (!current) {
return false;
}
if (!current.isComplete) {
this.stack.push(current);
return false;
}
const parent = this.safeCurrent;

if (parent) {
if (parent.type === 'attribute' && fragments.isValue(current)) {
parent.value = current;
parent.isComplete = true;
return true;
}
if (parent.type === 'array' && fragments.isPrimitive(current)) {
parent.items.push(current);
return true;
}
if (
parent.type === 'record' &&
current.type === 'attribute' &&
current.value !== undefined
) {
parent.children[current.name] = current.value;
return true;
}
} else if (current.type === 'record') {
this.results.push(current);
return true;
}

throw new CtxProcessingError(current, parent);
}

private processStack(): Ctx {
while (this.popStack()) {
// Nothing to do
}
return this;
}

addString(value: string): Ctx {
this.stack.push(fragments.string(value));
return this.processStack();
}

addBoolean(value: string | boolean): Ctx {
this.stack.push(fragments.boolean(value));
return this.processStack();
}

startRecord(children: ChildFragments = {}): Ctx {
const record = fragments.record(children);
this.stack.push(record);
return this;
}

endRecord(): Ctx {
const record = this.currentRecord;
record.isComplete = true;
return this.processStack();
}

startRule(name: string): Ctx {
return this.startRecord({ rule: fragments.string(name) });
}

endRule(): Ctx {
return this.endRecord();
}

startAttribute(name: string): Ctx {
this.stack.push(fragments.attribute(name));
return this.processStack();
}

startArray(): Ctx {
this.stack.push(fragments.array());
return this.processStack();
}

endArray(): Ctx {
const array = this.currentArray;
array.isComplete = true;
return this.processStack();
}
}
58 changes: 58 additions & 0 deletions lib/modules/manager/bazel-module/extract.spec.ts
cgrindel marked this conversation as resolved.
Show resolved Hide resolved
@@ -0,0 +1,58 @@
import { codeBlock } from 'common-tags';
import { BazelDatasource } from '../../datasource/bazel';
import * as parser from './parser';
import { extractPackageFile } from '.';

describe('modules/manager/bazel-module/extract', () => {
describe('extractPackageFile()', () => {
it('returns null if fails to parse', () => {
const result = extractPackageFile('blahhhhh:foo:@what\n', 'MODULE.bazel');
expect(result).toBeNull();
});

it('returns null if something throws an error', () => {
jest.spyOn(parser, 'parse').mockImplementationOnce((input) => {
throw new Error('Test error');
});
const result = extractPackageFile('content', 'MODULE.bazel');
expect(result).toBeNull();
});

it('returns null if file is empty', () => {
const result = extractPackageFile('', 'MODULE.bazel');
expect(result).toBeNull();
});

it('returns null if file has not recognized declarations', () => {
const input = codeBlock`
ignore_me(name = "rules_foo", version = "1.2.3")
`;
const result = extractPackageFile(input, 'MODULE.bazel');
expect(result).toBeNull();
});

it('returns dependencies', () => {
const input = codeBlock`
bazel_dep(name = "rules_foo", version = "1.2.3")
bazel_dep(name = "rules_bar", version = "1.0.0", dev_dependency = True)
`;
const result = extractPackageFile(input, 'MODULE.bazel');
expect(result).toEqual({
deps: [
{
datasource: BazelDatasource.id,
depType: 'bazel_dep',
depName: 'rules_foo',
currentValue: '1.2.3',
},
{
datasource: BazelDatasource.id,
depType: 'bazel_dep',
depName: 'rules_bar',
currentValue: '1.0.0',
},
],
});
});
});
});