Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bpf: port verifier test to Go #24538

Merged
merged 3 commits into from
Apr 3, 2023
Merged

bpf: port verifier test to Go #24538

merged 3 commits into from
Apr 3, 2023

Conversation

ti-mo
Copy link
Contributor

@ti-mo ti-mo commented Mar 23, 2023

Supersedes #24526.

Description taken from the main commit:

This commit switches package test/verifier over from running verifier-test.sh
to loading programs using Go natively.

A few modifications are made to the CollectionSpec before loading:
- Remove map pinning flags so each ELF gets its own copies of maps that would
  normally be shared across multiple ELFs by pinning them.
- Run some probes over ProgramType/AttachType combinations to make sure we
  don't try to load cgroup programs on kernels that don't support specific
  attach types.

verifier_test.go now tests bpf_sock as well.

Also converted each line in bpf/complexity-tests/*/*.txt to run its own
subtest so it's easier to spot which configuration in the list caused an
error, and specific subtests can be run using go test -run=...

Closes #24522

Port verifier tests to Go

@maintainer-s-little-helper maintainer-s-little-helper bot added the dont-merge/needs-release-note-label The author needs to describe the release impact of these changes. label Mar 23, 2023
@ti-mo ti-mo added sig/loader Impacts the loading of BPF programs into the kernel. area/CI Continuous Integration testing issue or flake sig/datapath Impacts bpf/ or low-level forwarding details, including map management and monitor messages. release-note/ci This PR makes changes to the CI. labels Mar 23, 2023
@maintainer-s-little-helper maintainer-s-little-helper bot removed dont-merge/needs-release-note-label The author needs to describe the release impact of these changes. labels Mar 23, 2023
@ti-mo ti-mo force-pushed the tb/go-verifier-test branch 9 times, most recently from e4cd070 to 6decf63 Compare March 24, 2023 09:14
@ti-mo ti-mo mentioned this pull request Mar 27, 2023
@ti-mo ti-mo force-pushed the tb/go-verifier-test branch 3 times, most recently from 2606a7f to 51e8017 Compare March 27, 2023 13:50
@ti-mo
Copy link
Contributor Author

ti-mo commented Mar 27, 2023

/test

@ti-mo ti-mo force-pushed the tb/go-verifier-test branch 2 times, most recently from 24dfbb0 to f972891 Compare March 30, 2023 14:22
@ti-mo ti-mo added the needs-backport/1.13 This PR / issue needs backporting to the v1.13 branch label Mar 30, 2023
@maintainer-s-little-helper maintainer-s-little-helper bot added this to Needs backport from master in 1.13.2 Mar 30, 2023
@ti-mo ti-mo marked this pull request as ready for review March 30, 2023 14:23
@ti-mo ti-mo requested a review from a team as a code owner March 30, 2023 14:23
@ti-mo ti-mo removed the dont-merge/blocked Another PR must be merged before this one. label Mar 31, 2023
@ti-mo
Copy link
Contributor Author

ti-mo commented Mar 31, 2023

@aanm Thanks! Removed the commit and the label. New workflows ran successfully: https://github.com/cilium/cilium/actions/runs/4565504149?pr=24538.

@ti-mo
Copy link
Contributor Author

ti-mo commented Mar 31, 2023

/test

Job 'Cilium-PR-K8s-1.16-kernel-4.19' failed:

Click to show.

Test Name

K8sDatapathConfig Host firewall With VXLAN

Failure Output

FAIL: Error deleting resource /home/jenkins/workspace/Cilium-PR-K8s-1.16-kernel-4.19/src/github.com/cilium/cilium/test/k8s/manifests/host-policies.yaml: Cannot retrieve "cilium-qwwq4"'s policy revision: cannot get policy revision: ""

If it is a flake and a GitHub issue doesn't already exist to track it, comment /mlh new-flake Cilium-PR-K8s-1.16-kernel-4.19 so I can create one.

As documented in the code, since we quadruple the buffer in the loop,
the next step up from 4MiB is 16MiB, which would overshoot the limit
of <5.2 kernels by one byte.

I did not opt for doubling instead of quadrupling the buffer, since
that means logs over 8MiB would also fail to load on kernels <5.2.

Signed-off-by: Timo Beckers <timo@isovalent.com>
Before this patch, getCIKernelVersion would only be run if -ci-kernel-version
was not specified, but on CI, it always is. This function also outputs the running
kernel version to the test log. Make it so the function is always run, outputting
the kernel version, and handle the flag parsing logic in the function as well.

Signed-off-by: Timo Beckers <timo@isovalent.com>
This commit switches package test/verifier over from running verifier-test.sh
to loading programs using Go natively.

A few modifications are made to the CollectionSpec before loading:
- Remove map pinning flags so each ELF gets its own copies of maps that would
  normally be shared across multiple ELFs by pinning them.
- Run some probes over ProgramType/AttachType combinations to make sure we
  don't try to load cgroup programs on kernels that don't support specific
  attach types.

verifier_test.go now tests bpf_sock as well.

Also converted each line in bpf/complexity-tests/*/*.txt to run its own
subtest so it's easier to spot which configuration in the list caused an
error, and specific subtests can be run using go test -run=...

Signed-off-by: Timo Beckers <timo@isovalent.com>
@ti-mo
Copy link
Contributor Author

ti-mo commented Apr 3, 2023

/test

@ti-mo ti-mo merged commit 450b772 into master Apr 3, 2023
41 of 42 checks passed
@ti-mo ti-mo deleted the tb/go-verifier-test branch April 3, 2023 12:37
@jibi
Copy link
Member

jibi commented Apr 3, 2023

I think this depends on #22754, which was not marked for backporting. Dropping the labels here until we decide what to do

@jibi jibi removed the needs-backport/1.13 This PR / issue needs backporting to the v1.13 branch label Apr 3, 2023
@maintainer-s-little-helper maintainer-s-little-helper bot removed this from Needs backport from master in 1.13.2 Apr 3, 2023
@jibi jibi added the needs-backport/1.13 This PR / issue needs backporting to the v1.13 branch label Apr 3, 2023
@maintainer-s-little-helper maintainer-s-little-helper bot added this to Needs backport from master in 1.13.2 Apr 3, 2023
@jibi
Copy link
Member

jibi commented Apr 3, 2023

Only backporting af5d551 to v1.13

@jibi jibi mentioned this pull request Apr 3, 2023
11 tasks
@jibi jibi added backport-pending/1.13 The backport for Cilium 1.13.x for this PR is in progress. and removed needs-backport/1.13 This PR / issue needs backporting to the v1.13 branch labels Apr 3, 2023
@maintainer-s-little-helper maintainer-s-little-helper bot moved this from Needs backport from master to Backport pending to v1.13 in 1.13.2 Apr 3, 2023
@jibi jibi added backport-done/1.13 The backport for Cilium 1.13.x for this PR is done. and removed backport-pending/1.13 The backport for Cilium 1.13.x for this PR is in progress. labels Apr 7, 2023
@maintainer-s-little-helper maintainer-s-little-helper bot moved this from Backport pending to v1.13 to Backport done to v1.13 in 1.13.2 Apr 7, 2023
tklauser added a commit to cilium/little-vm-helper-images that referenced this pull request Jul 7, 2023
The verifier complexity test no longer uses the patched iproute2 tools
since it was ported to Go, see cilium/cilium#24538

Signed-off-by: Tobias Klauser <tobias@cilium.io>
tklauser added a commit to cilium/little-vm-helper-images that referenced this pull request Jul 7, 2023
The verifier complexity test no longer uses the iproute2 tools since it
was ported to Go, see cilium/cilium#24538

Signed-off-by: Tobias Klauser <tobias@cilium.io>
tklauser added a commit to cilium/little-vm-helper-images that referenced this pull request Jul 7, 2023
The verifier complexity test no longer uses the iproute2 tools since it
was ported to Go, see cilium/cilium#24538

Signed-off-by: Tobias Klauser <tobias@cilium.io>
kkourt pushed a commit to cilium/little-vm-helper-images that referenced this pull request Jul 10, 2023
The verifier complexity test no longer uses the iproute2 tools since it
was ported to Go, see cilium/cilium#24538

Signed-off-by: Tobias Klauser <tobias@cilium.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/CI Continuous Integration testing issue or flake backport-done/1.13 The backport for Cilium 1.13.x for this PR is done. ci/hyperjump release-note/ci This PR makes changes to the CI. sig/datapath Impacts bpf/ or low-level forwarding details, including map management and monitor messages. sig/loader Impacts the loading of BPF programs into the kernel.
Projects
No open projects
1.13.2
Backport done to v1.13
Development

Successfully merging this pull request may close these issues.

Port verifier-test.sh to Go
4 participants