oss-fuzz integration is split across (at least) 3 repositories and is fragile

[samuelkarp]

oss-fuzz provides fuzzing for open source software such as containerd. Our integration with it is split in (at least) 3 repositories:

1. The contrib/fuzz directory and a workflow in our main repository.
2. The projects/containerd directory in the main oss-fuzz repository.
3. github.com/AdamKorcz/instrumentation.

This can make it challenging to find issues, and resolving them can take many separate pull requests.

Let's use this issue as a discussion forum for improvements.

[samuelkarp]

The Dockerfile in oss-fuzz pulls in the instrumentation repo and then runs a script which invokes another script held in this repository. Can we remove the indirection in the oss-fuzz repo and just clone instrumentation directly in our oss_fuzz_build.sh script? That seems to remove at least one coordination point.

[samuelkarp]

The use of instrumentation is not versioned at all. This can be helpful for the evolution of fuzzers, but introduces risk of breakage into our CI (i.e., cloning HEAD is not reproducible). Should we pin in our script and then bump when needed?

[estesp]

Also note that this checkout of the containerd repo does not use any information about the branch or PR in flight, although I think the "run fuzzers" does try and handle something PR related? (the default mode for the "run fuzzers" action is "code change" testing). However, I can't find the actual checking out the PR branch, but it could be hidden in the python code that gets kicked off after this checkout of main clearly seen in the project Dockerfile.

What this definitely means is that the release/1.7 fuzzer CI action and the main fuzzer CI action are doing the exact same thing for "build fuzzers" only again the 2.0 codebase; it should have broken in main when the packages were moved, but it is just silently exiting when it doesn't find the images package (moved from images/ to core/images/). Until we figure out if it is actually testing something in release/1.7 we might as well disable the action.

[vishalRGurrala]

Hi there,

I understand your integration with oss-fuzz. I feel it is better to maintain the entire build process in the build.sh because we're anyway cloning our entire repo in the Dockerfile and also incase of any issue, it would direct all the pull requests to oss-fuzz. It is easy to maintain.

Yes, I agree it is better to maintain a specific state of instrumentation, not doing so might effect the fuzz test results.

Yes, CI action for build fuzzers is always taking the latest commit on the default branch. I verified it from the build job logs. But according to this line, the run fuzzers uses a default mode which should fetch the changes from pull request in theory. I can verify this by explicitly adding a crash scenario on my PR.