Skip to content

appsec: integrate the new purego-based go-libddwaf #2090

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 14 commits into from
Jul 5, 2023

Conversation

Julio-Guerra
Copy link
Contributor

@Julio-Guerra Julio-Guerra commented Jul 3, 2023

What does this PR do?

This go-libddwaf version now relies on the dynamic liddwaf library instead of the static one, via dlopen() and dlsym() internally.
This allows us to no longer rely on CGO at compilation time, with all the requirements it needed such as a proper C toolchain. To do so, we now rely on https://github.com/ebitengine/purego which mainly provides a high-level API on top of runtime/cgo to call the C world.

The overall API of the package didn't change, but we split the former Health() function in two to better integrate it into dd-trace-go and handle the different failure points correctly:

  1. Log an error and quit appsec if we are in a host environment not supported by libddwaf, as early as possible, avoiding for instance to start remote configuration.
  2. Log an error and quit appsec in case of unexpected errors when loading libddwaf (eg. if the filesystem is read-only and we can't dump the libddwaf.so file)
  3. Log and error but keep going in case of unexpected non-critical errors.

Motivation

No more CGO prerequisites, mainly:

  1. C toolchain, with extra extra complexity when cross-compiling (even on simple cases like from a debian (glibc-based) to alpine (musl-based)
  2. CGO_ENABLED=1

Describe how to test/QA your changes

Reviewer's Checklist

  • Changed code has unit tests for its functionality.
  • If this interacts with the agent in a new way, a system test has been added.

Sorry, something went wrong.

@pr-commenter
Copy link

pr-commenter bot commented Jul 3, 2023

Benchmarks

Benchmark execution time: 2023-07-05 13:18:51

Comparing candidate commit c753165 in PR branch julio.guerra/new-go-libddwaf-purego-api with baseline commit c10c902 in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 27 metrics, 0 unstable metrics.

@Julio-Guerra Julio-Guerra changed the title Julio.guerra/new go libddwaf purego api appsec: integrate the new purego-based go-libddwaf Jul 3, 2023
@eliottness
Copy link
Contributor

Should we wait until the go-libddwaf final PR is merged to merge this one ?

Julio-Guerra added a commit to DataDog/go-libddwaf that referenced this pull request Jul 5, 2023
- `func Load() (ok bool, err error)` to ease the integration in
dd-trace-go so that we can distinguish the different possible error
cases:
- Panic errors that must be used to gracefully abort the caller and
prevent it from continuing. Such errors are represented by the new
`PanicError` error type.
- Unsupported target OS and/or architecture, which must be handled as an
informative error, is worth logging to the user, and prevents the caller
from continuing. This error is represented by the new
`UnsupportedTarget` error type.
- Every other unexpected error that can possible happen is returned
as-is, but `ok` is `false`, so that the caller knows it cannot use the
package.
- Informative errors, about internal non-critical failures, can also
happen (eg. if os.Remove fails removing the temporary files), and are
returned as-is, but with `ok` being `true` so that the caller can log
the error and continue its startup.
- The dd-trace-go integration of this PR can be found here
DataDog/dd-trace-go#2090
- Note that `Health()` has been kept for the sake of simplicity
health-checking cases such as in the tests, where it returns the
`Load()` error only if `ok` is false, meaning that go-libddwaf is not
healthy.
- Simplified build tags, removed everywhere it was possible, down to
purego which isn't properly stubbed for unsupported targets and which we
stub ourselves at our level in the new `waf_dl_unsupported.go` file.

---------

Co-authored-by: Eliott Bouhana <47679741+eliottness@users.noreply.github.com>
Co-authored-by: François Mazeau <francois.mazeau@datadoghq.com>
Copy link
Contributor

@Hellzy Hellzy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM overall, a a few comments about debug logs

@Julio-Guerra Julio-Guerra marked this pull request as ready for review July 5, 2023 13:08
@Julio-Guerra Julio-Guerra requested review from a team as code owners July 5, 2023 13:08
@Julio-Guerra Julio-Guerra requested review from Hellzy and eliottness July 5, 2023 13:12
Copy link
Contributor

@katiehockman katiehockman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Didn't review the code, but the overall idea looks good. This is so awesome!
Approving just based on the go.mod and go.sum files. LMK if you want another set of eyes on the internal/appsec changes you made here, too, and I can check it out.

@Julio-Guerra Julio-Guerra merged commit 7f9ab67 into main Jul 5, 2023
@Julio-Guerra Julio-Guerra deleted the julio.guerra/new-go-libddwaf-purego-api branch July 5, 2023 19:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants