Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: add fuzzing set up #1497

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 2 additions & 0 deletions requirements/dev.pip
Original file line number Diff line number Diff line change
Expand Up @@ -531,3 +531,5 @@ setuptools==65.6.3 \
--hash=sha256:57f6f22bde4e042978bcd50176fdb381d7c21a9efa4041202288d3737a0c6a54 \
--hash=sha256:a7620757bf984b58deaf32fc8a4577a9bbc0850cf92c20e1ce41c38c19e5fb75
# via check-manifest
atheris==2.1.1 \
--hash=sha256:97d5111b9de4d581b8a51cac3ebcacd81b2d2a69661f99b9316194ea6a01f6c7
2 changes: 2 additions & 0 deletions requirements/pytest.pip
Original file line number Diff line number Diff line change
Expand Up @@ -106,3 +106,5 @@ zipp==3.11.0 \
--hash=sha256:83a28fcb75844b5c0cdaf5aa4003c2d728c77e05f5aeabe8e95e56727005fbaa \
--hash=sha256:a7a22e05929290a67401440b39690ae6563279bced5f314609d9d03798f56766
# via importlib-metadata
atheris==2.1.1 \
--hash=sha256:97d5111b9de4d581b8a51cac3ebcacd81b2d2a69661f99b9316194ea6a01f6c7
2 changes: 2 additions & 0 deletions requirements/tox.pip
Original file line number Diff line number Diff line change
Expand Up @@ -81,3 +81,5 @@ zipp==3.11.0 \
# via
# importlib-metadata
# importlib-resources
atheris==2.1.1 \
--hash=sha256:97d5111b9de4d581b8a51cac3ebcacd81b2d2a69661f99b9316194ea6a01f6c7
65 changes: 65 additions & 0 deletions tests/test_fuzz_parser.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
# Licensed under the Apache License: http://www.apache.org/licenses/LICENSE-2.0

"""Fuzz test PythonParser.parse_source

This runs on OSS-Fuzz where coveragepy's set is located at:
https://github.com/google/oss-fuzz/tree/master/projects/coveragepy

It is configured to be a unit test as well, which makes it easier to test
during development, e.g. to catch breaking changes.

The goal of the fuzzing by way of OSS-Fuzz is to:
- Find any uncaught illegitimate exceptions.
- Find any security vulnerabilities as identified by pysecsan:
https://pypi.org/project/pysecsan/
Notice, pysecsan will be enabled by OSS-Fuzz and is not explicitly enabled
here.
"""

import sys
import atheris
import pytest

from coverage.exceptions import NotPython
from coverage.parser import PythonParser


@pytest.mark.parametrize(
"data",
[
b"random_data",
b"more random data"
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand how atheris works. What does it do with these two data strings?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I only added these strings just to make it such that the fuzzer could be tested with pytest -- to e.g. check that the fuzzer won't break.

The data from pytest.mark.parametrize( is not used in the actual fuzzing runs.

]
)
def TestOneInput(data):
"""Fuzzer for PythonParser."""
fdp = atheris.FuzzedDataProvider(data)

t = fdp.ConsumeUnicodeNoSurrogates(1024)
if not t:
return

try:
p = PythonParser(text = t)
p.parse_source()
except (NotPython, MemoryError) as e2:
# Catch Memory error to avoid reporting stack overflows.
# Catch NotPython issues as these do not signal a bug.
pass
except ValueError as e:
if "source code string cannot contain null bytes" in str(e):
# Not interesting
pass
else:
raise e


def main():
"""Launch fuzzing campaign."""
atheris.instrument_all()
atheris.Setup(sys.argv, TestOneInput, enable_python_coverage=True)
atheris.Fuzz()


if __name__ == "__main__":
main()