Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(internal): bytecode wrapping context #9222

Merged
merged 7 commits into from May 21, 2024

Conversation

P403n1x87
Copy link
Contributor

@P403n1x87 P403n1x87 commented May 10, 2024

We introduce a new mechanism of wrapping functions via a special context manager that is capable of capturing return values as well. The goal is to allow observability into the called functions, to have access to local variables on exit. This approach has the extra benefit of not introducing any extra frames in the call stack of the wrapped function.

Checklist

  • Change(s) are motivated and described in the PR description
  • Testing strategy is described if automated tests are not included in the PR
  • Risks are described (performance impact, potential for breakage, maintainability)
  • Change is maintainable (easy to change, telemetry, documentation)
  • Library release note guidelines are followed or label changelog/no-changelog is set
  • Documentation is included (in-code, generated user docs, public corp docs)
  • Backport labels are set (if applicable)
  • If this PR changes the public interface, I've notified @DataDog/apm-tees.

Reviewer Checklist

  • Title is accurate
  • All changes are related to the pull request's stated goal
  • Description motivates each change
  • Avoids breaking API changes
  • Testing strategy adequately addresses listed risks
  • Change is maintainable (easy to change, telemetry, documentation)
  • Release note makes sense to a user of the library
  • Author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment
  • Backport labels are set in a manner that is consistent with the release branch maintenance policy

@P403n1x87 P403n1x87 added changelog/no-changelog A changelog entry is not required for this PR. Dynamic Instrumentation Dynamic Instrumentation/Live Debugger labels May 10, 2024
@P403n1x87 P403n1x87 self-assigned this May 10, 2024
@P403n1x87 P403n1x87 force-pushed the chore/wrapping-context branch 2 times, most recently from 2d92bf5 to 24e47ef Compare May 10, 2024 15:26
@codecov-commenter
Copy link

codecov-commenter commented May 10, 2024

Codecov Report

Attention: Patch coverage is 0.19608% with 509 lines in your changes are missing coverage. Please review.

Project coverage is 10.29%. Comparing base (c045d04) to head (8e6e5d5).

Files Patch % Lines
ddtrace/internal/wrapping/context.py 0.00% 325 Missing ⚠️
tests/internal/test_wrapping.py 0.00% 183 Missing ⚠️
ddtrace/internal/assembly.py 50.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##             main    #9222       +/-   ##
===========================================
- Coverage   76.17%   10.29%   -65.88%     
===========================================
  Files        1287     1258       -29     
  Lines      121979   120643     -1336     
===========================================
- Hits        92913    12425    -80488     
- Misses      29066   108218    +79152     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@P403n1x87 P403n1x87 marked this pull request as ready for review May 10, 2024 15:57
@P403n1x87 P403n1x87 requested review from a team as code owners May 10, 2024 15:57
@P403n1x87 P403n1x87 enabled auto-merge (squash) May 10, 2024 15:57
@P403n1x87 P403n1x87 force-pushed the chore/wrapping-context branch 3 times, most recently from 6f33192 to 6fb1599 Compare May 14, 2024 11:35
@P403n1x87 P403n1x87 force-pushed the chore/wrapping-context branch 3 times, most recently from 528268d to 4585201 Compare May 15, 2024 11:58
ddtrace/internal/wrapping/context.py Outdated Show resolved Hide resolved
ddtrace/internal/wrapping/context.py Outdated Show resolved Hide resolved
ddtrace/internal/wrapping/context.py Outdated Show resolved Hide resolved
Copy link
Member

@brettlangdon brettlangdon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mostly looks good, just some minor conversation pieces.

tests/internal/test_wrapping.py Show resolved Hide resolved
tests/internal/test_wrapping.py Show resolved Hide resolved
tests/internal/test_wrapping.py Show resolved Hide resolved
@P403n1x87 P403n1x87 force-pushed the chore/wrapping-context branch 2 times, most recently from d0cb7b1 to c14488c Compare May 17, 2024 15:27
We introduce a new mechanism of wrapping functions via a special context
manager that is capable of capturing return values as well. The goal is
to allow observability into the called functions, to have access to
local variables on exit. This approach has the extra benefit of not
introducing any extra frames in the call stack of the wrapped function.
tests/internal/test_wrapping.py Show resolved Hide resolved
tests/internal/test_wrapping.py Show resolved Hide resolved
@P403n1x87 P403n1x87 merged commit c5cf56f into DataDog:main May 21, 2024
199 of 204 checks passed
@P403n1x87 P403n1x87 deleted the chore/wrapping-context branch May 21, 2024 10:29
Copy link
Contributor

@shatzi shatzi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

few questions regarding this.

wc = BrokenExitWrappingContext(foo)
wc.wrap()

with pytest.raises(RuntimeError):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand why this is a feature.

This seems to me that broken wrapping can change the function logic/return value.

Does that the current behaviour of wrapping logic? Or this is something new?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is just to check that we get the right exception, in case __exit__ raises one. We basically test for the expected behaviour to ensure that exceptions don't get lost, or that we get the wrong one, making debugging harder

# the return value of the function. The __exit__ method is only called if the
# function raises an exception.
#
# Because CPython 3.11 introduced zero-cost exceptions, we cannot nest try
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not just use Python 3.11 method only to reduce complexity of the code. As I assume future versions will not go back on zero-cost exceptions.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We would have to use different opcodes anyway, so this shouldn't be much of a deal. The code is already written, so there is no need to change that 🙂

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
changelog/no-changelog A changelog entry is not required for this PR. Dynamic Instrumentation Dynamic Instrumentation/Live Debugger
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants