Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deadline package leaks memory #208

Closed
floatingstatic opened this issue Oct 20, 2022 · 0 comments · Fixed by #209
Closed

Deadline package leaks memory #208

floatingstatic opened this issue Oct 20, 2022 · 0 comments · Fixed by #209

Comments

@floatingstatic
Copy link
Contributor

floatingstatic commented Oct 20, 2022

Your environment.

  • Version: v0.13.1
  • Browser: N/A
  • Other Information: N/A

What did you do?

I utilize the pion/dtls package which imports pion/transport for a private project. I noticed some scaling issues with my service and upon running pprof I noticed a lot of garbage collection activity related to the deadline package within this repo. This is due to updating the deadline on a DTLS connection via the pion/dtls SetReadDeadline() method on every packet (we receive upwards of 50kpps on my service) so as you can imagine we are hitting that function call a lot as means of detecting dead DTLS connections.
Some output from pprof showing most of the allocated memory and objects come from deadline.Set() which uses time.After() which as currently implemented doesn't release the underlying timer to the garbage collector until it fires (we use a deadline of about 20 seconds).

196728683-6c9195ed-d2c9-47a0-9414-4964d9657315

time.After 's documentation states that The underlying Timer is not recovered by the garbage collector until the timer fires. If efficiency is a concern, use NewTimer instead and call Timer.Stop if the timer is no longer needed.

A useful blog post explaining a very similar to problem to the one I hit can be found here

I think moving towards using timer.NewTimer() directly would be more efficient (though I have currently worked around this via other means).

What did you expect?

Calling deadline.Set() should efficiently release unused memory to the garbage collector.

What happened?

My service that indirectly imports pion/transport spent too much time in garbage collection as a result of calling deadline.Set() at a high rate.

floatingstatic added a commit to floatingstatic/transport that referenced this issue Oct 20, 2022
Signed-off-by: Jeremiah Millay <jmillay@fastly.com>
floatingstatic added a commit to floatingstatic/transport that referenced this issue Oct 20, 2022
Signed-off-by: Jeremiah Millay <jmillay@fastly.com>
stv0g pushed a commit that referenced this issue Nov 15, 2022
Signed-off-by: Jeremiah Millay <jmillay@fastly.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant