Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Perf: Reduce memory overhead in transitive dependency #78651

Closed
wants to merge 2 commits into from

Conversation

toddtreece
Copy link
Member

@toddtreece toddtreece commented Nov 24, 2023

What is this feature?

image

Reduces go_memstats_heap_inuse_bytes from ~65MB to ~45MB on a mostly idle (except profiling) Grafana instance, by switching to lazy initialization of global variables in go-json.

Related PR: goccy/go-json#490

Why do we need this feature?

go-json is a dependency of Apache Arrow v13. Grafana does not currently use Arrow's JSON encoding/decoding, but Arrow's import of go-json is adding additional memory overhead due to how some global variables are initialized.

This is pretty easy to spot when looking at Pyroscope flamegraphs:
image

Who is this feature for?

[Add information on what kind of user the feature is for.]

Which issue(s) does this PR fix?:

Fixes #

Special notes for your reviewer:

Please check that:

  • It works as expected from a user's perspective.
  • If this is a pre-GA feature, it is behind a feature toggle.
  • The docs are updated, and if this is a notable improvement, it's added to our What's New doc.

@grafana-delivery-bot grafana-delivery-bot bot added this to the 10.3.x milestone Nov 24, 2023
@toddtreece toddtreece added no-backport Skip backport of PR no-changelog Skip including change in changelog/release notes labels Dec 19, 2023
@grafana grafana deleted a comment from ephemeral-instances-bot bot Dec 19, 2023
@grafana grafana deleted a comment from ephemeral-instances-bot bot Dec 19, 2023
@toddtreece
Copy link
Member Author

/deploy-to-hg

@ephemeral-instances-bot
Copy link

  • Preparing your instance. A comment containing your instance's url will be added to this PR when the instance is ready.
  • Your instance will be ready in ~10 minutes.
  • Check the GitHub actions tab to follow the workflow progress
  • Slack channel: #proj-ephemeral-hg-instances
  • Building instance with toddtreece/arrow-json oss branch and main enterprise branch. How to choose a branch

@toddtreece toddtreece marked this pull request as ready for review December 19, 2023 03:07
@toddtreece toddtreece requested a review from a team as a code owner December 19, 2023 03:07
@toddtreece toddtreece requested review from zserge, idafurjes, yangkb09 and a team and removed request for a team December 19, 2023 03:07
@toddtreece toddtreece changed the title Perf: Reduce Apache Arrow memory overhead Perf: Reduce memory overhead in transitive dependency Dec 19, 2023
@ephemeral-instances-bot
Copy link

Copy link
Contributor

This pull request has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in 2 weeks if no further activity occurs. Please feel free to give a status update or ping for review. Thank you for your contributions!

@github-actions github-actions bot added the stale Issue with no recent activity label Feb 25, 2024
Copy link
Contributor

This pull request has been automatically closed because it has not had any further activity in the last 2 weeks. Thank you for your contributions!

@github-actions github-actions bot closed this Mar 11, 2024
@yuri-tceretian yuri-tceretian modified the milestones: 10.3.x, 10.3.5 Mar 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/backend no-backport Skip backport of PR no-changelog Skip including change in changelog/release notes stale Issue with no recent activity
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants