-
Notifications
You must be signed in to change notification settings - Fork 221
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Pyinstrument hiding useful frames #275
Comments
Hmm. Possibly there's a better heuristic I can use than The other thing I wonder is if you could put some pyinstrument options in a pyproject.toml, similar to how tools like black store options there. Would that work in your project structure? |
Might be better to use the internal import machinery tools to have better heuristics about when a package is third-party (or built-in). Also, this would be harder to do, but it would be useful to have some interactive way on the HTML pages to filter/unhide frames. And just generally speaking, anything that can be done in the HTML pages to make it easier to navigate really big traces would be useful. |
Thanks for chipping in @asmeurer ! What import mechanisms do you speak of? I've only looked into sys.path, but found that I couldn't easily distinguish between system and user paths. On the broader point , I'd love to see some examples of the kinds of profile sessions that you're seeing that would benefit from more interactivity? Perhaps you could share the pyisession files and the specific thing that you're trying to drill into? I tend to work on smaller scale projects so maybe don't see the need for that kind of feature but I'd love to improve the functionality for larger projects. |
I think this is what you're looking for. It's honestly really complicated and very customizable, but most people don't customize it so you can probably make some good heuristics https://docs.python.org/3/reference/import.html (sorry for not giving more details. I'm not really an expert on the Python import system) |
I did have another idea just now... to have another mechanism that captures |
@joerick Did you verify your idea? I am afraid I did not understand how would you differentiate the site wide packages from |
Profiling an internal functions with pyinstrument becomes difficult in non-interactive mode, as pyinstrument hides the frames with filepath containing
/lib/
by default.I found this old comment from the maintainer on an old related issue:
Since, all most all our (D.E. Shaw) python production code contains
/lib/
, pyinstrument thinks it as third party library code.Though, these frames are expandable in HTML format.
The workaround for non interactive mode we are following:
pyinstrument --show '*/prod/*' <your-file.py>
One solution we thought internally was to have better global defaults for our installation. What do you think about this idea, or could you suggest better alternatives?
The text was updated successfully, but these errors were encountered: