Remove one fallback to full indexes on big gemfiles #6578
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
deivid-rodriguez
force-pushed
the
remove-full-index-fallback-on-big-gemfiles
branch
4 times, most recently
from
653cef0
to
438b636
Compare
If Gemfile has a lot of dependencies, we have an optimization that uses the full index in that case, assuming it's going to be faster. I think this is an old optimization that predates compact index API times, I believe we no longer need it these days. Also, since a few releases ago we check for circular dependencies when resolving by looping through all versions of each name and removing those that have circular dependencies that would trip up the resolver. This loop becomes actually very slow when full indexes are used because to find dependencies of a gemspec, we need to explicitly fetch the marshaled gemspec (`gemspec.rz` endpoint) for it, so the optimization has the opposite effect of making things very slow.
deivid-rodriguez
force-pushed
the
remove-full-index-fallback-on-big-gemfiles
branch
from
438b636
to
2f46289
Compare
|
Thanks for this! I was just investigating why our Would you be the right person to release an update for this? |
deivid-rodriguez
added a commit
that referenced
this pull request
Apr 10, 2023
…big-gemfiles Remove one fallback to full indexes on big gemfiles (cherry picked from commit 560e37f)
|
Currently preparing releases 👍. |
|
Thank you! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What was the end-user or developer problem that led to this PR?
If Gemfile has a lot of dependencies, we have an optimization that uses the full index in that case, assuming it's going to be faster.
I think this is an old optimization that predates compact index API times, I believe we no longer need it these days.
Also, since a few releases ago we check for circular dependencies when resolving by looping through all versions of each name and removing those that have circular dependencies that would trip up the resolver.
This loop becomes actually very slow when full indexes are used because to find dependencies of a gemspec, we need to explicitly fetch the marshaled gemspec (
gemspec.rzendpoint) for it, so the optimization has the opposite effect of making things very slow.What is your fix for the problem, implemented in this PR?
Consistently use dependency APIs without special casing Gemfiles with a lot of dependencies.
Make sure the following tasks are checked