Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dedicated cache directory per ruff version #8333

Merged
merged 1 commit into from Oct 30, 2023

Conversation

MichaReiser
Copy link
Member

Summary

This PR changes our cache to use a dedicated cache directory per ruff version.

  • Before: .ruff_cache/content
  • After: .ruff_cache/0.1.3

The motivation is that I want to rule out that hash collisions between cache keys causing ruff to load a cache created by another version.
Loading incompatible caches might be the root cause for #8147 (although unlikely, because users mention that they didn't upgrade ruff)
Other hash collissions are less problematic, because the Cache checks if the loaded file is for the same package, and otherwise clears the cache for that package.

Open questions

I'm unsure if this is considered a breaking change or not. It does change observable behavior of Ruff but we never included it in our public API.

Test Plan

Verified that the cache is still used by comparing performance before/after

@MichaReiser
Copy link
Member Author

Current dependencies on/for this PR:

This comment was auto-generated by Graphite.

@@ -102,9 +101,8 @@ impl Cache {
pub(crate) fn open(package_root: PathBuf, settings: &Settings) -> Self {
debug_assert!(package_root.is_absolute(), "package root not canonicalized");

let mut buf = itoa::Buffer::new();
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using itoa to write a single number felt overkill.

Copy link
Member

@charliermarsh charliermarsh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will we still automatically clear old entries with this approach?

@github-actions
Copy link

PR Check Results

Ecosystem

✅ ecosystem check detected no linter changes.

✅ ecosystem check detected no format changes.

@MichaReiser
Copy link
Member Author

Will we still automatically clear old entries with this approach?

We continue to automatically clear old file entries, the same as the old version.

What the version doesn't (nor did the baseline) is to clean up caches of old versions (except if you run ruff clean)

@charliermarsh
Copy link
Member

Oh interesting, so even the existing cache will never remove files from old versions?

@MichaReiser
Copy link
Member Author

Oh interesting, so even the existing cache will never remove files from old versions?

Not as far as I know. The only cleanup code that I see is the removal of stale entries:

// Remove cached files that we haven't seen in a while.
let now = self.last_seen_cache;
self.package.files.retain(|_, file| {
// SAFETY: this will be truncated to the year ~2554.
(now - *file.last_seen.get_mut()) <= MAX_LAST_SEEN.as_millis() as u64
});

But there's no operation that iterates over all files and cleans unused entries, other than the ruff clean command

@MichaReiser MichaReiser merged commit 8cc97f7 into main Oct 30, 2023
17 checks passed
@MichaReiser MichaReiser deleted the 10-30-Dedicated_cache_directory_per_ruff_version branch October 30, 2023 09:08
@MichaReiser MichaReiser added the cli Related to the command-line interface label Oct 30, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cli Related to the command-line interface
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants