Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cleanup: Remove importlib.invalidate_caches calls #4565

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

hoodmane
Copy link
Member

It hasn't been needed since we brought in the patch that fixed the time functions on Emscripten:
emscripten-core/emscripten#18301

It hasn't been needed since we brought in the patch that fixed the time
functions on Emscripten:
emscripten-core/emscripten#18301
import sys
invalidate_caches()
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm test fails without invalidate_caches. I suspect this is due to a bug in the IDBFS...

@ryanking13
Copy link
Member

IIRC, there was a browser timer precision issue about this (#3311). But I guess it is worth to experiment again.

@hoodmane
Copy link
Member Author

Well we fixed that with #3313. I think it would be good to remove this because if there are still time resolution problems, we would find out about them faster. Otherwise it's a bit confusing when users directly writing files can't import them but our package resolution system won't ever see problems like that.

@ryanking13
Copy link
Member

ryanking13 commented Feb 27, 2024

But according to Roman's comment here: #3311 (comment), if the browser's timer precision is the problem, then mtime stored by emscripten won't be very accurate anyway as emscripten relies on browser APIs (Date.now()).

@hoodmane
Copy link
Member Author

Well but to reproduce the problem, you need to

  1. modify the directory
  2. import from the directory
  3. modify the directory again

All within a 1ms window, and then not modify the directory any further. Before we made this fix the window for triggering the problem was a full second. I just don't see that it's very likely to be a problem.

I guess what we should do that will make everyone happy is have the time functions always increment the current time by a nanosecond:

let lastTime;
let lastDelta = 0;
export function monotonicNanoseconds() {
  const now = Date.now();
  if (now === lastTime) {
    lastDelta++;
  } else {
    lastTime = now;
    lastDelta = 0;
  }
  return now * 1000 + lastDelta;
}

I think this will make it actually impossible to trigger problems with the importlib cache.

@hoodmane hoodmane marked this pull request as draft February 28, 2024 00:49
@ryanking13
Copy link
Member

I just don't see that it's very likely to be a problem.

Yeah, actually, I was just thinking it may be a browser issue, not that I think it's a real cause. In any case, if the import works well without invalidate_caches(), this is definitely a big improvement, so thanks for working on this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants