Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reset interpreter session and reuse the interpreter (isolation level) #114

Open
eldipa opened this issue Jan 25, 2020 · 1 comment
Open
Labels
enhancement something nice to have but it is not neither critical nor urgent

Comments

@eldipa
Copy link
Collaborator

eldipa commented Jan 25, 2020

Describe the feature you'd like
For each file, byexample spawns an interpreter to run the examples and it will shutdown after processing the file.

If multiple files have examples of the same language, byexample will still spawn and shutdown the interpreter for each file.

This is quite slow.

Some interpreters have the concept of sessions: independent execution sessions with a clean environment.

We could use it to reset the session instead of spawning another interpreter which it should be much faster.

This opens several isolation levels:

  • process: restart each interpreter, always
  • session: restart the session if supported, restart the interpreter otherwise
  • cleanup: within the same session, perform some actions to clean up it.
  • none: do not do anything, it is the user that must code her examples correctly. byexample must still do some clean up like resetting the clipboard (isolation == none is not a way to concatenate files)
@eldipa eldipa added the enhancement something nice to have but it is not neither critical nor urgent label Jan 25, 2020
@eldipa eldipa added this to the 10.0.0 milestone Mar 13, 2021
@eldipa
Copy link
Collaborator Author

eldipa commented Mar 15, 2021

Spawning and stopping an interpreter per file ensures that the interpreter is closed. If the next file to exec does not use that interpreter it is okay.
But if we impl this "reset" feature, an interpreter will survive the exec of one file even if the next file does not require it.

We need to ensure that the interpreter is shutdown if no other file will require it.

Offtopic: it would be interesting to make byexample to schedule the files to exec based on the interpreters that requires. When we run them in parallel it would be cool to balance the load: mix files that require less or more interpreters so we don't spawn a lot of interpreters for some moments and then run very few in others.

eldipa added a commit that referenced this issue Mar 17, 2021
If we don't want to force a shutdown and the runner supports a reset(),
do it instead of a shutdown.

A reseted runner will still be alive and a subsequent initialization
will be skipped.

This allows the runner to implement a reset of the interpreter's state
instead of a full shutdown/re-spawn.

Documented runner's initialize(), reset() and shutdown() and refactored
FileExecutor's initialize_runners() and shutdown_runners() (now called
reset_runners()).

This should backward compatible.

FileExecutor for now it always force a shutdown so the whole thing works
as before.

Also it is missing the shutdown of the still alive runners.
eldipa added a commit that referenced this issue Mar 17, 2021
At the end of the job, after processing all the files, call close on
FileExecutor and Harvester.

This allows to do clean up stuff like shutting down any runner still
alive in the FileExecutor.
@eldipa eldipa removed this from the 10.0.0 milestone Mar 17, 2021
eldipa added a commit that referenced this issue Mar 24, 2021
If we don't want to force a shutdown and the runner supports a reset(),
do it instead of a shutdown.

A reseted runner will still be alive and a subsequent initialization
will be skipped.

This allows the runner to implement a reset of the interpreter's state
instead of a full shutdown/re-spawn.

Documented runner's initialize(), reset() and shutdown() and refactored
FileExecutor's initialize_runners() and shutdown_runners() (now called
reset_runners()).

This should backward compatible.

FileExecutor for now it always force a shutdown so the whole thing works
as before.

Also it is missing the shutdown of the still alive runners.
eldipa added a commit that referenced this issue Mar 24, 2021
At the end of the job, after processing all the files, call close on
FileExecutor and Harvester.

This allows to do clean up stuff like shutting down any runner still
alive in the FileExecutor.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement something nice to have but it is not neither critical nor urgent
Projects
None yet
Development

No branches or pull requests

1 participant