Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use last line of the expected output to sync with the interpreter in addition to the prompt #181

Open
eldipa opened this issue Jun 25, 2021 · 0 comments
Labels
enhancement something nice to have but it is not neither critical nor urgent

Comments

@eldipa
Copy link
Collaborator

eldipa commented Jun 25, 2021

Describe the feature you'd like
byexample expects the main prompt (PS1) to know when an example finished, recollects all the output before the prompt and compares it with the expected output for that example.

This reasonably assumes that all the output arrives before the prompt and nothing arrives after.

But not always is the case: an example may run in background and the interpreter may print the prompt before the example finishes. The result would be the last part of the output of the example may be missing and byexample will mark the example as failed but it also the missing output may appear at the begin of the next example output, unexpected for it, and byexample will mark it as failed too.

In #176 we added a -x-delayafterprompt to wait a fixed amount of time and try to read any output after the prompt but a fixed sleep isn't good for the performance and it is fragile anyways.

byexample could use the example's expected last lines as part of the synchronization.

Proposed solution

We could expect the prompt as usual and then check if the last expected line is in the output read. If isn't, try to do a second expect for that line and add any new output to the already read.

This extra check may fail due a timeout if:

  1. the output never arrives and we are still out of sync
  2. there is no output at all because the example is really printing something totally different

We cannot do much for 1 except increase its timeout. But that will go against 2: the second case is the happy case where we are not out of sync (but the example still fails) and increasing the timeout of the extra check will slowdown the execution.

We also needs to define which it means "last line". Which would be the minimum length for such thing to be useful for synchronization?.

Additional context (optional)
This issue probably a hard-dependency for #176 because evcxr, the rust interpreter redirects the examples' outputs (stdout and stderr) to the terminal in background. This seems to affect only when the example does println!()-like something so it is not going to be the most common case but certainly if the user hit the issue it will be confusing for him/her.

@eldipa eldipa added the enhancement something nice to have but it is not neither critical nor urgent label Jun 25, 2021
@eldipa eldipa mentioned this issue Jun 25, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement something nice to have but it is not neither critical nor urgent
Projects
None yet
Development

No branches or pull requests

1 participant