Stop yielding once all evaluations are started

[andrews05]

This improves upon the previous change in #524. Rather than yielding as long there are tasks available, it keeps of track of how many evaluations have been started and only yields when there are still evaluations to be done.

This is important if there is a large queue of other (non-evaluation) tasks available. It doesn't guarantee that it won't pick up a non-evaluation task but it does make it highly unlikely and effectively eliminates any problems that may arise from that happening too often. (@Pr0methean has yet to confirm whether this fixes his stack overflow problem in #517 but I'm pretty confident that it does)

@shssoichiro Note my other PRs are also ready for review 🙂

[cuviper]

I commented on rayon before I had seen your change: rayon-rs/rayon#1064 (comment)

My idea was to use try_recv until disconnected. That's not much different than what you're doing, except it would let you use the channel's internal connection count instead of adding a separate counter yourself.

[andrews05]

@cuviper Thanks for the suggestion! I tried this out just now - the difference is the separate counter here allows it to stop yielding as soon the evaluations have all been started, whereas checking try_recv will keep yielding until the evaluations are all finished, meaning it will most likely pick up some other task while waiting. It probably wouldn't be a problem (since it would still be a major improvement over the previous code), but I think it would be preferable to avoid that as much as possible.

[cuviper]

Makes sense! Although I will point out that your commit and PR subject say "done", not "started". :)
(... and then you did say "started" in the body.)

[andrews05]

Ha, good point - changed :)

[Pr0methean]

This fix seems to work so far.