Capture stdout and stderr

[rpkak]

Closes #886

Captures the stdout and the stderr of a job and makes it accessible using job.stdout ans job.stderr.

[codecov]

Codecov Report

Merging #1515 (ae3ed94) into master (76ba690) will decrease coverage by 0.10%.
The diff coverage is 87.35%.

@@            Coverage Diff             @@
##           master    #1515      +/-   ##
==========================================
- Coverage   95.59%   95.49%   -0.11%     
==========================================
  Files          46       46              
  Lines        7058     7145      +87     
==========================================
+ Hits         6747     6823      +76     
- Misses        311      322      +11     

Impacted Files	Coverage Δ
tests/fixtures.py	72.04% <33.33%> (-0.74%)	⬇️
rq/worker.py	88.03% <57.14%> (-0.60%)	⬇️
rq/job.py	97.85% <93.61%> (-0.33%)	⬇️
rq/queue.py	93.92% <100.00%> (+0.03%)	⬆️
tests/test_job.py	100.00% <100.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 76ba690...ae3ed94. Read the comment docs.

[selwin]

I like the idea of workers writing to stdout and stderr, think should be achievable by tweaking the default logging configuration.

I'm not sure about saving stdout and stderr it to Redis though, it may significantly increase storage usage.

[rpkak]

I made this optional now.

You have to activate this using queue.enqueue(func, capture_stdout=True, capture_stderr=True).

[kuraga]

Any news?

[Hongyuy]

looking forward to using this feature.

[caiobsilva]

any reason as to why this PR hasn't been merged?

[dberardo-com]

i could confirm that this PR works as expected, but the streams are not closed when the function has run till the end. is this by design ?

i need to use this workaround:

job=q.enqueue(test_loop, capture_stdout=True, capture_stderr=True, retry=Retry(max=1, interval=[.2, 1, 2]), iterations=6)
try:
for line in iter(job._stdout.strio.readline, b''):
if line:
print(">>> " + line.rstrip())
if job.is_finished and not job._stdout.strio.closed: job._stdout.strio.close()
except ValueError: pass
assert job._stdout.strio.closed and job.is_finished