Fix doc building in CI

[pmeier]

Fixes #6875.

[pmeier]

Built docs: https://app.circleci.com/pipelines/github/pytorch/vision/21648/workflows/96527bb7-408d-4def-b84b-958be0aa0a8f/jobs/1754081

[NicolasHug]

Thanks for the PR @pmeier . The related VideoReader issue seems a bit fishy #6875 (comment), I think we should figure out whether there's something wrong with it before committing to removing -j, which is fairly useful to make the doc build fast

[pmeier]

This was missing

[pmeier]

These are no longer needed. flake8 didn't pick up on this since we blanket ignore F401, i.e. unused imports, in __init__.py files:

vision/setup.cfg

Lines 14 to 15 in c4c0ef9

	per-file-ignores =
	__init__.py: F401, F403, F405

[pmeier]

These are no longer available.

[pmeier]

Although the build hangs in the reading sources... stage, the error likely happens during the gallery build. Removing plot_video_api.py (or prefixing it with an _) resolved the issue with multiprocessing activated.

I'll investigate further what exactly is causing the issue.

[pmeier]

Narrowed it down to

vision/gallery/plot_video_api.py

Lines 98 to 107 in 1921613

	import itertools
	video.set_current_stream("video")
	frames = [] # we are going to save the frames here.
	# We seek into a second second of the video and use islice to get 10 frames since
	for frame, pts in itertools.islice(video.seek(2), 10):
	frames.append(frame)
	print("Total number of frames: ", len(frames))

Behavior can be reproduced by replacing this with

video.set_current_stream("video")

for _ in video.seek(2):
    pass

and commenting out the rest of file.

@jdsgomes @bjuncek any idea what your PR changed that could interfere with sphinx multiprocessing?

[pmeier]

I've pushed the reproduction to this PR. To reproduce locally, check this PR out and do

cd docs
pip install -r requirements.txt
make clean
make html

You probably also want to apply sphinx-doc/sphinx#10952 locally so using ctrl+c to terminate the hanging build actually terminates all sub-processes.

[jdsgomes]

build_docs is green now, will trigger rebuild to see it it is flaky

[pmeier]

It is still red on main: https://app.circleci.com/pipelines/github/pytorch/vision/21716/workflows/d4a070b9-cdb0-4929-a518-56e8b43fbe20/jobs/1759489. I'll try to reproduce again locally, but I could do so without issues yesterday.

[pmeier]

It seems I reduced it down to much. Trying to reproduce it again with the reduced version today did no longer work. No idea why it worked yesterday. Sorry for that.

However, reverting the file back to its full content surfaces the error again. I pushed the change to this PR. Docs CI is red the same as on main now: https://app.circleci.com/pipelines/github/pytorch/vision/21733/workflows/d97d374f-03f2-4795-bf51-6eadf8daa235/jobs/1760798

[pmeier]

The offending PR was reverted in #6908. Thus, I'm closing this.

From the investigation with @jdsgomes, the missing piece to reproduce this was a CPU only installation of PyTorch. Apart from that, CI uses

• python=3.7
• ffmpeg=4.2
• av<10