Unable to switch latest from master to main and build successfully

[kevingreenman]

I had posted this in a comment in a closed issue a couple weeks ago, but decided to open a separate issue because I think it might be a different problem than what others were facing in #7907

I'm having an issue with this while using RTD with the chemprop package. Chemprop has been on RTD for several years and has previously always had master as its primary branch. In the RTD settings, the default branch was never set (it said --------- and thus pointed to the default of master).

We recently released a new version of chemprop that we had been developing on our v2/dev branch, which was also active as a version on RTD. When we were ready to release our new stable version, we renamed v2/dev to main and then changed our primary branch in our GitHub repo settings from master to main. On RTD, I deactivated the v2/dev version and changed the default branch from --------- to main. The latest build now fails.

The error we're seeing is

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/chemprop/envs/latest/lib/python3.11/site-packages/sphinx/cmd/build.py", line 332, in build_main
    app = Sphinx(args.sourcedir, args.confdir, args.outputdir,
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/chemprop/envs/latest/lib/python3.11/site-packages/sphinx/application.py", line 268, in __init__
    self._init_builder()
  File "/home/docs/checkouts/readthedocs.org/user_builds/chemprop/envs/latest/lib/python3.11/site-packages/sphinx/application.py", line 339, in _init_builder
    self.events.emit('builder-inited')
  File "/home/docs/checkouts/readthedocs.org/user_builds/chemprop/envs/latest/lib/python3.11/site-packages/sphinx/events.py", line 97, in emit
    results.append(listener.handler(self.app, *args))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/chemprop/envs/latest/lib/python3.11/site-packages/autoapi/extension.py", line 124, in run_autoapi
    if sphinx_mapper_obj.load(
       ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/chemprop/envs/latest/lib/python3.11/site-packages/autoapi/mappers/python/mapper.py", line 287, in load
    raise ExtensionError(f"No source files found in: {','.join(dirs)}")
sphinx.errors.ExtensionError: No source files found in: /home/docs/checkouts/readthedocs.org/user_builds/chemprop/checkouts/latest
Extension error:
No source files found in: /home/docs/checkouts/readthedocs.org/user_builds/chemprop/checkouts/latest

I created a new RTD version that points directly to main, and that build succeeds, even though the code is exactly the same as for the latest version.

We're also tracking this in our own issue: chemprop/chemprop#828

Originally posted by @kevingreenman in #7907 (comment)

[humitos]

I took a quick look at this and it seems that AutoAPI is ignoring all the Python files on the build that fails. Example:

[AutoAPI] Reading files... [ 51%] /home/docs/checkouts/readthedocs.org/user_builds/chemprop/checkouts/main/chemprop/schedulers.py

and

[AutoAPI] Ignoring /home/docs/checkouts/readthedocs.org/user_builds/chemprop/checkouts/latest/chemprop/schedulers.py

It seems that's the problem. Maybe there is a Sphinx configuration that needs to be changed?

[kevingreenman]

@humitos thanks for your reply. I'm not sure I understand why a sphinx change would be necessary if the build already succeeds when it's pointed directly to main, instead of when it's trying to build latest, which points to main (the code it's trying to build from is exactly the same in both cases). Do you have any ideas of what sphinx configurations would be good to try changing?

[humitos]

@kevingreenman No, I'm not sure what it could be 🤷🏼 . The only differences that I found taking a look at the logs was that reading/ignoring coming from Sphinx.

[agjohnson]

@kevingreenman As @humitos noted, this seems to be a Sphinx configuration issue with your project. If you inspect the raw logs between the two builds, you'll see that the build on version main doesn't find the same files as the build on version latest, but otherwise all the installed dependencies are the same versions. So this implies your Sphinx configuration is conditionally changing.

I found the following in your configuration:

autoapi_ignore = ["*test*", "*cli*"]

https://beta.readthedocs.org/projects/chemprop/builds/24434895/#231043343--42

This pattern match is too open, it will match a path with any instance of the string test. Because the working path is named /home/docs/checkouts/readthedocs.org/user_builds/chemprop/checkouts/latest/... every file is ignored in this path -- that is, test matches latest.

Hopefully that points you in the right direction. I'll close this issue as it is not an issue with Read the Docs but is instead a content issue.

[kevingreenman]

@agjohnson thank you! This was fixed in our most recent build by changing from autoapi_ignore = ["*test*", "*cli*"] to autoapi_ignore = ["*/tests/*", "*/cli/*"].