Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reference targets not found when using autodoc_typehints = "both" #10305

Closed
AbstractUmbra opened this issue Mar 28, 2022 · 9 comments
Closed

Comments

@AbstractUmbra
Copy link

Describe the bug

When using the above directive in my conf.py for sphinx, I suddenly get the following output:

(hondana-epKOD00U-py3.9)  π hondana/docs docs/fixup ✗ ❯ poe docs
Poe => poetry run sphinx-build -a -E -n -T -W --keep-going docs/ docs/_build
Running Sphinx v4.5.0
loading intersphinx inventory from https://docs.python.org/3/objects.inv...
loading intersphinx inventory from https://docs.aiohttp.org/en/stable/objects.inv...
building [mo]: all of 0 po files
building [html]: all source files
updating environment: [new config] 5 added, 0 changed, 0 removed
reading sources... [100%] types                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [100%] types                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
/home/penumbra/projects/personal/hondana/hondana/client.py:docstring of hondana.client.Client.follow_manga:: WARNING: py:class reference target not found: hondana.enums.ReadingStatus
/home/penumbra/projects/personal/hondana/hondana/client.py:docstring of hondana.client.Client.update_manga_reading_status:: WARNING: py:class reference target not found: hondana.enums.ReadingStatus
/home/penumbra/projects/personal/hondana/hondana/client.py:docstring of hondana.client.Client.create_manga_relation:: WARNING: py:class reference target not found: hondana.enums.MangaRelationType
/home/penumbra/projects/personal/hondana/hondana/client.py:docstring of hondana.client.Client.upload_cover:: WARNING: py:class reference target not found: common.LanguageCode
/home/penumbra/projects/personal/hondana/hondana/client.py:docstring of hondana.client.Client.legacy_id_mapping:: WARNING: py:class reference target not found: legacy.LegacyMappingType
/home/penumbra/projects/personal/hondana/hondana/client.py:docstring of hondana.client.Client.create_custom_list:: WARNING: py:class reference target not found: hondana.enums.CustomListVisibility
/home/penumbra/projects/personal/hondana/hondana/client.py:docstring of hondana.client.Client.update_custom_list:: WARNING: py:class reference target not found: hondana.enums.CustomListVisibility
/home/penumbra/projects/personal/hondana/hondana/client.py:docstring of hondana.client.Client.get_custom_list_manga_feed:: WARNING: py:class reference target not found: common.LanguageCode
/home/penumbra/projects/personal/hondana/hondana/client.py:docstring of hondana.client.Client.get_custom_list_manga_feed:: WARNING: py:class reference target not found: common.LanguageCode
/home/penumbra/projects/personal/hondana/hondana/client.py:docstring of hondana.client.Client.get_custom_list_manga_feed:: WARNING: py:class reference target not found: common.LanguageCode
/home/penumbra/projects/personal/hondana/hondana/client.py:docstring of hondana.client.Client.update_scanlation_group:: WARNING: py:class reference target not found: common.LanguageCode
/home/penumbra/projects/personal/hondana/hondana/client.py:docstring of hondana.client.Client.get_report_list:: WARNING: py:class reference target not found: hondana.enums.ReportCategory
/home/penumbra/projects/personal/hondana/hondana/client.py:docstring of hondana.client.Client.create_report:: WARNING: py:class reference target not found: hondana.enums.ReportCategory
/home/penumbra/projects/personal/hondana/hondana/client.py:docstring of hondana.client.Client.upload_session:: WARNING: py:class reference target not found: common.LanguageCode
/home/penumbra/projects/personal/hondana/hondana/client.py:docstring of hondana.client.Client.upload_chapter:: WARNING: py:class reference target not found: common.LanguageCode
/home/penumbra/projects/personal/hondana/hondana/chapter.py:docstring of hondana.chapter.Chapter.download:: WARNING: py:class reference target not found: PathLike
/home/penumbra/projects/personal/hondana/hondana/chapter.py:docstring of hondana.chapter.ChapterUpload:: WARNING: py:class reference target not found: HTTPClient
/home/penumbra/projects/personal/hondana/hondana/chapter.py:docstring of hondana.chapter.ChapterUpload:: WARNING: py:class reference target not found: LanguageCode
/home/penumbra/projects/personal/hondana/hondana/collections.py:docstring of hondana.collections.BaseCollection.items:: WARNING: py:class reference target not found: T
/home/penumbra/projects/personal/hondana/hondana/manga.py:docstring of hondana.manga.Manga.follow:: WARNING: py:class reference target not found: hondana.enums.ReadingStatus
/home/penumbra/projects/personal/hondana/hondana/manga.py:docstring of hondana.manga.Manga.localised_title:: WARNING: py:class reference target not found: LanguageCode
/home/penumbra/projects/personal/hondana/hondana/manga.py:docstring of hondana.manga.Manga.localised_title:: WARNING: py:class reference target not found: LanguageCode
/home/penumbra/projects/personal/hondana/hondana/manga.py:docstring of hondana.manga.Manga.localised_description:: WARNING: py:class reference target not found: LanguageCode
/home/penumbra/projects/personal/hondana/hondana/manga.py:docstring of hondana.manga.Manga.localised_description:: WARNING: py:class reference target not found: LanguageCode
/home/penumbra/projects/personal/hondana/hondana/manga.py:docstring of hondana.manga.Manga.update_reading_status:: WARNING: py:class reference target not found: hondana.enums.ReadingStatus
/home/penumbra/projects/personal/hondana/hondana/manga.py:docstring of hondana.manga.Manga.upload_cover:: WARNING: py:class reference target not found: LanguageCode
/home/penumbra/projects/personal/hondana/hondana/manga.py:docstring of hondana.manga.Manga.create_relation:: WARNING: py:class reference target not found: hondana.enums.MangaRelationType
/home/penumbra/projects/personal/hondana/hondana/query.py:docstring of hondana.query.MangaListOrderQuery:: WARNING: py:class reference target not found: hondana.enums.Order
/home/penumbra/projects/personal/hondana/hondana/query.py:docstring of hondana.query.FeedOrderQuery:: WARNING: py:class reference target not found: hondana.enums.Order
/home/penumbra/projects/personal/hondana/hondana/query.py:docstring of hondana.query.MangaDraftListOrderQuery:: WARNING: py:class reference target not found: hondana.enums.Order
/home/penumbra/projects/personal/hondana/hondana/query.py:docstring of hondana.query.CoverArtListOrderQuery:: WARNING: py:class reference target not found: hondana.enums.Order
/home/penumbra/projects/personal/hondana/hondana/query.py:docstring of hondana.query.ScanlatorGroupListOrderQuery:: WARNING: py:class reference target not found: hondana.enums.Order
/home/penumbra/projects/personal/hondana/hondana/query.py:docstring of hondana.query.AuthorListOrderQuery:: WARNING: py:class reference target not found: hondana.enums.Order
/home/penumbra/projects/personal/hondana/hondana/query.py:docstring of hondana.query.UserListOrderQuery:: WARNING: py:class reference target not found: hondana.enums.Order
/home/penumbra/projects/personal/hondana/hondana/scanlator_group.py:docstring of hondana.scanlator_group.ScanlatorGroup.update:: WARNING: py:class reference target not found: LanguageCode

Previously I had autodoc_typehints = "none", but when changing this behaviour suddenly it cannot find these names/targets.

NOTE: I use the following command to generate my docs, hence being able to see all warnings as errors:-
sphinx-build -a -E -n -T -W --keep-going docs/ docs/_build

How to Reproduce

$ git clone -b docs/fixup https://github.com/AbstractUmbra/Hondana
$ cd Hondana
$ poetry install
$ sphinx-build -a -E -n -T -W --keep-going docs/ docs/_build
$ # see the resulting errors in console, as well as the generated html pages having text but no resolved items/links.

Expected behavior

I expected the docs to build correctly with the resolved types and reference links.

Your project

N/A I think

Screenshots

Code_StPl3JdtDP

OS

Linux - Debian 10

Python version

3.9.6

Sphinx version

4.5.0

Sphinx extensions

sphinx.ext.autodoc, sphinx.ext.extlinks, sphinx.ext.intersphinx, sphinx.ext.napoleon, sphinxcontrib_trio, resource_links

Extra tools

No response

Additional context

No response

@tk0miya
Copy link
Member

tk0miya commented Mar 31, 2022

It seems some objects are described manually via class directive. For example, hondana.enums.ReadingStatus is described as following:

.. currentmdoule:: hondana

(snip)

.. class:: ReadingStatus

   Specifies the current reading status for this manga

But autodoc generates a reference hondana.enums.ReadingStatus instead of hondana.ReadingStatus because autodoc generates document from real python objects.

To avoid such mismatches, you can use :canonical: option to this declaration:

.. currentmdoule:: hondana

(snip)

.. class:: ReadingStatus
   :canonical: hondana.enums.ReadingStatus

   Specifies the current reading status for this manga

This describe ReadingStatus class is described as hondana.ReadingStatus in document, and its canonical name is hondana.enums.ReadingStatus. This helps to resolve cross references from autodoc.

@tk0miya
Copy link
Member

tk0miya commented Mar 31, 2022

Note: Dockerfile to reproduce the warnings

FROM python:3.9-slim

RUN apt update; apt install -y build-essential curl git unzip vim
RUN git clone -b docs/fixup https://github.com/AbstractUmbra/Hondana
WORKDIR Hondana
RUN pip install poetry
RUN poetry install -E docs
RUN poetry run sphinx-build -a -E -n -T docs/ docs/_build

@AbstractUmbra
Copy link
Author

AbstractUmbra commented Mar 31, 2022

Wow, thank you for this, I appreciate it a lot.

Can I be a bother once more and query something else. If you have a look at this patch:

iff --git a/hondana/client.py b/hondana/client.py
index f6e8782..4d59373 100644
--- a/hondana/client.py
+++ b/hondana/client.py
@@ -89,7 +89,7 @@ from .utils import MISSING, require_authentication
 
 
 if TYPE_CHECKING:
-    import aiohttp
+    from aiohttp import ClientSession
 
     from .tags import QueryTags
     from .types import common, legacy, manga
@@ -141,7 +141,7 @@ class Client:
         username: None = ...,
         email: None = ...,
         password: None = ...,
-        session: Optional[aiohttp.ClientSession] = ...,
+        session: Optional[ClientSession] = ...,
         refresh_token: None = ...,
     ) -> None:
         ...
... truncated for brevity, but it's just more overloads being edited

This then generates the following error:

Poe => poetry run sphinx-build -a -E -n -T -W --keep-going docs/ docs/_build
Running Sphinx v4.5.0
loading intersphinx inventory from https://docs.python.org/3/objects.inv...
loading intersphinx inventory from https://docs.aiohttp.org/en/stable/objects.inv...
building [mo]: all of 0 po files
building [html]: all source files
updating environment: [new config] 5 added, 0 changed, 0 removed
reading sources... [100%] types                                                                                                                                                                                                                                                                                                                 
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [100%] types                                                                                                                                                                                                                                                                                                                  
/home/penumbra/projects/personal/hondana/hondana/client.py:docstring of hondana.client.Client:: WARNING: py:class reference target not found: ClientSession
/home/penumbra/projects/personal/hondana/hondana/client.py:docstring of hondana.client.Client:: WARNING: py:class reference target not found: ClientSession
/home/penumbra/projects/personal/hondana/hondana/client.py:docstring of hondana.client.Client:: WARNING: py:class reference target not found: ClientSession
/home/penumbra/projects/personal/hondana/hondana/client.py:docstring of hondana.client.Client:: WARNING: py:class reference target not found: ClientSession
/home/penumbra/projects/personal/hondana/hondana/client.py:docstring of hondana.client.Client:: WARNING: py:class reference target not found: ClientSession
...

As this is a third party dependency I cannot edit how this reference is found.
The "fix" I found was to import the whole module and use the whole path but this is not really ideal. Is there a way I can resolve this, or?

I have pushed the changes you suggested to the same branch but am still having some issues with more types. Could I bother you for some more guidance on those too?

@AbstractUmbra
Copy link
Author

Is there no way to add the :canonical: directive to :autoclass: objects?
I have these definitions and it seems sphinx does not like these paths for the autodoc_typehints:
Code_J9pgTG4s9s
I can see that they are documented just fine, but no matter what I try I cannot stop the errors in the console go away.

The file in question is here.

@tk0miya
Copy link
Member

tk0miya commented Apr 16, 2022

Thank you for your explanation. I understand what is your problem.

It seems hondana/client.py uses "Postponed Evaluation of Annotations" technique defined in PEP-563 (a.k.a __future__ annotations). It prevents autodoc to process typehints correctly because autodoc can't investigate where the type defined.

For example, some code refers hondana.types.LanguageCode as common.LanguageCode.
https://github.com/AbstractUmbra/Hondana/blob/acdd5b31c0ce8876bf3b144dab71cc81cc85f6ac/hondana/client.py#L426-L428

But autodoc can't know its real object name because it's not evaluated yet (postponed). As a result, autodoc failed to refer it correctly.

As a workaround, autodoc provides an option autodoc_type_aliases to give a hint to resolve this problem.
https://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html#confval-autodoc_type_aliases

You'll avoid the warnings via this setting:

autodoc_type_aliases = {
  'common.LanguageCode': 'hondana.types.LanguageCode',
}

Additionally, I found a bug of autodoc. autodoc_type_alias can't work with Optional type. So it can't resolve an annotation like Optional[ClientSession]. It will be fixed by #10353.

@tk0miya tk0miya added this to the 5.0.0 milestone Apr 16, 2022
@AbstractUmbra
Copy link
Author

Thank you so much for the explanation here! It is much appreciated.
I have one final question however, when using autodoc_typehints = "both" I can see that the build documentation will reveal all possible overload types. Is there a setting or so that can change or even disable this behaviour?

I have one method with 15 or so overloads, it does not look too nice, hah!

@tk0miya
Copy link
Member

tk0miya commented Apr 17, 2022

Sorry, I'm not sure what you're saying. Is the behavior related to autodoc_typehints option?

@AbstractUmbra
Copy link
Author

Sorry, yes this feature is only present when this option is enabled:

With autodoc_typehints = "both":
Code_7hPM5g2RLm
This seemingly is due to the typing.overload's I have on this method:
Code_KoMEw2Kjxs

With autodoc_typehints = "none" it presents as:
Code_M735IdADkr

@tk0miya
Copy link
Member

tk0miya commented Apr 17, 2022

Thank you for your explanation. I understand. You'll also see the overloads when autodoc_typehints = 'signature' (default).

Is there a setting or so that can change or even disable this behaviour?

Nothing. Please post a feature request as another issue.

@tk0miya tk0miya closed this as completed in a66adb1 May 2, 2022
tk0miya added a commit that referenced this issue May 2, 2022
…tations

Fix #10305: autodoc: Failed to extract optional forwardrefs
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jun 2, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

2 participants