Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: proper error messages if autosummary fails to import module #9555

Closed
wants to merge 8 commits into from
Closed
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
24 changes: 15 additions & 9 deletions sphinx/ext/autosummary/__init__.py
Expand Up @@ -614,9 +614,11 @@ def get_import_prefixes_from_env(env: BuildEnvironment) -> List[str]:
return prefixes


def import_by_name(name: str, prefixes: List[str] = [None]) -> Tuple[str, Any, Any, str]:
def import_by_name(name: str, prefixes: List[str] = [None], last_errors = None) -> Tuple[str, Any, Any, str]:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel this change is strange because last_errors should be a part of the return value. I understand using argument gives us compatibility. But...

BTW, it should be annotated.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also thought the same, but that would probably break backwards compatibility? Or should I change last_errors to be a retun value?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have two ideas:

  1. Add another version of import_by_name that also returns list of errors. It will be named as different name.
  2. Add a boolean argument like raise_error to import_by_name. And it raises a custom exception that contains list of errors.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, raise_error would not work that well since we need to collect all errors during the various import methods there are.

Having another version of the import_by_name is IMHO also not such an ideal solution, since then we would need to duplicate code, not only for the import_by_name but also for the sub-methods such as _import_by_name which is called inside of import_by_name

"""Import a Python object that has the given *name*, under one of the
*prefixes*. The first name that succeeds is used.
If importing fails, *last_errors* will contain the exceptions raised during
import and may help to identify the problem.
"""
tried = []
for prefix in prefixes:
Expand All @@ -625,14 +627,14 @@ def import_by_name(name: str, prefixes: List[str] = [None]) -> Tuple[str, Any, A
prefixed_name = '.'.join([prefix, name])
else:
prefixed_name = name
obj, parent, modname = _import_by_name(prefixed_name)
obj, parent, modname = _import_by_name(prefixed_name, last_errors)
return prefixed_name, obj, parent, modname
except ImportError:
tried.append(prefixed_name)
raise ImportError('no module named %s' % ' or '.join(tried))


def _import_by_name(name: str) -> Tuple[Any, Any, str]:
def _import_by_name(name: str, last_errors = None) -> Tuple[Any, Any, str]:
"""Import a Python object given its full name."""
try:
name_parts = name.split('.')
Expand All @@ -643,8 +645,9 @@ def _import_by_name(name: str) -> Tuple[Any, Any, str]:
try:
mod = import_module(modname)
return getattr(mod, name_parts[-1]), mod, modname
except (ImportError, IndexError, AttributeError):
pass
except (ImportError, IndexError, AttributeError) as e:
if last_errors is not None:
last_errors.append(e)

# ... then as MODNAME, MODNAME.OBJ1, MODNAME.OBJ1.OBJ2, ...
last_j = 0
Expand All @@ -654,7 +657,9 @@ def _import_by_name(name: str) -> Tuple[Any, Any, str]:
modname = '.'.join(name_parts[:j])
try:
import_module(modname)
except ImportError:
except ImportError as e:
if last_errors is not None:
last_errors.append(e)
continue

if modname in sys.modules:
Expand All @@ -673,7 +678,7 @@ def _import_by_name(name: str) -> Tuple[Any, Any, str]:
raise ImportError(*e.args) from e


def import_ivar_by_name(name: str, prefixes: List[str] = [None]) -> Tuple[str, Any, Any, str]:
def import_ivar_by_name(name: str, prefixes: List[str] = [None], last_errors = None) -> Tuple[str, Any, Any, str]:
"""Import an instance variable that has the given *name*, under one of the
*prefixes*. The first name that succeeds is used.
"""
Expand All @@ -686,8 +691,9 @@ def import_ivar_by_name(name: str, prefixes: List[str] = [None]) -> Tuple[str, A
# check for presence in `annotations` to include dataclass attributes
if (qualname, attr) in analyzer.attr_docs or (qualname, attr) in analyzer.annotations:
return real_name + "." + attr, INSTANCEATTR, obj, modname
except (ImportError, ValueError, PycodeError):
pass
except (ImportError, ValueError, PycodeError) as e:
if last_errors is not None:
last_errors.append(e)

raise ImportError

Expand Down
14 changes: 10 additions & 4 deletions sphinx/ext/autosummary/generate.py
Expand Up @@ -413,16 +413,22 @@ def generate_autosummary_docs(sources: List[str], output_dir: str = None,
path = output_dir or os.path.abspath(entry.path)
ensuredir(path)

last_errors = []

try:
name, obj, parent, modname = import_by_name(entry.name)
name, obj, parent, modname = import_by_name(entry.name, last_errors=last_errors)
qualname = name.replace(modname + ".", "")
except ImportError as e:
try:
# try to importl as an instance attribute
name, obj, parent, modname = import_ivar_by_name(entry.name)
# try to import as an instance attribute
name, obj, parent, modname = import_ivar_by_name(entry.name, last_errors=last_errors)
qualname = name.replace(modname + ".", "")
except ImportError:
logger.warning(__('[autosummary] failed to import %r: %s') % (entry.name, e))
# Convert exceptions to strings and remove duplicates (covert to set, then to list)
Pro marked this conversation as resolved.
Show resolved Hide resolved
last_errors_str = list([str(e) for e in last_errors])
last_errors_str = list(set(last_errors_str))
Pro marked this conversation as resolved.
Show resolved Hide resolved
logger.warning(__('[autosummary] failed to import %r: %s. Possible hints: %s') %
(entry.name, e, last_errors_str))
continue

context: Dict[str, Any] = {}
Expand Down