[ENH] performance optimization for _check_soft_dependencies, speed up test collection time

[fkiraly]

This PR optimizes the runtime of _check_soft_dependencies, speeding up test collection time, towards #6344.

The speed-up comes from using importlib find_spec instead of actually attempting an import and catching the exception.

The "suppress std out" argument becomes obsolete, as no output is generated when not importing the module.

This PR also fixes an unreported bug in the sktime code base, where the package (PEP 440) scikit-learn was referred to as sklearn.

[fkiraly]

@benHeid, @yarnabrina, I have no reasonable explanation on the discrepancies between package versions found.

The discrepancy seems to be between version("packagename") (with version from packaging) and from packagename import __version__ - why and how would that differ?

[yarnabrina]

@benHeid, @yarnabrina, I have no reasonable explanation on the discrepancies between package versions found.

The discrepancy seems to be between version("packagename") (with version from packaging) and from packagename import __version__ - why and how would that differ?

I think I can explain a situation where this will happen. Whether this is happening here or not that we need to check.

Suppose in pyproject.toml someone mentioned version="1.2.3". And in the top level __init__.py one mentions __version__="4.5.6". That will lead to discrepancy. There is no verification of the same during packaging. It is perfectly acceptable if a package do not even define __version__, it's just a convention.

[yarnabrina]

Another question, as I probably misunderstood you:

>>> from packaging import version
>>> version("numpy")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'module' object is not callable

Are you doing something like this? Why is it failing for me but working for you?

[fkiraly]

Ah, sorry, I meant from importlib.metadata import version

[yarnabrina]

@fkiraly here key is sklearn, as that's what mentioned here:

sktime/sktime/utils/_maint/_show_versions.py

Line 40 in 8bd40f1

"sklearn",

However, _check_soft_dependencies expect scikit-learn as the input, not sklearn.

[fkiraly]

Yes, I think that should be ok, because _get_deps_info uses the import path, not the environment specs.

[yarnabrina]

But this is causing key_is_available to be False in case of scikit-learn, do we want that? Is it not expected to be True for installed packages?

What is your expected value for this?

_check_soft_dependencies("scikit-learn", severity="none", package_import_alias={'scikit-learn':"sklearn"})

On main, it is True, on this branch it is False.

[fkiraly]

oh, I see! That's the mistake.

[fkiraly]

attempted fix