apidoc: Double entries for cython modules (both .pyx and .so found)

[kalvdans]

Describe the bug

sphinx-apidoc adds both .pyx files and .so files to the generated .rst files. This causes duplication if both are in the same directory, or causes module lookup errors in sphinx-build if they are in different directories.

Support for .pyx was added in #1117 and .so in #2546

How to Reproduce

python3 -m venv env
env/bin/pip install cython sphinx
mkdir mypackage
echo "# cython: language_level = 3" > mypackage/mymodule.pyx
cat > setup.py <<EOF
from setuptools import setup, Extension
setup(ext_modules=[Extension("mypackage.mymodule", sources=["mypackage/mymodule.pyx"])])
EOF
env/bin/python setup.py build_ext --inplace
env/bin/sphinx-apidoc --implicit-namespaces -fT mypackage -o doc/
cat doc/mypackage.rst

You will see mymodule listed twice:

mypackage namespace
===================

.. py:module:: mypackage

Submodules
----------

mypackage.mymodule module
-------------------------

.. automodule:: mypackage.mymodule
   :members:
   :undoc-members:
   :show-inheritance:

mypackage.mymodule module
-------------------------

.. automodule:: mypackage.mymodule
   :members:
   :undoc-members:
   :show-inheritance:

Expected behavior

Expect mymodule to be listed once.

Your project

reproducer is standalone

Screenshots

No response

OS

Linux

Python version

3.8.10

Sphinx version

4.4.0

Sphinx extensions

No response

Extra tools

No response

Additional context

No response

[emlys]

I got around this by using the optional EXCLUDE_PATTERNS argument documented at the top of this page:
sphinx-apidoc -o doc/ src/ /*.so

but I agree it's a bug, duplicate entries should be prevented by default since they cause a WARNING: duplicated entry found in toctree.

What's not documented is that EXCLUDE_PATTERNS need to be absolute paths in order to match files outside of the OUTPUT_PATH directory.

sphinx/sphinx/ext/apidoc.py

Line 416 in 5eeeb9c

excludes = [path.abspath(exclude) for exclude in args.exclude_pattern]

For example, if my OUTPUT_PATH is /Users/emily/invest/doc/api-docs and my MODULE_PATH is /Users/emily/invest/lib.macosx-10.9-x86_64-cpython-39, the pattern '*.so' expands to '/Users/emily/invest/doc/api-docs/*.so' because OUTPUT_PATH is the working directory used by path.abspath. This pattern matches none of the .so files I'm trying to exclude in my MODULE_PATH. But '/*.so'doesn't expand, so it matches .so files anywhere, including those in my MODULE_PATH.