Support for relative base_path in markdown_extensions

[dersimn]

Many extensions need to know the base path of the currently rendered Markdown file, as they are supposed to include assets relative to the markdown file. Two of them are:

• markdown-include
• pymdownx.snippets

Both make use of the config variable base_path for including other files, so in order to configure it, you would specify:

markdown_extensions:
  - pymdownx.snippets:
      base_path: docs
  - markdown_include.include:
      base_path: docs

Currently

Please take a look at this example repository.

|-- README.md
|-- docs
|   |-- big_topic
|   |   |-- index.md
|   |   `-- more_important_code.c
|   |-- index.md
|   `-- smaller_topic
|       |-- hello_world.c
|       `-- index.md
`-- mkdocs.yml

Currently you would have to include the *.c files with their full path, so in both files you would have to write:

    --8<-- "smaller_topic/hello_world.c"

    --8<-- "big_topic/more_important_code.c"

Why is this bad?

This is fine for Markdown files that are created and maintained for just one website, but we are currently including generic sub topics into our site that are shared among many repositories via git submodule. For e.g. some introduction topic on how to set up Docker. It's obvious that include paths must be relative to the Markdown file in this case, like:

    --8<-- "hello_world.c"

    --8<-- "more_important_code.c"

Desirable solution

If we want to be able to specify relative paths in every Markdown file, the base_path variable need to change with every Markdown file to be rendered, so it can't simply be included in mkdocs.yml.
This makes my feature request different from #998 #777 #761, because I'm not able to alter the .md files as they are shared over multiple projects.

It would be nice if we could specify something like

markdown_extensions:
  - pymdownx.snippets:
      base_path: $relative
  - markdown_include.include:
      base_path: $relative
  - one of many other extensions that use exacly 'base_path' as variable:
      base_path: $relative

that would then make MkDocs add the currently valid path as base_path variable for each rendered Markdown file.

This is an issue for the one who calls the plugin, and not the plugin itself, as an additional (dynamic) base_path must be exchanged at this point in code:

md = markdown.Markdown(
    extensions=extensions,
    extension_configs=config['mdx_configs'] or {}
)

Currently (with the upper config from my example), extension_configs would contain something in the structure of this:

extension_configs = {
    'pymdownx.snippets': {
        'base_path': '$relative'
    },
    'markdown_include.include': {
        'base_path': '$relative'
    }
}

We need to replace $relative here with the proper path depending on the current navigation.

[ahojukka5]

I just first time tried pymdownx.snippets and yeah, indeed, this would be an awesome feature. I'm still structuring my document tree and probably I have to change all those paths to snippets yet so many times...

[dersimn]

Patched version of mkdocs: https://github.com/dersimn/mkdocs
..the change is quite small by the way: https://github.com/dersimn/mkdocs/compare/775d506c63d4b57553110ee56b8c1f94d336e1f7..HEAD#diff-81f285125d041f8ec1fca83e7d0a170b496aea52e8f288edf3fd9a4d5773bfa4
Demo repository + instructions: https://github.com/dersimn/mkdocs-snippets-issue

[jorgegomezcq]

Hi, what's the status on this?

[pawamoy]

I don't think there was any progress. But I didn't know about this issue and I opened this discussion some time ago: #2833

[pawamoy]

I actually very much like the idea of variables provided by MkDocs: $docs_dir, $config_dir, and whatever else is relevant. I don't care about the syntax: there might be a way to use env vars instead, if MkDocs itself exports them:

markdown_extensions:
- pymdownx.snippets:
    base_path: !ENV MKDOCS_CONFIG_DIR

Depending on the order of things, env vars might be impossible to use. This might need a post-process step where $config_dir and others are replaced with their values.

[oprypin]

I think I'd like to make it work like this:

markdown_extensions:
  - pymdownx.snippets:
      base_path:
        - !relative  # Relative to current.md
        - !docs_dir  # Relative to docs_dir
        - !config_dir  # Relative to mkdocs.yml

The issue is that indeed now we need some way to dig through the parsed YAML and find and replace the placeholders. There doesn't seem to be any standard way for that.

Alternatively for a solution that's much much cleaner on MkDocs side but worse on the extension side, the value would actually be dynamically computed, but then we'd require each destination extension to call str(value) on anything that expects these dynamic values.

Otherwise of course places like this one get an error
https://github.com/facelessuser/pymdown-extensions/blob/67b084e6d14cbfa0c6c457e14aeb0e007f67a91a/pymdownx/snippets.py#L161
TypeError: stat: path should be string, bytes, os.PathLike or integer, not _YamlPlaceholder

[oprypin]

That gave me an idea- the value will respond not only to __str__ but also to __fspath__, totally avoiding at least this class of errors. So this works seamlessly with current pymdownx.snippets with the example above.

Linked a pull request accordingly.

cc @facelessuser

[facelessuser]

@oprypin does this allow things like !docs_dir/some/child/dir?

[oprypin]

No.
That's not a valid YAML tag syntax and I don't have good ideas how to make this work in a different way.

[oprypin]

Hm actually... I suppose !docs_dir some/child/dir is valid YAML syntax

[oprypin]

Maybe this idea would be better:

        - !relative  # Relative to current.md
        - !relative $docs_dir  # Relative to docs_dir
        - !relative $docs_dir/some/child/dir
        - !relative $config_dir  # Relative to mkdocs.yml

[pawamoy]

I find "relative" a bit confusing, but cannot find a better alternative. "path" would be too vague. "relpath" maybe too technical. In fact, relative-ness is not in question here since we use a variable ($docs_dir, $config_dir) which could return an absolute path for all we know. configpath? specialpath? dynamicpath? buildpath? docspath? mkdocspath?

[oprypin]

Any other ideas? ¯\_(ツ)_/¯

[pawamoy]

Nope, just go with what you prefer :)

[oprypin]

The other thing that's bugging me is this arbitrary $ symbol that has no precedent in MkDocs configs so far. There would be some precedent for making it like !relative {docs_dir} but that's more confusing.