Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

How to enable markdown extensions? #158

Open
MPvHarmelen opened this issue Feb 7, 2020 · 5 comments · May be fixed by #440
Open

How to enable markdown extensions? #158

MPvHarmelen opened this issue Feb 7, 2020 · 5 comments · May be fixed by #440
Labels
enhancement New feature or request

Comments

@MPvHarmelen
Copy link

Expected Behavior

I am looking for some configuration switch to tell Python Markdown which extensions to use, specifically something like PyMdown Extensions to enable task-lists and strike-through.

Actual Behavior

The list of extensions seems to be hard-coded and not amendable using templates (or consequently command-line arguments).

See:

pdoc/pdoc/html_helpers.py

Line 69 in 1709915

_md = markdown.Markdown(

Steps to Reproduce

N.A.

Additional info

  • pdoc version: 0.7.4
@kernc
Copy link
Member

kernc commented Feb 7, 2020

It's non-amendable as nobody needed/proposed it yet. Seems reasonable. I think template tunables would allow for greater configurability (e.g. extension configs). A PR would be much appreciated. 👍

@kernc kernc added enhancement New feature or request help wanted Extra attention is needed labels Feb 7, 2020
@MPvHarmelen
Copy link
Author

I could spend some time on implementing this, but have no idea what you mean by "template tunables" and haven't worked with mako templates before. Could you point me in the right direction? Specifically I'm not quite sure in which of the templates this should be added.

I'm assuming you would want to move the whole list of extensions and config into some template, so users can also choose to leave out extensions, or are there any extensions that pdoc depends on, so must be kept in?

@kernc
Copy link
Member

kernc commented Feb 7, 2020

Great! Sorry, I meant config.mako template, which is consulted before rendering the real html/text/pdf templates.

users can also choose to leave out extensions, or are there any extensions that pdoc depends on

I'm not sure, best run the tests and see. toc is tested for sure. Re appending, I guess either way is fine, although the current defaults seem also quite fine, and I don't want to pollute user-facing config with pointless and never-to-be-customized internals. 🤔
Either way, with extensions defined in a template, you'll need a way to efficiently enable/cache them on the markdown converter instance, somewhat like the MathJax preprocessor is if latex_math config tunable is set:

pdoc/pdoc/html_helpers.py

Lines 356 to 365 in 1709915

# Optionally register our math syntax processor
if not latex_math and _MathPattern.NAME in _md.inlinePatterns:
_md.inlinePatterns.deregister(_MathPattern.NAME)
elif latex_math and _MathPattern.NAME not in _md.inlinePatterns:
_md.inlinePatterns.register(_MathPattern(_MathPattern.PATTERN),
_MathPattern.NAME,
_MathPattern.PRIORITY)
md = to_markdown(text, docformat=docformat, module=module, link=link)
return _md.reset().convert(md)

I'm afraid creating a new markdown converter instance for each piece of docstring will be too inefficient.

The alternative is a CLI switch --markdown-extension-configs taking JSON dict with its keys marking extra extensions to enable. I maybe like this simpler option ...

@MPvHarmelen
Copy link
Author

Okay, let's append things then. I think I like a template-based solution, as it keeps the functionality for CLI and programmatic usage the same.

Would handling it exactly the same as the LaTeX maths be an acceptable solution?
That would mean adding a list(-like?) variable to the config.mako template, propagating that to html_helpers.to_html in html.mako (like latex_math) and then registering the extensions exactly where the patterns for LaTeX maths are also registered.

pdoc/pdoc/templates/html.mako

Lines 17 to 18 in 1709915

def to_html(text):
return _to_html(text, module=module, link=link, latex_math=latex_math)

A problem with this solution is that you can't deregister an extension like you can deregister patterns, meaning that you can't use different extensions during the same run. Is this a problem? If so, the global Markdown instance must be made local in some way.

pdoc/pdoc/html_helpers.py

Lines 69 to 90 in 1709915

_md = markdown.Markdown(
output_format='html5',
extensions=[
"markdown.extensions.abbr",
"markdown.extensions.attr_list",
"markdown.extensions.def_list",
"markdown.extensions.fenced_code",
"markdown.extensions.footnotes",
"markdown.extensions.tables",
"markdown.extensions.admonition",
"markdown.extensions.smarty",
"markdown.extensions.toc",
],
extension_configs={
"markdown.extensions.smarty": dict(
smart_dashes=True,
smart_ellipses=True,
smart_quotes=False,
smart_angled_quotes=False,
),
},
)

Some stuff for (future) reference

Markdown.registerExtensions, to, well, register extensions...

https://github.com/Python-Markdown/markdown/blob/1de3083d82a5245015f0f0b9471a965916134d6d/markdown/core.py#L110-L120

@kernc
Copy link
Member

kernc commented Feb 10, 2020 •
edited

exactly where the patterns for LaTeX maths are also registered.

pdoc/pdoc/templates/html.mako

Lines 17 to 18 in 1709915

def to_html(text):
return _to_html(text, module=module, link=link, latex_math=latex_math)

I must say, this looks one hella ugly piece of hack, sending values back and forth like insane. I'm not sure that wasn't a mistake. I wonder if most preferences couldn't instead be applied earlier, like in _render_template() where the config is read? 🤔

pdoc/pdoc/__init__.py

Lines 93 to 121 in 1709915

def _render_template(template_name, **kwargs):
"""
Returns the Mako template with the given name. If the template
cannot be found, a nicer error message is displayed.
"""
# Apply config.mako configuration
MAKO_INTERNALS = Template('').module.__dict__.keys()
DEFAULT_CONFIG = path.join(path.dirname(__file__), 'templates', 'config.mako')
config = {}
for config_module in (Template(filename=DEFAULT_CONFIG).module,
tpl_lookup.get_template('/config.mako').module):
config.update((var, getattr(config_module, var, None))
for var in config_module.__dict__
if var not in MAKO_INTERNALS)
known_keys = set(config) | {'module', 'modules', 'http_server', 'external_links'} # deprecated
invalid_keys = {k: v for k, v in kwargs.items() if k not in known_keys}
if invalid_keys:
warn('Unknown configuration variables (not in config.mako): {}'.format(invalid_keys))
config.update(kwargs)
try:
t = tpl_lookup.get_template(template_name)
except TopLevelLookupException:
raise OSError(
"No template found at any of: {}".format(
', '.join(path.join(p, template_name.lstrip("/"))
for p in tpl_lookup.directories)))
try:
return t.render(**config).strip()

@peterjc peterjc mentioned this issue Feb 21, 2020
Closed
@kernc kernc removed the help wanted Extra attention is needed label Sep 9, 2020
@Jessseee Jessseee mentioned this issue Jul 20, 2023
Closed
@kernc kernc linked a pull request Mar 11, 2024 that will close this issue
Open
@thehappycheese thehappycheese mentioned this issue Mar 20, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

ENH: Support configurable markdown extensions kernc/pdoc
2 participants
@kernc @MPvHarmelen