Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: executablebooks/MyST-Parser
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: v0.17.2
Choose a base ref
...
head repository: executablebooks/MyST-Parser
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: v0.18.0
Choose a head ref
  • 7 commits
  • 141 files changed
  • 2 contributors

Commits on Apr 20, 2022

  1. 👌 IMPROVE: Do not let sphinx check the config type (#559)

    This is already validated by myst-parser
    chrisjsewell authored Apr 20, 2022
    Copy the full SHA
    7dda7b5 View commit details

Commits on May 12, 2022

  1. ♻️📚 Restructure code base and documentation (#566)

    This PR restructures the code base into modules, to be more coherent as to the purpose of each module.
    
    It also restructures the documentation, to make it easier for users to follow, and expose the core concerns at the top-level.
    
    Finally, it introduces document-level configuration *via* the Markdown top-matter, under the `myst` key.
    This configuration is generated from the code `MdParserConfig` dataclass, so is inherently consistent with global configuration.
    The (YAML) top-matter of a MyST file is always read (within the docutils/sphinx parsers) before the full document is parsed, in order to acquire this file configuration, which overrides the default/global configuration (see `docs/configuration.md`).
    
    ## Breaking changes
    
    This should not be breaking, for general users of the sphinx extension,
    but will be for anyone directly using the Python API, mainly just requiring changes in import module paths.
    
    The `to_docutils`, `to_html`, `to_tokens` (from `myst_parser/main.py`) and `mock_sphinx_env`/`parse` (from `myst_parser.sphinx_renderer.py`) functions have been removed.
    These were really just for testing, and were confusing for users, since they did not run the full sphinx build.
    Instead, for single page builds, users should use the recently added docutils parser API/CLI (see `docs/docutils.md`),
    and for testing, functionality has been moved to https://github.com/chrisjsewell/sphinx-pytest.
     
    The top-level `html_meta` and `substitutions` top-matter keys have also been deprecated (i.e. they will still work but will emit a warning), as they now form part of the `myst` config, e.g.
    
    ```yaml
    ---
    html_meta:
      "description lang=en": "metadata description"
    substitutions:
      key1: I'm a **substitution**
    ---
    ```
    
    is replaced by:
    
    ```yaml
    ---
    myst:
      html_meta:
        "description lang=en": "metadata description"
      substitutions:
        key1: I'm a **substitution**
    ---
    ```
    chrisjsewell authored May 12, 2022
    Copy the full SHA
    602470e View commit details

Commits on May 15, 2022

  1. Copy the full SHA
    3c45b7e View commit details

Commits on Jun 7, 2022

  1. 2
    Copy the full SHA
    2e91dd8 View commit details
  2. Copy the full SHA
    cc44a35 View commit details
  3. Copy the full SHA
    c17d855 View commit details
  4. Copy the full SHA
    75ef9cb View commit details
Showing with 3,678 additions and 6,586 deletions.
  1. +2 −2 .github/workflows/docutils_setup.py
  2. +9 −10 .github/workflows/tests.yml
  3. +8 −2 .pre-commit-config.yaml
  4. +1 −1 .readthedocs.yml
  5. +48 −0 CHANGELOG.md
  6. +20 −7 docs/_static/custom.css
  7. +0 −14 docs/api/index.md
  8. +0 −301 docs/api/parsers.md
  9. +43 −57 docs/api/reference.rst
  10. +0 −146 docs/api/renderers.md
  11. +28 −68 docs/conf.py
  12. +106 −0 docs/configuration.md
  13. +1 −1 docs/{explain/index.md → develop/background.md}
  14. +1 −1 docs/develop/index.md
  15. +1 −1 docs/docutils.md
  16. +0 −5 docs/examples/heavy_tails.md
  17. BIN docs/examples/htop_again.png
  18. +0 −32 docs/examples/index.md
  19. +0 −4 docs/examples/kesten_processes.md
  20. +0 −12 docs/examples/references.bib
  21. +0 −482 docs/examples/wealth_dynamics_md.md
  22. +0 −493 docs/examples/wealth_dynamics_rst.rst
  23. +51 −14 docs/{sphinx/use.md → faq/index.md}
  24. 0 docs/{sphinx → faq}/snippets/include-md.md
  25. 0 docs/{sphinx → faq}/snippets/include-rst.rst
  26. +101 −80 docs/index.md
  27. +250 −0 docs/intro.md
  28. +0 −16 docs/sphinx/faq.md
  29. +0 −17 docs/sphinx/index.md
  30. +0 −423 docs/sphinx/intro.md
  31. +0 −129 docs/sphinx/reference.md
  32. +0 −23 docs/sphinx/roles-and-directives.md
  33. +41 −35 docs/syntax/optional.md
  34. +2 −5 docs/syntax/reference.md
  35. +421 −0 docs/syntax/roles-and-directives.md
  36. +59 −393 docs/syntax/syntax.md
  37. +1 −1 example-include.md
  38. +4 −61 myst_parser/__init__.py
  39. +11 −0 myst_parser/_compat.py
  40. +198 −0 myst_parser/_docs.py
  41. +2 −1 myst_parser/cli.py
  42. +1 −0 myst_parser/config/__init__.py
  43. +46 −27 myst_parser/{ → config}/dc_validators.py
  44. +408 −0 myst_parser/config/main.py
  45. +1 −263 myst_parser/docutils_.py
  46. +0 −442 myst_parser/main.py
  47. +1 −0 myst_parser/mdit_to_docutils/__init__.py
  48. +127 −126 myst_parser/{docutils_renderer.py → mdit_to_docutils/base.py}
  49. +8 −6 myst_parser/{ → mdit_to_docutils}/html_to_nodes.py
  50. +45 −138 myst_parser/{sphinx_renderer.py → mdit_to_docutils/sphinx_.py}
  51. 0 myst_parser/{ → mdit_to_docutils}/utils.py
  52. +23 −21 myst_parser/mocking.py
  53. +1 −0 myst_parser/parsers/__init__.py
  54. +11 −14 myst_parser/{parse_directives.py → parsers/directives.py}
  55. +275 −0 myst_parser/parsers/docutils_.py
  56. +120 −0 myst_parser/parsers/mdit.py
  57. +29 −29 myst_parser/{ → parsers}/parse_html.py
  58. +69 −0 myst_parser/parsers/sphinx_.py
  59. +1 −1 myst_parser/sphinx_.py
  60. +1 −0 myst_parser/sphinx_ext/__init__.py
  61. +1 −1 myst_parser/{ → sphinx_ext}/directives.py
  62. +60 −0 myst_parser/sphinx_ext/main.py
  63. +3 −3 myst_parser/{ → sphinx_ext}/mathjax.py
  64. +7 −1 myst_parser/{ → sphinx_ext}/myst_refs.py
  65. +0 −80 myst_parser/sphinx_parser.py
  66. +13 −11 pyproject.toml
  67. +1 −0 test.rst
  68. +0 −8 tests/conftest.py
  69. +5 −2 tests/test_commonmark/test_commonmark.py
  70. +2 −2 tests/test_docutils.py
  71. +2 −2 tests/test_html/test_html_to_nodes.py
  72. +1 −1 tests/test_html/test_parse_html.py
  73. +7 −11 tests/test_renderers/fixtures/amsmath.md
  74. +3 −5 tests/test_renderers/fixtures/containers.md
  75. +1 −2 tests/test_renderers/fixtures/definition_lists.md
  76. +23 −34 tests/test_renderers/fixtures/directive_options.md
  77. +141 −0 tests/test_renderers/fixtures/directive_parsing.txt
  78. +8 −53 tests/test_renderers/fixtures/docutil_syntax_elements.md
  79. +8 −14 tests/test_renderers/fixtures/dollarmath.md
  80. +2 −2 tests/test_renderers/fixtures/eval_rst.md
  81. +29 −10 tests/test_renderers/fixtures/reporter_warnings.md
  82. +44 −80 tests/test_renderers/fixtures/sphinx_directives.md
  83. +102 −160 tests/test_renderers/fixtures/sphinx_roles.md
  84. +59 −104 tests/test_renderers/fixtures/sphinx_syntax_elements.md
  85. +17 −22 tests/test_renderers/fixtures/tables.md
  86. +1 −1 tests/test_renderers/test_error_reporting.py
  87. +45 −20 tests/test_renderers/test_fixtures_docutils.py
  88. +57 −62 tests/test_renderers/test_fixtures_sphinx.py
  89. +30 −22 tests/test_renderers/test_include_directive.py
  90. +1 −1 tests/test_renderers/test_myst_config.py
  91. +18 −26 tests/test_renderers/test_myst_refs.py
  92. +8 −6 tests/test_renderers/test_myst_refs/duplicate.xml
  93. +1 −2 tests/test_renderers/test_myst_refs/missing.xml
  94. +8 −7 tests/test_renderers/test_myst_refs/ref.xml
  95. +8 −7 tests/test_renderers/test_myst_refs/ref_colon.xml
  96. +9 −8 tests/test_renderers/test_myst_refs/ref_nested.xml
  97. +25 −11 tests/test_renderers/test_parse_directives.py
  98. +0 −7 tests/test_renderers/test_parse_directives/test_parsing_Note___class__name_n_na_.yml
  99. +0 −5 tests/test_renderers/test_parse_directives/test_parsing_Note__a_.yml
  100. +0 −5 tests/test_renderers/test_parse_directives/test_parsing_Note_a__.yml
  101. +4 −2 tests/test_sphinx/conftest.py
  102. +19 −18 tests/test_sphinx/sourcedirs/substitutions/index.md
  103. +2 −0 tests/test_sphinx/sourcedirs/texi_table/conf.py
  104. +3 −0 tests/test_sphinx/sourcedirs/texi_table/index.md
  105. +39 −27 tests/test_sphinx/test_sphinx_builds.py
  106. +0 −154 tests/test_sphinx/test_sphinx_builds/test_basic.resolved.sphinx3.xml
  107. +2 −2 tests/test_sphinx/test_sphinx_builds/{test_basic.resolved.sphinx4.xml → test_basic.resolved.xml}
  108. +0 −155 tests/test_sphinx/test_sphinx_builds/test_basic.sphinx3.xml
  109. +19 −17 tests/test_sphinx/test_sphinx_builds/{test_basic.sphinx3.html → test_basic.sphinx5.html}
  110. +2 −2 tests/test_sphinx/test_sphinx_builds/{test_basic.sphinx4.xml → test_basic.xml}
  111. +1 −1 tests/test_sphinx/test_sphinx_builds/{test_commonmark_only.sphinx4.html → test_commonmark_only.html}
  112. +0 −28 tests/test_sphinx/test_sphinx_builds/test_commonmark_only.sphinx3.html
  113. +1 −1 ...est_sphinx/test_sphinx_builds/{test_extended_syntaxes.sphinx4.html → test_extended_syntaxes.html}
  114. +0 −173 tests/test_sphinx/test_sphinx_builds/test_extended_syntaxes.sphinx3.html
  115. +0 −90 tests/test_sphinx/test_sphinx_builds/test_extended_syntaxes.sphinx3.xml
  116. 0 .../test_sphinx/test_sphinx_builds/{test_extended_syntaxes.sphinx4.xml → test_extended_syntaxes.xml}
  117. +0 −82 tests/test_sphinx/test_sphinx_builds/test_fieldlist_extension.sphinx3.xml
  118. +24 −6 ...est_sphinx_builds/{test_fieldlist_extension.sphinx3.html → test_fieldlist_extension.sphinx5.html}
  119. 0 ...t_sphinx/test_sphinx_builds/{test_fieldlist_extension.sphinx4.xml → test_fieldlist_extension.xml}
  120. +0 −147 tests/test_sphinx/test_sphinx_builds/test_footnotes.sphinx3.html
  121. +211 −0 tests/test_sphinx/test_sphinx_builds/test_footnotes.sphinx5.html
  122. 0 tests/test_sphinx/test_sphinx_builds/{test_gettext.sphinx4.pot → test_gettext.pot}
  123. +0 −69 tests/test_sphinx/test_sphinx_builds/test_gettext.sphinx3.pot
  124. 0 ...sphinx_builds/{test_gettext_additional_targets.sphinx4.pot → test_gettext_additional_targets.pot}
  125. +0 −126 tests/test_sphinx/test_sphinx_builds/test_gettext_additional_targets.sphinx3.pot
  126. +0 −93 tests/test_sphinx/test_sphinx_builds/test_gettext_html.resolved.sphinx3.xml
  127. +1 −1 ...hinx/test_sphinx_builds/{test_gettext_html.resolved.sphinx4.xml → test_gettext_html.resolved.xml}
  128. +0 −93 tests/test_sphinx/test_sphinx_builds/test_gettext_html.sphinx3.xml
  129. +8 −8 .../test_sphinx/test_sphinx_builds/{test_gettext_html.sphinx3.html → test_gettext_html.sphinx5.html}
  130. +1 −1 tests/test_sphinx/test_sphinx_builds/{test_gettext_html.sphinx4.xml → test_gettext_html.xml}
  131. +2 −2 ...est_sphinx/test_sphinx_builds/{test_heading_slug_func.sphinx4.html → test_heading_slug_func.html}
  132. +0 −22 tests/test_sphinx/test_sphinx_builds/test_heading_slug_func.sphinx3.html
  133. +4 −4 tests/test_sphinx/test_sphinx_builds/{test_includes.sphinx4.html → test_includes.html}
  134. +0 −131 tests/test_sphinx/test_sphinx_builds/test_includes.sphinx3.html
  135. +0 −113 tests/test_sphinx/test_sphinx_builds/test_includes.sphinx3.xml
  136. 0 tests/test_sphinx/test_sphinx_builds/{test_includes.sphinx4.xml → test_includes.xml}
  137. +2 −2 tests/test_sphinx/test_sphinx_builds/{test_references.sphinx4.html → test_references.html}
  138. +0 −195 tests/test_sphinx/test_sphinx_builds/test_references.sphinx3.html
  139. +4 −4 ...nx/test_sphinx_builds/{test_references_singlehtml.sphinx4.html → test_references_singlehtml.html}
  140. +0 −111 tests/test_sphinx/test_sphinx_builds/test_references_singlehtml.sphinx3.html
  141. +3 −4 tox.ini
4 changes: 2 additions & 2 deletions .github/workflows/docutils_setup.py
Original file line number Diff line number Diff line change
@@ -44,12 +44,12 @@ def modify_readme(content: str) -> str:
if __name__ == "__main__":
project_path = sys.argv[1]
readme_path = sys.argv[2]
with open(project_path, "r") as f:
with open(project_path) as f:
content = f.read()
content = modify_toml(content)
with open(project_path, "w") as f:
f.write(content)
with open(readme_path, "r") as f:
with open(readme_path) as f:
content = f.read()
content = modify_readme(content)
with open(readme_path, "w") as f:
19 changes: 9 additions & 10 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
@@ -26,16 +26,15 @@ jobs:
fail-fast: false
matrix:
python-version: ["3.7", "3.8", "3.9", "3.10"]
sphinx: [">=4,<5"]
sphinx: [">=5,<6"]
os: [ubuntu-latest]
include:
# fails because of: https://github.com/sphinx-doc/sphinx/issues/10291
# - os: ubuntu-latest
# python-version: "3.8"
# sphinx: ">=3,<4"
- os: windows-latest
python-version: "3.8"
sphinx: ">=4,<5"
- os: ubuntu-latest
python-version: "3.8"
sphinx: ">=4,<5"
- os: windows-latest
python-version: "3.8"
sphinx: ">=4,<5"

runs-on: ${{ matrix.os }}

@@ -89,7 +88,7 @@ jobs:
- name: Install dependencies
run: |
pip install .
pip install pytest~=6.2 pytest-param-files~=0.3.3 docutils==${{ matrix.docutils-version }}
pip install pytest~=6.2 pytest-param-files~=0.3.3 pygments docutils==${{ matrix.docutils-version }}
- name: ensure sphinx is not installed
run: |
python -c "\
@@ -100,7 +99,7 @@ jobs:
else:
raise AssertionError()"
- name: Run pytest for docutils-only tests
run: pytest tests/test_docutils.py tests/test_renderers/test_fixtures_docutils.py
run: pytest tests/test_docutils.py tests/test_renderers/test_fixtures_docutils.py tests/test_renderers/test_include_directive.py
- name: Run docutils CLI
run: echo "test" | myst-docutils-html

10 changes: 8 additions & 2 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -19,6 +19,12 @@ repos:
- id: end-of-file-fixer
- id: trailing-whitespace

- repo: https://github.com/asottile/pyupgrade
rev: v2.34.0
hooks:
- id: pyupgrade
args: [--py37-plus]

- repo: https://github.com/PyCQA/isort
rev: 5.10.1
hooks:
@@ -39,12 +45,12 @@ repos:
# - flake8-self~=0.2.2

- repo: https://github.com/pre-commit/mirrors-mypy
rev: v0.942
rev: v0.961
hooks:
- id: mypy
args: [--config-file=pyproject.toml]
additional_dependencies:
- sphinx~=3.3
- sphinx~=5.0
- markdown-it-py>=1.0.0,<3.0.0
- mdit-py-plugins~=0.3.0
files: >
2 changes: 1 addition & 1 deletion .readthedocs.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
version: 2

python:
version: 3
version: "3"
install:
- method: pip
path: .
48 changes: 48 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,53 @@
# Changelog

## 0.18.0 - 2022-06-07

Full Changelog: [v0.17.2...v0.18.0](https://github.com/executablebooks/MyST-Parser/compare/v0.17.2...v0.18.0)

This release adds support for Sphinx v5 (dropping v3), restructures the code base into modules, and also restructures the documentation, to make it easier for developers/users to follow.

It also introduces **document-level configuration** *via* the Markdown top-matter, under the `myst` key.
See the [Local configuration](docs/configuration.md) section for more information.

### Breaking changes

This should not be breaking, for general users of the sphinx extension (with `sphinx>3`),
but will be for anyone directly using the Python API, mainly just requiring changes in import module paths.

The `to_docutils`, `to_html`, `to_tokens` (from `myst_parser/main.py`) and `mock_sphinx_env`/`parse` (from `myst_parser.sphinx_renderer.py`) functions have been removed, since these were primarily for internal testing.
Instead, for single page builds, users should use the docutils parser API/CLI (see [](docs/docutils.md)),
and for testing, functionality has been moved to <https://github.com/chrisjsewell/sphinx-pytest>.

The top-level `html_meta` and `substitutions` top-matter keys have also been deprecated (i.e. they will still work but will emit a warning), as they now form part of the `myst` config, e.g.

```yaml
---
html_meta:
"description lang=en": "metadata description"
substitutions:
key1: I'm a **substitution**
---
```

is replaced by:

```yaml
---
myst:
html_meta:
"description lang=en": "metadata description"
substitutions:
key1: I'm a **substitution**
---
```

### Key PRs

- ♻️📚 Restructure code base and documentation (#566)
- ⬆️ Drop Sphinx 3 and add Sphinx 5 support (#579)
- 🐛 FIX: `parse_directive_text` when body followed by options (#580)
- 🐛 FIX: floor table column widths to integers (#568), thanks to @Jean-Abou-Samra!

## 0.17.2 - 2022-04-17

Full Changelog: [v0.17.1...v0.17.2](https://github.com/executablebooks/MyST-Parser/compare/v0.17.1...v0.17.2)
27 changes: 20 additions & 7 deletions docs/_static/custom.css
Original file line number Diff line number Diff line change
@@ -1,11 +1,24 @@
.bg-myst-one {
background-color: #52d16f3b;
/** Add a counter before subsections **/
h1 {
counter-reset: subsection;
text-decoration: underline;
}

.bg-myst-two {
background-color: #e7dd7b73;
h2 {
counter-reset: subsubsection;
}
h2::before {
counter-increment: subsection;
content: counter(subsection) ". ";
}
h3::before {
counter-increment: subsubsection;
content: counter(subsection) "." counter(subsubsection) ". ";
}

.bg-myst-three {
background-color: #e7b07b96;
/** No icon for admonitions with no-icon class */
.admonition > .admonition-title, div.admonition.no-icon > .admonition-title::before {
content: "";
}
.admonition > .admonition-title, div.admonition.no-icon > .admonition-title {
padding-left: .6rem;
}
14 changes: 0 additions & 14 deletions docs/api/index.md

This file was deleted.

Loading