Add a separate experimentalOptions table

[Witiko]

Introduction

Since the release of Markdown v2.0.0, a large number of new options was added. However, to retain backwards compatibility, most of these options are disabled by default. This makes it difficult to enable everything back again for a user that is not concerned with backwards compatibility and wants to use the Markdown package with all the bells and whistles.

Proposed solution

Add a separate experimentalOptions table that will have the same structure as the current defaultOptions table, but that will enable the majority of new options and syntax extensions. Add a Lua option experimental that will make the Lua parser use the options in the experimentalOptions table as the defaults.

[Witiko]

After thinking more about this, the experimental option looks like a nightmare for someone who has to maintain legacy documents, since any new syntax extensions would potentially break the documents. It is also against the spirit of (La)TeX as a stable document format. It would be convenient, but perhaps too convenient to offset the long-term cost.

A more sensible solution would be to have an option such as enableEverythingUntil=2.8.1, which would enable all syntax extensions introduced before and in the specified version of the Markdown package. The use of enableEverything=\markdownVersion, which would produce behavior equivalent to the proposed experimental option, should be prohibited, explaining the rationale in the error message. This last provision can be circumvented by expanding \markdownVersion, but at least it discourages.

A similar solution would be to have an option such as enableEverythingUntil=2019-05-09, which would enable all syntax extensions introduced before a given date. I am less in favor of this solution, because a document typeset using a legacy version of Markdown will use fewer syntax extensions than a newer version of Markdown released before 2019-05-09. To put it another way, enableEverythingUntil=2.8.1 specifies precisely which syntax extensions will be used, whereas enableEverythingUntil=2019-05-09 is ambiguous.