[api-documenter] Emit HTML tags for tables instead of Markdown code.

[phryneas]

Summary

This would fix #2950.

Currently, tables in generated Markdown do not work with multiline comments.

This PR changes over to HTML tables (which also work in Markdown and are in fact more standardized).

As a result, a lot of "emit different things in tables" code can be removed.

Details

As an example of a broken generated table, see the end of the "Classes" table in this markdown file

How it was tested

Generating the code with this PR instead will result in this diff and as you can see here the tables are not broken anymore.

Impacted documentation

I believe this does not need any documentation changes.

[phryneas]

I believe I did update the test, so I'm not 100% sure why this is still failing.

Nevermind, forgot to build before updating the tests again.

[Lightning00Blade]

This PR introduces a regression. Opened #4586.

[octogonz]

#4586 (comment)

Some background

There is a longnstanding issue that API Documenter generates "markdown" yet there is no standardized syntax for Markdown. Of course CommonMark is such a standard, but all real world implementations add additional custom syntaxes, and because CommonMark lacks formalized extensibility, such custom syntaxes will always break the ability to parse true CommonMark correctly. As one example, suppose we introduce :::tip as mentioned above - consider this case:

:::tip
```
[link](http://example.com)
:::
```

Is [link](http://example.com) a hyperlink or not? Most likely the :::tip parser will naively snip out the code and parse it as Markdown, interpreting ``` as literal backticks because the closing ``` is now outside the chunk being parsed. As a result [link](http://example.com) is a real hyperlink. Whereas CommonMark would interpret [link](http://example.com) as being inside a code fence and NOT a hyperlink.

This mutual intelligibility is a mild inconvenience for human authors, but a major frustration for automated tools such as API Documenter that need to emit thousands of pages of API docs and have them render correctly. (BTW TSDoc explicitly aims to solve this problem by forbidding custom syntaxes and relying instead on HTML elements for advanced custom syntaxes.)

What it means

Until now, API Documenter has aimed to produce a balanced dialect of markdown syntax that is likely to get rendered correctly by the most popular engines (Jekyll and GitHub at the time). Where it doesn't work, people have implemented various hacks like api-documenter-docusaurus-plugin that try rewrite the incorrect expressions. That is exactly what @Lightning00Blade did, too.

PR #4578 will break that delicate equilibrium, so unfortunately we probably shouldn't have merged this PR. The safest move may be to revert it. 😅

A better solution

But how to solve @phryneas's problem then?

What we can do instead is introduce api-documenter.json settings to control the output dialect. In this way API Documenter can be optimized for a known Markdown engine rather than trying to be a "least common denominator." We've been planning this already, since Docusuarus 3 has finally adopted standard MDX (whose syntax is based on JSX not CommonMark and thus really needs its own target profile).

This week I'll put together a design proposal.

👉 Please reply to issue #4586