False positive "Block and modifier ..." in code blocks

[iliubinskii]

Search terms

at-sign, block and modifier tags

Expected Behavior

README.md:

Some text @abc

    Code block @xyz

Should show "Block and modifier tags will be ignored within the readme" error for "abc" only.

Actual Behavior

Shows "Block and modifier tags will be ignored within the readme" error for "abc" and "xyz".

The problem is that @ symbol can't be escaped inside code block (https://www.markdownguide.org/basic-syntax/#code-blocks-1).
If you replace @ with \@ in code block it will just show both characters.

Steps to reproduce the bug

README.md:

Some text @abc

    Code block @xyz

[Gerrit0]

If you use ``` for a code block instead, TypeDoc can recognize it. TypeDoc doesn't recognize this style of code blocks because doing so would require parsing a lot more markdown (4 spaces doesn't always mean code block)

[akphi]

@Gerrit0 I got the same problem and I don't think it has to do with the block comment in the code, this is the readme file we try to load either by specifying readme in typedoc.json or by typedoc.readmeFile in package.json. The problem is we parse the content of the readme file using parseComment and this will throw warning as well as chunk off the parts of the readme with the strange tag. I think we need to alter the logic of rendering readme a bit, though I have no idea yet 😢

[Gerrit0]

Yes, readme files are parsed with parseComment, just like comments are. We have to do some parsing in it so that {@link} tags within the readme can be resolved. When implementing the parsing I had two options:

1. Make it as similar to regular comments as possible
2. Only parse inline tags, markdown file parser becomes very different from the comment parser

I ended up going with option 1, because while today readme files aren't allowed to use block/modifier tags, I could foresee a use case where this might no longer be the case with the upcoming guides work, and it's easier to be more restrictive initially and relax those restrictions later.

[akphi]

@Gerrit0 hm, sorry I'm quite new to TypeDoc, but is there a way to exclude some phrases from being considered a tag?,

For example, can I specify somewhere so that @abc and @xyz are not considered tags so that parseComment will leave them as text? I feel like grabbing everything prefixed by @ is a rather aggressive 😭

[Gerrit0]

Nope, that's not supported.

TypeDoc doesn't grab everything after a @ as a tag. me@example.com will not be picked up as a tag

[akphi]

Nope, that's not supported.

Does it make sense to be a feature in your vision for the project?

TypeDoc doesn't grab everything after a @ as a tag. me@example.com will not be picked up as a tag

I meant token starting with @, I just looked at the lexer and it seems so

[Gerrit0]

I don't think it does. @tags are tags, having them sometimes be tags, or be tags only if in a comment means there's more things that people have to remember when working on documentation...

This could be that I don't understand why you're asking... Are false positives in readmes more common than I think? Maybe there's a good argument that I haven't heard yet for why I should have gone with option 2 in my previous post.

[akphi]

Yeah, I think I'm obviously biassed since I'm trying to pledge my case, but for me, in particular, it's npm scoped package's name, e.g.@jest/globals, @typescript-eslint/parser, etc. that occurs a lot in our guides and doc since we provide libraries

But then again, even when such an option exists, what if we have a package name @example/something and the user has to pick the prefix for exclusion. In short, it's not a good option, so let's probably not do it 😅

I think maybe we can step back and just treat readme as markdown text. So maybe, we can use something like remark.js to build the markdown, and use react-markdown to render them. I just browsed through, seems like they support adding classes to their component (e.g. BlockQuote, Link, etc.) so we can use typedoc CSS classes.

[Gerrit0]

See, now there's a reason that I hadn't seen before - at the very least, TypeDoc should not create a tag if the lookahead when reading is - or /. (I think actually it should not unless there's whitespace/eof)

I'll go ahead and make this change, will be curious if that takes care of most of your warnings, or reveals another common case.

[akphi]

@Gerrit0 Yes, as always, thank you for your care and quick turnaround! 0.23.7 clears my use case for sure. I'm excited to see what we offer for Guides in 0.24.x, but I hope we could consider allowing having a mode to parse markdown file as text rather than as a comment.