linkcheck: Improve robustness during anchor checks

[jayaddison]

Feature or Bugfix

• Bugfix

Purpose

• Improve robustness during anchor-checks, when the linkcheck decodes response content.
• Begin to add more validation of parsed HTML.

Detail

• Do not attempt to check for anchors when UTF8 decoding fails; report affected hyperlinks as ignored.
• Report hyperlinks as broken when their content fails validation checks, but allow this to be overridden using URI-match patterns.

Relates

• Resolves [linkcheck] PDF anchor (...pdf#anchor) leads to 'utf-8' codec can't decode byte ... #11041.

[jayaddison]

@AA-Turner is it OK if we pause on this one for a while? I think/hope it's a good idea, but I'm slightly concerned that the distinction between lenient and strict parsing might be a bit arbitrary. I should perhaps move this back into a draft status.

[jayaddison]

Another way to phrase that: I'd like to canvas more (and perhaps leaning skeptical) opinions about this before proceeding.

[jayaddison]

Thanks!