Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added regex test coverage #2640

Closed

Conversation

RunDevelopment
Copy link
Member

@RunDevelopment RunDevelopment commented Nov 15, 2020

This adds a little regex test coverage tool/test. It tracks all matches (= the result of /some regex/.exec(code)) of all of Prism's regexes on all language tests. If a regex doesn't produce any non-null matches, it will be reported. Reported regexes are grouped by language, so the error message will look like this:

  1) Pattern test coverage
       Coverage
         - should cover all patterns in bison:
     AssertionError: 1 pattern(s) in bison are untested:
You can learn more about writing tests at https://prismjs.com/test-suite.html#writing-tests

Prism.languages.bison.bison.inside.number.pattern:
        /(^|[^@])\b(?:0x[\da-f]+|\d+)/gi
This pattern is completely untested. Add test files that match this pattern.
      at Context.<anonymous> (tests\coverage.js:98:13)
      at processImmediate (internal/timers.js:456:21)

  2) Pattern test coverage
       Coverage
         - should cover all patterns in gml:
     AssertionError: 5 pattern(s) in gml are untested:
You can learn more about writing tests at https://prismjs.com/test-suite.html#writing-tests

Prism.languages.gml.keyword:
        /\b(?:if|else|switch|case|default|break|for|repeat|while|do|until|continue|exit|…
This pattern is completely untested. Add test files that match this pattern.

Prism.languages.gml.number:
        /(?:\b0x[\da-f]+|(?:\b\d+\.?\d*|\B\.\d+)(?:e[+-]?\d+)?)[ulf]*/gi
This pattern is completely untested. Add test files that match this pattern.

Prism.languages.gml.operator:
        /[-+%=]=?|!=|\*\*?=?|\/\/?=?|<[<=>]?|>[=>]?|[&|^~]|\b(?:or|and|not|with|at|xor)\…
This pattern is completely untested. Add test files that match this pattern.

Prism.languages.gml.constant:
        /\b(?:self|other|all|noone|global|local|undefined|pointer_(?:invalid|null)|actio…
This pattern is completely untested. Add test files that match this pattern.

Prism.languages.gml.variable:
        /\b(?:x|y|(?:x|y)(?:previous|start)|(?:h|v)speed|direction|speed|friction|gravit…
This pattern is completely untested. Add test files that match this pattern.
      at Context.<anonymous> (tests\coverage.js:98:13)
      at processImmediate (internal/timers.js:456:21)

This also means that all languages without any tests will be reported (we had 3: GML, Ren'py, and Arduino).

The current coverage only reports completely untested patterns and simple keyword lists that are not exhaustively tested. We might want to expand this in the future. Those future applications are not part of this PR because it takes a lot of time to make all of Prism's >200 languages conform to a new standard.
Edit: The only reason I included a test for exhaustive keyword tests is that I only needed to add 13 tests. It was quite easy to add.

The main motivation for this PR is code review.
With this test, we will never merge a language with missing tests ever again.


I'm not quite done fixing all tests yet. I'll do the rest tomorrow or the day after.
Edit: I'm done.

@RunDevelopment
Copy link
Member Author

@mAAdhaTTah Could you please review this? I change a lot of files, so I kinda don't want to resolve merge conflicts for this. I would appreciate it if we could merge this quickly.

The main reason so many language definitions change is that I found and fixed bugs. (As it turns out, untested code can contain bugs.)
Most of the new code snippets in the new (and changed) test files are either from the official documentation of the respective programming language or from various sources (credited) I found while googling.

I also want to point out that the code coverage is by far not perfect. A regex (except for keyword lists) is considered covered if it has 1 match and since we don't know what parts of the pattern its matches cover, there is probably still a lot of untested code. I plan to address this problem in the future but the PR is already big enough for now.

@mAAdhaTTah
Copy link
Member

@RunDevelopment For the future, for big PRs like this, it would be helpful to ship it without whitespace or style changes. It would make it easier to review, as I wouldn't have to determine whether a change is style or functional.

@RunDevelopment
Copy link
Member Author

For the future, for big PRs like this, it would be helpful to ship it without whitespace or style changes.

Sorry about that.

@github-actions
Copy link

github-actions bot commented May 4, 2021

JS File Size Changes (gzipped)

A total of 19 files have changed, with a combined diff of +46 B (+0.2%).

file master pull size diff % diff
components/prism-birb.min.js 399 B 413 B +14 B +3.5%
components/prism-bsl.min.js 747 B 751 B +4 B +0.5%
components/prism-clojure.min.js 1.4 KB 1.4 KB -3 B -0.2%
components/prism-dataweave.min.js 507 B 508 B +1 B +0.2%
components/prism-gml.min.js 3.74 KB 3.75 KB +8 B +0.2%
components/prism-idris.min.js 377 B 397 B +20 B +5.3%
components/prism-jsstacktrace.min.js 411 B 409 B -2 B -0.5%
components/prism-lisp.min.js 995 B 1.01 KB +15 B +1.5%
components/prism-nim.min.js 659 B 665 B +6 B +0.9%
components/prism-nix.min.js 766 B 759 B -7 B -0.9%
components/prism-ocaml.min.js 577 B 576 B -1 B -0.2%
components/prism-php.min.js 1.96 KB 1.95 KB -11 B -0.6%
components/prism-powerquery.min.js 1.02 KB 1.02 KB -3 B -0.3%
components/prism-purebasic.min.js 1.13 KB 1.13 KB -1 B -0.1%
components/prism-regex.min.js 597 B 596 B -1 B -0.2%
components/prism-renpy.min.js 2.04 KB 2.03 KB -7 B -0.3%
components/prism-typescript.min.js 557 B 551 B -6 B -1.1%
components/prism-uri.min.js 518 B 517 B -1 B -0.2%
components/prism-zig.min.js 1.15 KB 1.17 KB +21 B +1.8%

Generated by 🚫 dangerJS against 2aa89a7

@RunDevelopment
Copy link
Member Author

It would be really nice if we could merge this. @mAAdhaTTah

@RunDevelopment
Copy link
Member Author

Closed in favor of #3138.

@RunDevelopment RunDevelopment deleted the regex-test-coverage branch October 5, 2021 22:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants