Skip to content

Commit

Permalink
Update READMEs to be more obvious about XML lexers.
Browse files Browse the repository at this point in the history
  • Loading branch information
alecthomas committed Oct 28, 2022
1 parent 9b9b245 commit 06f7007
Show file tree
Hide file tree
Showing 2 changed files with 13 additions and 8 deletions.
7 changes: 3 additions & 4 deletions README.md
Expand Up @@ -207,14 +207,13 @@ for details on implementing lexers. Most concepts apply directly to Chroma,
but see existing lexer implementations for real examples.

In many cases lexers can be automatically converted directly from Pygments by
using the included Python 3 script `pygments2chroma.py`. I use something like
using the included Python 3 script `pygments2chroma_xml.py`. I use something like
the following:

```sh
python3 _tools/pygments2chroma.py \
python3 _tools/pygments2chroma_xml.py \
pygments.lexers.jvm.KotlinLexer \
> lexers/k/kotlin.go \
&& gofmt -s -w lexers/k/kotlin.go
> lexers/embedded/kotlin.xml
```

See notes in [pygments-lexers.txt](https://github.com/alecthomas/chroma/blob/master/pygments-lexers.txt)
Expand Down
14 changes: 10 additions & 4 deletions lexers/README.md
@@ -1,19 +1,24 @@
# Lexer tests
# Chroma lexers

All lexers in Chroma should now be defined in XML unless they require custom code.

## Lexer tests

The tests in this directory feed a known input `testdata/<name>.actual` into the parser for `<name>` and check
that its output matches `<name>.exported`.

It is also possible to perform several tests on a same parser `<name>`, by placing know inputs `*.actual` into a
directory `testdata/<name>/`.

## Running the tests
### Running the tests

Run the tests as normal:
```go
go test ./lexers
```

## Update existing tests
### Update existing tests

When you add a new test data file (`*.actual`), you need to regenerate all tests. That's how Chroma creates the `*.expected` test file based on the corresponding lexer.

To regenerate all tests, type in your terminal:
Expand All @@ -26,7 +31,8 @@ This first sets the `RECORD` environment variable to `true`. Then it runs `go te

(That environment variable tells Chroma it needs to output test data. After running `go test ./lexers` you can remove or reset that variable.)

### Windows users
#### Windows users

Windows users will find that the `RECORD=true go test ./lexers` command fails in both the standard command prompt terminal and in PowerShell.

Instead we have to perform both steps separately:
Expand Down

0 comments on commit 06f7007

Please sign in to comment.