Repo: move shared-fixtures
tests to ast-spec
fixtures
#6065
Labels
accepting prs
Go ahead, send a pull request that resolves this issue
repo maintenance
things to do with maintenance of the repo, and not with code/docs
tests
anything to do with testing
Suggestion
In #3258 we added a new testing framework for our AST parsing tests.
We did this for a few reasons:
shared-fixtures
tests we rely on sets of manual patches to the AST to get our "alignment" tests to pass (which is obviously pretty hacky and hard to maintain)shared-fixtures
tests we also rely on a set of ignores to document and skip fixtures we know don't align.With a new framework comes a migration! We need to create fixtures for all of our AST nodes to test all the many combinations!
There's already a number of examples present in the the
ast-spec
package (for example all of thedeclaration
nodes are done), but to walk through the process:First, setup the fixture folder:
spec.ts
egpackages/ast-spec/src/declaration/ClassDeclaration
).fixtures
folderNext, add "success" fixture cases:
fixtures
folder, create a folder with a short kebab-cased name that describes the case you're coveringimplements-one
. Each fixture should only cover one hyper-targeted, specific case to keep things easy to debug and review.fixture.ts
(egpackages/ast-spec/src/declaration/ClassDeclaration/implements-one/fixture.ts
) and fill this file with your test case.For example - in our "Class declaration
implements-one
test case, we shouldn't add a matching interface declaration!cd packages/ast-spec && yarn jest /fixtures
snapshots
subfolder next to yourfixture.ts
that contains a number of snapshots generated from your test. At the time of writing that's:1-TSESTree-AST.shot
- the AST output from parsing your fixture (parsed using ourtypescript-estree
parser)2-TSESTree-Tokens.shot
- the AST tokens parsed out of your fixture (parsed using ourtypescript-estree
parser)3-Babel-AST.shot
- same as (1) but parsed using babel4-Babel-Tokens.shot
- same as (2) but parsed using babel5-AST-Alignment-AST.shot
- the diff of (1) and (3)6-AST-Alignment-Tokens.shot
- the diff of (2) and (4)fixtures-with-differences-errors.shot
. This is intentional - it's essentially an automatically generated register of the places where we our parser and babel's don't align!To add "error" fixture cases:
fixtures
folder, create a folder called_error_
._error_
and then afixture.ts
within that folder, then run jest).1-TSESTree-Error.shot
- the error caught during the parse or"NO ERROR"
if no error was thrown (parsed using ourtypescript-estree
parser)2-Babel-Error.shot
- same as (1) but parsed with babel3-Alignment-Error.shot
- a string describing the error state:"Babel errored but TSESTree didn't"
"TSESTree errored but Babel didn't"
"Both errored"
When creating fixtures it's important to stress that each fixture should be as hyper-focused as you can make it! The more code there is, the more AST gets generated and the harder it is for a human to parse the AST output to find the relevant bits of the AST. More code also means more surface area for changes to impact which means that there's more and more snapshots that unnecessarily need updating when we make parser changes (which just creates noise!).
How to decide what to create a fixture for? The best way it to look at the spec for the AST node and try to create a fixture that targets each case for each property. For example
ClassDeclaration
has a boolean propertyabstract: boolean
- so we want to ensure we have at least one fixture which generates atrue
value for the property, and one that generates afalse
value for the property. Again - fixtures are cheap and easy to create, so if in doubt, just add another fixture to cover a case you're thinking about.Make sure you consider both good and bad cases! Even though our parser doesn't currently throw many syntax errors (#1852), it's good practice to also add bad code fixtures so that we can clearly document the expected current behaviour of our parser and how it aligns with babel!
Finally - don't endeavour to cover every single case in a single PR (unless you really, really want to) - feel free to just create one PR that covers one or a small number of nodes. The beauty of this framework is that we don't need to do everything at once!
The text was updated successfully, but these errors were encountered: