Organizational options for elm tests in other opinionated ecosystems?

[MattCheely]

I mentioned this in Slack today, and @avh4 suggested I open an issue here describing my use case, so here goes:

At work we have an Ember project that I've planted a few Elm seeds in (here's hoping they grow). One thing I encountered in the upgrade to 0.19 is that both Ember and Elm have opinions about what should go in the tests directory at the root of a project. Ember wants some configuration and a lot of sub-directories for different types of tests in there, and elm-test expects all of the test modules to live there. With 0.18, I could just push my elm tests down to tests/elm/ since the tests were more like a second project.

Just dropping a bunch of FooTest.elm files on top of the Ember testing structure is a bit rough organizationally, so I've considered a few different options as workarounds:

1. Make a Test directory inside tests and have modules named Test.Foo instead of FooTest. This is what I did just to get things working for now. It's not likely to be particularly intuitive for new devs trying to find their way around in the repo though, and I'm going to get a lot of "turtles all the way down" jokes from the rest of my team.

2. Stick with a pattern similar to tests in 0.18 where there's a sub-project in tests/elm that shares source directories and dependencies with the main project elm.json. The downside here is that there's manual overhead in keeping the two configurations in sync.

3. Lift all of the Elm code and elm.json out of Ember's folder structure and just put it at the top of the repo as it's own sub-project. This would require updates to addon we're using for integration with Ember's build system, and probably adding some tooling for ember-cli so that installing Elm dependencies didn't require always dropping into the sub-project. I suspect this is the best long-term option for us.

The other option would of course be to make the location of tests configurable when running elm-test, but I'm not sure that's worth it just to accommodate this particular use case. I'd be curious if folks embedding Elm in other opinionated frameworks like Angular have similar problems, or if anyone can think of additional/better approaches.

[noiseOnTheNet]

@MattCheely , I have started using elm-test in a project with elm and Django; even if I don't have strict requirements so far I'm trying to keep python and elm code in the same package, so currently I have a "test" directory in the package root; of course I'd like to rename it in a more clear way, so your latest proposal is interesting for me too.

[rtfeldman]

See #306 - would this address your use case?

[mdevlamynck]

It wouldn't be ideal to have to specify a glob each time we want to run the tests in an application project, though it would work.

How about adding an optional tests-directory field to the elm.json file that defaults to ./tests when not specified?

Pros:

• That way we don't break existing projects and it nicely mirrors the existing sources-directories.
• If in the future the field tests-directory is added automatically to the elm.json file (by mean elm init or elm install), it promotes accidental discovery of the feature.
• That would be easier for new elm user than searching through github issues to discover the glob behavior discussed in Only care about tests/ if no globs given #306.
• Encourages your goal of "You can open up any project and run the command elm-test and have it run all the project's tests." and does not conflict with your plan to integrate elm-test in elm publish. (mentioned in Only care about tests/ if no globs given #306).

Cons:

• An other field added to elm.json
• Tie elm-test more to elm itself (though it's already the case with the test-dependencies field and I would actually like to see elm-test distributed along with elm)

IMHO that's the best approach.

[mgold]

We don't control elm.json.

If you're going off the rails enough to have tests in a separate folder, likely because you have an existing JS app, then you can wire up a script to run your JS and Elm tests with one command.

[rtfeldman]

@noiseOnTheNet @MattCheely - if you have time, would be great to know if #306 would address your use case!

(I realize there are possible ways to make it more ergonomic, but I want to be conservative with this because it's hard to take back, so I'm particularly interested in whether #306 would be sufficient, even if not ideal, for addressing the use case!)

[MattCheely]

@rtfeldman I think #306 is a good solution. In our case, developers and CI tools are always running these tests through npm scripts anyway, so configuring some file globs in those scripts is pretty easy. I think that's a good practice in general (elm-test gets installed as a dev dependency in the project, so everybody's using the same version, nobody has to muck around with global installs, etc.) and #306 fits nicely with that.

[noiseOnTheNet]

@rtfeldman I like #306 too. I'm automating CI through Jenkins so no problem using this solution.