New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
v18 roadmap #821
Comments
Thank you @isaacs . Tap is the only library I found I could use for my NodeJS-based TypeScript library. Though I am only using the If you do consider it, I know that'll likely not happen anytime soon and it could very well make it more complicated for you to maintain consistency across multiple packages. So I can understand why it wouldn't be a consideration either. |
I am planning to implement those things as plugins, so yes, they'll be separated out better than they are today. |
@isaacs , do you have any progress on that? There are too many empty checkboxes, and task count is really huge. I'm asking due to find options to leave Jest, because in ts/esm/esbuild/c8 world it looks a bit 'legacy'. |
I haven't made much progress on it, unfortunately. Doing a new startup, and have been getting a bit into some other hobbies. I have a feeling I'll get to this soon, though. There's a rough proof of concept about how to do plugins in a way that preserves type information so that TS can grok how each plugin decorates the main tap object, and you can get all the autocorrect goodness in your editor. That was really the hardest part, the rest is mostly just typing. The downside is that the plugin system won't be quite as flexible as I'd designed in the comment at the start of this issue, unfortunately. There's just no way to get static analysis to play nice with dynamically mutated objects. So, basically, it'll require a build step to create the plugin types. If you've ever used So that's big rock number 1. I have a proof of concept, but no working code for it yet. Big rock number 2 is doing coverage and tracking process info like tap was using nyc for (which does more than just coverage, also used for Big rock number 3 is mocking. I have a proof of concept approach, but no fully working integration-ready code for it yet. In the meantime, there are a few approaches you can use, here's what I've been relying on while using TypeScript: In package.json, put this: {
"scripts": {
"prepare": "tsc",
"test": "c8 tap test/*.ts",
"snap": "c8 tap test/*.ts",
"pretest": "tsc"
},
"tap": {
"coverage": false,
"node-arg": [
"--loader",
"ts-node/esm"
],
"ts": false
}
} Then you can do |
("working", for me and for tap, means "has full test coverage and docs", along with the actual code that does the thing) |
Thx for such detailed answer! Currently I work with And |
Can we do something like this? |
I tested all the common JS testing frameworks, and I hate all of them except |
Progress continues on this, but it's been slow going.
I have a project now where I have a bunch of utils (testing with the c8/ts config described above) but then I also have a bunch of react components I want to test with react-testing-library and global-jsdom, and routes that I'll want to run end-to-end tests with puppeteer. So the I hope to pull out of the depth-first traversal through lowlevel deps now, and get back to building up the The class hierarchy will look like this:
The types are all working in my tests, with plugins like this: import { TestBase, AssertionExtra } from '@tapjs/core'
export const plugin = <
T extends new (...a: any[]) => TestBase
>(
Base: T
) =>
class IsString extends Base {
isString(x: unknown, msg: string = 'expect string', extra: AssertionExtra = {}) {
const e = { ...extra, found: typeof x, expect: 'string' }
return typeof x === 'string' ? this.pass(msg, e) : this.fail(msg, e)
}
} then you'd install the plugin with:
And load it with: # .taprc
plugins:
- 'tap-plugin-is-string' And then the system can build up all the configured and builtin plugins into a single class defined in In the fullness of time, I imagine that puppeteer and react-testing-library will be plugins, so nested configs would mean having separate plugin builds for each folder. Which... idk, seems tricky. But since every test is an isolated process, it could be done by walking the test folder tree for all the config files, and building the plugged-in client on demand with a sha of the plugins used. You'd still have to run |
@isaacs , looks promising, thx! But what if config files... are "normal" programs too, like tests? ) What I mean - it's a bit wrong idea to import tap (and types, which is more important) from static package. Let's turn config (because you need config to specify plugins) into another entry point, like that: import { config } from "./tap.js"; // my local Tap implementation in 11 lines + config function
import { TapPluginIsString } from "./tap-plugin-is-string.js"; // simple instanceof-based implementation
class TapPluginXxx {
isXxx() {
return true;
}
}
export default config(new TapPluginIsString(), new TapPluginXxx()); Next, import tap from your config, not from original package: import tap from "./config.js";
await tap.test("Some test", async () => {
console.log("isString(1): ", tap.isString(1));
console.log("isXxx(): ", tap.isXxx());
}); The magic is inside 'config' function - Proxy to add new methods to the Tap instance, and some types to merge plugin types together: export const config = <T extends object[]>(
...plugins: [...T]
): Tap & Spread<T> => {
const tap = new Tap();
// @ts-ignore
return new Proxy(tap, {
get(target, p) {
if (p in target) {
// @ts-ignore
return target[p];
}
for (const plugin of plugins) {
if (p in plugin) {
return (...args: any[]) =>
// @ts-ignore
Reflect.apply(plugin[p], plugin, args);
}
}
},
});
}; What do you think about this pattern? |
That could definitely work, and actually isn't too far off from the build script I'm imagining. But it's kind of annoying to have to manually load up your plugins in that way. What I want to do is this:
It'll still be possible to configure plugins at run time if you like, but I just don't wanna do that in all my projects, too lazy 😅 Trying to reduce the boilerplate as much as possible. |
Oh, actually, the mechanism you're proposing is actually pretty different. I'm just stacking mixins. But that also means I can detect if there are method incompatibilities. The main drawback of using mixins in TypeScript is that you can't mark fields as protected/private/readonly. So if you want something to be private or readonly, you have to do it with javascript |
In my solution you can build any object what you want, you free to use anything including mixins ) What about boilerplate - theoretically you are right, especially if you do not expect that user will not edit configuration manually. You can use some strange file format for configuration (or binary) and provide set of commands to work with it to hide details from users. Okay. But what is the difference between autogenerated crappy .xxxxrc and autogenerated config.ts? No difference. Except ability to import config.ts and get all types without 'tap build'. Also, you can add config.ts to the 'imports' section of package.json (at plugin installation step), and write in readme: '...after adding plugins in TS project, you can use tap via "import tap from "#tap"'. |
@koshic Running into some issues with my "mixins everywhere" approach, actually. TypeScript really does not give them first class service, it's a shame, because mixins really are "the pattern" for this kind of problem. But I'm already so tired of "Base constructor must have same return type" and useless I had previously explored a somewhat approach to what you suggested, but still using a build step that generates the types and |
The proxy used in my example is intended only for simplification, in real code it can be replaced with a more or less complex object building. Any pattern what you want. I believe that 'classic' plugin-based architecture may be middle-term solution, a kind of step into ESM world. Time is running out - Node test-runner grown enough to be used as main solution for small projects. |
As I was reading the new messages was wondering when you'd run into this issue. It's the reason why my plugin system didn't use classes, which they were at first, but just couldn't get the types to work right. Also soft agree with @koshic on having config as entry point rather than having a generation step. Though, if we do end up having a generation step, I hope it will be possible to generate the client outside |
The client can be anywhere really, but the point is that it should be returned when doing I'm thinking of just going back to the "function that takes a Test object and decorates it" approach. Then I can specify the Test type as |
I'm not worried. People still use mkdirp and rimraf very extensively. The built in methods are good enough for many cases, but I don't see node adding the entire feature set of tap or jest into core. When people need that, they look to user land projects. And I need that, so I'm building it. Already, even with the current inconveniences of using tap with typescript, I find its added features useful, and very much miss stuff like --watch and --changed, which don't work with the current state of things, since they rely on nyc. |
The challenge seems to be getting |
Ah, nevermind. I see what I was doing wrong. Need to make it less generic, and also explicitly declare the inheritance chain as a multiple-inheritance interface. This actually works fine. https://bit.ly/ts-mixin-plugins (shortened ts playground link) |
Hmm. Actually that won't work unless I figure out a way to make Test objects not be Streams or EventEmitters, or else it blows up with a slew of |
@isaacs looks a bit strange - how are DOM types related to tap? did you set 'lib' in tsconfig to use only 'esxxxx' instead of default one, includes DOM? |
The issue is that the Base class derives from Minipass, which derives from node's Stream. Stream has a bunch of references to WebStreams (so you can do stuff like |
Ok, this is actually kinda working? A Promise returns are broken for some reason, but it's kind of working with
|
They also were against .ts extensions in imports. 5, 6, 7 years? Don't remember ) Now we have an ugly flag and mandatory 'noEmit' as a bonus. |
Yeah... between the garbage properties and the completely unnecessary So, I'm thinking, ditch the mixins and plugins entirely, and instead start from a clean slate with a dead simple module that just generates tap and can be extended classically in a plain old OOP kind of way. Then to add something like a plugin to add There are some ways that TypeScript makes JS much more powerful, but this is one aspect where it really feels like it's holding the language back. |
Oh yeah, I forgot to do t.error, whoops. Good catch. The import error is weird. It seems like tap 18 is getting the older version of tap-yaml, even though it's dep should be pinned to the new version. I'll try to repro that as well. |
@jsumners Oh, hey, this is a thing that's been an open question for years now, but I never changed it because it was annoying to do in the old codebase and would've been a breaking change (and I don't really use t.error() much, so never felt too motivated) but now that I'm looking at it, you get to decide ;) If you have a failing Ie, if you have a test like this: import t from 'tap'
t.test('t.error', t => {
const er = new Error('this could be anywhere')
t.pass('this is fine')
t.error(er)
t.end()
}) Is it more useful to see this:
or this
|
I guess, could also just do both?
|
/me gets back in from doing yard work... Happy to have helped 🤣 Seriously though, I don't have a strong opinion. If I had to choose, I'd probably choose showing the |
@jsumners Ok, tap@18.0.0-16 published with |
"keep"? 😂 |
Added a Remaining blockers:
|
|
Website deployed, such as it is: https://node-tap.surge.sh This will live on https://node-tap.org once it's finished, of course. |
I like the style |
Docs are now fully styled. I found a weird issue with how |
I don't know if this is me not understanding something or if there is something missing from the docs but I've upgraded to try the
However
|
@alexgoldstone Can you post this issue over at https://github.com/tapjs/tapjs/issues? Please include the way that you're running your tests. If you run them just as |
Ok, issue with All that's left now is production deploy, probably going to ship tomorrow or Monday.
|
Only thing remaining is for ts-node to land TypeStrong/ts-node#2009, and update to that (probably SemVer-major) release. Going to look into forking it to |
Just to confirm I didn't raise an issue as it was resolved by updating to the latest Node LTS (I was still on the previous). |
Got everything working on node 20 and typescript 5.2, and swapped to using a forked ts-node so the install is now much much faster. In the process, made a new build tool. The only task at this point is to get the tests for tshy completed, so that I can remove the dependency on an untested prerelease dep, but getting tap onto it was a great smoke-test, and it needed tap to work with 5.2 in order to use tap for its tests. That should be done today or tomorrow, and then release! |
Tshy is out, tap release tomorrow! |
Thanks for all the feedback and input in this super long mega-issue. I'll write a blog post this weekend and publish that Monday or Tuesday. In the meantime, |
@isaacs Are you willing to dual license all of the tap modules to MIT? It seems that OpenJSF projects are unable to include BlueOak licensed modules. I think that is silly since it's a clarified MIT, but 🤷♂️ what do I know about lawyering? |
@jsumners Not willing, no, sorry. OpenJSF can either accept BlueOak, buy a custom license from me, or stop using my code. |
I'm getting this error tap: 18.6.1 .taprc
.error
|
Please open new issues for new problems, rather than comment on closed issues.
|
This is the roadmap for node-tap v18.
Caveat: Tap is a side project, no one is paying me to do this. So this is not a timeline or schedule or itinerary, or even an estimate. It'll take as long as it takes, but this is what I'm thinking will go in it.
Might not all be in one v18, some of these might be pushed off until later. (Especially: the ESM/TS stuff is much more pressing, so Plugins might come in v19 or later.)
The roadmap for v17 is just that we'll start emitting
TAP version 14
. That's a breaking change, but not a big one.Overall Codebase/Architecture/API
"type":"module"
projects:node ./test.js
? Maybe if the loader is needed and not present, we crash and tell the user to donode --loader=tap/load ./test.js
? Automatically respawn if it's not present? Idk, but that'd be annoying to lose.t.mockESM(module, mocks)=>Promise
to mock either ESM or CJS modules.t.mock()
async? Bigger breaking change though, so idk.import { test } from 'tap'
continues to work. If you want to dosetTimeout(t.end)
, you'll have to usesetTimeout(() => t.end())
moving forward.t.mock
, snapshots,t.before
,t.beforeEach
,t.afterEach
,t.after
,t.fixture
,t.teardown
, and most assertions (probably all exceptpass
andfail
?) to be Plugins (see below).t.worker(...)
method, modelled aftert.spawn()
Idea: t.worker() method #812CLI
tap <noargs>
- Run all tests matching the test regexp.tap report
- Run the coverage report, and do not run any tests. Respond tocoverage-report
option.tap --coverage-report=html
is no longer a "run no tests and just run report" command.tap <files...>
- Run only the selected tests. If you have a test named'report'
, then this is weird, you'll have to dotap ./report
for that.Plugins
async loadPlugin(specifier)
to theTest
prototype. If the module provides of specific named function exports (not sure what those are yet), then they'll be called and provided aTest
object to work with. Special designatortapjs:...
can be used to reference built-in plugins.unloadPlugin(specifier)
to remove plugins from the set. This can be used, for example, to disable mocks in a given test, etc.addMethod
be inherited by all child tests (unless they callunloadPlugin
). To add a property (eg, at.nock
object), it'll have to be set up in thebefore
hook.load
, or one of its descendants.)load
.t.end()
, so assertions are allowed. Can return a promise, in which case it will be awaited before moving on to theteardown
phase.end
phase, but before moving on to the next test, after the test is already ended, so assertions not allowed. If it returns a promise, it's awaited before moving on to next test.beforeEach
orafterEach
, becausebefore
andafter
already are called for each subsequent child test. These methods will be added by the default builtintapjs:lifecycle
plugin.t.addMethod('name', (t, ...args) => {...})
to add a method to the relevantt
object which is inherited by all its descendants.t.addProperty('name', descriptor)
to add a property to the test object and all of its descendants. Simple values can bet.addProperty('foo', { value: 'bar' })
, but can get fancy witht.addProperty('foo', { get () { return this.bar }})
or whatever.t.addMethod(name, fn)
is technically justt.addProperty(name, { value: function () { return fn(this) }})
?t.matchSnapshot()
to specify a name which is appended to thet.snapshotFile
, before the.test.cjs
ext. (see nlfspec) So a plugin can easily createt.nock.snapshot()
by having the nock object callt.matchSnapshot(data, { snapshotExt: 'nock' })
or something.plugins
, which takes a list of names that can be passed torequire()
orimport()
. (So either package names or module specifiers will work, as long as they're resolvable in the project.) Plugins named in the config are loaded on the root TAP object.exclude-plugins
, which takes a list of plugin specifiers to not load. (Can be handy if you want to limit how much of tap loads for your tests, if you know you're not using mocks or snapshots or something? idk.)The text was updated successfully, but these errors were encountered: