Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feat] Add descriptive messages for skipped asserts #476

Merged
merged 1 commit into from Jun 29, 2019

Conversation

r0mflip
Copy link
Contributor

@r0mflip r0mflip commented Jun 26, 2019

Now asserts can take skip as a string or boolean, strings get printed as the reason for skipping the assert.

(ok|not ok) 3 <assert message> # SKIP <reason for skipping>

@ljharb
Copy link
Collaborator

ljharb commented Jun 26, 2019

Is this a convenience feature, or something that you'd expect as part of any tap client? Are their other TAP-producing runners that do this?

@r0mflip
Copy link
Contributor Author

r0mflip commented Jun 27, 2019

TAP indicates that any individual assert can be skipped or marked todo. node-tap allows this and this also plays really well with tap-parser. The same should work for todo, but node 0.8 has problems with the stack.

test('this skipps', { skip: true }, function (t) {
    t.fail('doesn\'t run');
    t.fail('this doesn\'t run too', { skip: false });
    t.end();
});

test('some tests might skip', function (t) {
    t.pass('this runs');
    t.fail('failing assert is skipped', { skip: true });
    t.pass('this runs');
    t.end();
});

test('incomplete test', function (t) {
    // var platform = process.platform; something like this needed
    var platform = 'win32';

    t.pass('run sh', { skip: platform !== 'win32' });
    t.pass('run openssl', { skip: platform === 'win32' });
    t.end();
});

test('incomplete test with explanation', function (t) {
    // var platform = process.platform; something like this needed
    var platform = 'win32';

    t.fail('run sh (conditional skip)', { skip: platform === 'win32' });
    t.fail('run openssl', { skip: platform === 'win32' && 'can\'t run on windows platforms' });
    t.pass('this runs');
    t.end();
});

tap-parser output

[
  [ 'version', 13 ],
  [ 'comment', '# SKIP this skipps\n' ],
  [ 'comment', '# some tests might skip\n' ],
  [
    'assert',
    Result { ok: true, id: 1, name: 'this runs', fullname: '' }
  ],
  [
    'assert',
    Result {
      ok: true,
      id: 2,
      skip: true,
      name: 'failing assert is skipped',
      fullname: ''
    }
  ],
  [
    'assert',
    Result { ok: true, id: 3, name: 'this runs', fullname: '' }
  ],
  [ 'comment', '# incomplete test\n' ],
  [
    'assert',
    Result { ok: true, id: 4, name: 'run sh', fullname: '' }
  ],
  [
    'assert',
    Result {
      ok: true,
      id: 5,
      skip: true,
      name: 'run openssl',
      fullname: ''
    }
  ],
  [ 'comment', '# incomplete test with explanation\n' ],
  [
    'assert',
    Result {
      ok: true,
      id: 6,
      skip: true,
      name: 'run sh (conditional skip)',
      fullname: ''
    }
  ],
  [
    'assert',
    Result {
      ok: true,
      id: 7,
      skip: "can't run on windows platforms",
      name: 'run openssl',
      fullname: ''
    }
  ],
  [
    'assert',
    Result { ok: true, id: 8, name: 'this runs', fullname: '' }
  ],
  [ 'plan', { start: 1, end: 8 } ],
  [ 'comment', '# tests 8\n' ],
  [ 'comment', '# pass  8\n' ],
  [ 'comment', '# ok\n' ],
  [ 'comment', '# skip: 4\n' ],
  [
    'complete',
    FinalResults {
      ok: true,
      count: 8,
      pass: 8,
      fail: 0,
      bailout: false,
      todo: 0,
      skip: 4,
      plan: FinalPlan {
        start: 1,
        end: 8,
        skipAll: false,
        skipReason: '',
        comment: ''
      },
      failures: []
    }
  ]
]

lib/results.js Outdated Show resolved Hide resolved
lib/results.js Outdated Show resolved Hide resolved
lib/results.js Outdated Show resolved Hide resolved
lib/results.js Outdated Show resolved Hide resolved
var common = require('./common');
var stripFullStack = common.stripFullStack;

tap.test('tape todo test', { todo: process.versions.node.match(/0\.8\.\d+/) ? 'Fails on node 0.8': false }, function (assert) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've temporarily marked it as todo

@ljharb ljharb merged commit 838d995 into tape-testing:master Jun 29, 2019
@ljharb ljharb added the semver-minor: new stuff Any additions. label Jun 29, 2019
ljharb added a commit that referenced this pull request Jun 29, 2019
 - [New] Add descriptive messages for skipped asserts (#476)
 - [Fix] emit skipped tests as objects (#473)
 - [Refactor] use `!!` over `Boolean()`
 - [meta] clean up license so github can detect it
 - [Deps] update `inherits`, `resolve`
 - [Tests] add tests for 'todo' exit codes (#471)
@r0mflip r0mflip deleted the descriptive-asserts branch January 7, 2020 11:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants