FR: parse jasmine logging based on a console.log instead of framework log

[wswebcreation]

Current situation:
Currently protractor-flake for jasmine can use 2 parsers, the standard.js and the multi.js parser. Both parsers are based on the logging of the framework and use complex regular expressions due to changes in the framework loggings.

Desired situation:
Replace both parsers with a new parser that will read the log based on a console.log users need to add to their framework. This is currently already advised for CucumberJS 1.
The advantage of parsing the log of protractor based on a console.log is that:

• it will be more stable and framework independent then the current parsing because that one is based on the logging of the framework itself.
• it will be less complex
• single logging can be used for both single and multiple instances

To do:

• [] create a new parser called jasmine, standard and multi will be removed and this will result in a new major release
• [] test this for Jasmine and make sure the current functionality won't break (support for suites and so on)
• [] create new unit tests
• [] create new docs for Jasmine with examples how to implement this

[tomyam1]

I got hit by this issue, i.e. had failing specs pass due to wrong detection of the spec file.

We have a spec file that contains a global afterEach that tests the console error message printed during the spec. Since it's a separate file, protractor-flake treats it as the file of the failed spec, even though the spec that failed is defined in a different file (the actual file in not in the stack trace so the current parser won't be able to detect it).

It seems the main challenge with jasmine is that it won't share the path of the spec, but we can easily hack it to get this info.

This is the approach we took, hope it'd help to solve this issue.

One just has to call this function on onPrepare:

// This must be called on protractor's onPrepare so our custom parser can pick up the names
// of the failed specs
exports.prepareJasmineForFlake = function () {
    const expectedJasmineLoadSpec =
        `function () {
  this.specFiles.forEach(function(file) {
    require(file);
  });
}`;

    let currSpecFile;

    const Jasmine = require('jasmine/lib/jasmine');

    if (Jasmine.prototype.loadSpecs.toString() !== expectedJasmineLoadSpec) {
        console.log(Jasmine.prototype.loadSpecs.toString());
        throw new Error(`Jasmine.prototype.loadSpecs is not as expected, refusing to modify it`);
    }

    Jasmine.prototype.loadSpecs = function () {
        this.specFiles.forEach(function (file) {
            currSpecFile = file;
            try {
                require(file);
            } finally {
                currSpecFile = null;
            }
        });
    };

    // A mapping of spec IDs to the path of the spec file they are defined at
    // More precisely this is the files that requiring it caused the spec to be created
    const specFiles = {};

    // Override jasmine's global it/fit functions to connect each spec ID with a file path
    const originalIt = global.it;
    if (!originalIt) throw new Error(`global.it is not defined. Can't mock it now`);
    global.it = function () {
        const spec = originalIt.apply(this, arguments);
        specFiles[spec.id] = currSpecFile;
        return spec;
    };

    const originalFit = global.fit;
    if (!originalFit) throw new Error(`global.fit is not defined. Can't mock it now`);
    global.fit = function () {
        const spec = originalFit.apply(this, arguments);
        specFiles[spec.id] = currSpecFile;
        return spec;
    };

    class JasmineFailuresReporter {
        specDone(result) {
            if (result.status === 'failed') {
                const failedSpecFile = specFiles[result.id];

                if (!failedSpecFile) {
                    // We might be here due to a declaration exception in the specs and these messages
                    // will help us understand why we have it
                    console.error(`Spec ${result.id} failed for the following reasons and we can't tell where it was defined`);
                    result.failedExpectations.forEach((expectation) => {
                        console.error(expectation.message);
                        console.error(expectation.stack);
                    });

                    throw new Error(`Can't tell spec file for failed spec ${result.id} ${result.fullName}`);
                }

                console.log(`protractor-flake-retry-spec:${failedSpecFile}`);
            }
        }
        jasmineDone() {
            console.log(`can use protractor-flake-retry-spec-parser`);
        }
    }

    jasmine.getEnv().addReporter(new JasmineFailuresReporter());
};

And use this as the parser:

// Custom parser for protractor-flake that picks up the console message created by the reporter
// in prepareJasmineForFlake
exports.flakeParser = {
    name: 'protractor-flake-retry-spec-parser',
    parse: function parse(output) {
        if (output.indexOf('can use protractor-flake-retry-spec-parser') < 0) {
            throw new Error(`Can't use protractor-flake-retry-spec-parser because you didn't call prepareJasmineForFlake`);
        }

        const failedSpecs = new Set();
        let match;
        const failedSpecRegex = /protractor-flake-retry-spec:(.*)/g;
        while (match = failedSpecRegex.exec(output)) { // eslint-disable-line no-cond-assign
            failedSpecs.add(match[1]);
        }
        return Array.from(failedSpecs);
    }
};