feat: add error snippets #919

KuznetsovRoman · 2024-04-25T00:21:10Z

What is done

Add error snippets

Examples

1

2

3

Notice how stacktrace says example.hermione.tsx?import:12:18, but we are pointing to line 18 because of source map evaluation

KuznetsovRoman · 2024-04-25T00:25:45Z

src/error-snippets/frames.ts

+import type { ResolvedFrame, SufficientStackFrame } from "./types";
+
+// prettier-ignore
+const frameRelevance: Record<string, { value: number; matcher: (fileName: string) => boolean }> = {


https://nodejs.org/api/errors.html#errorstack

The location information will be one of:

native, if the frame represents a call internal to V8 (as in [].forEach).

plain-filename.js:line:column, if the frame represents a call internal to Node.js.

/absolute/path/to/file.js:line:column, if the frame represents a call in a user program (using CommonJS module system), or its dependencies.

<transport-protocol>:///url/to/module/file.mjs:line:column, if the frame represents a call in a user program (using ES module system), or its dependencies.

Ranking them as:

0: can't extract code snippet; useless

1: better than nothing

2: better than wdio internals, but worse, than user code part (these could be testplane parts)

3: user code. If frame of user code appears, always should pick it

Ranking them as

this comment should be in the code

/** * @description * Rank values: * * 0: Can't extract code snippet; useless * * 1: WebdriverIO internals: Better than nothing * * 2: Project internals: Better than WebdriverIO internals, but worse, than user code part * * 3: User code: Best choice */ const FRAME_REELVANCE: Record<string, { value: number; matcher: (fileName: string) => boolean }> = { repl: { value: 0, matcher: fileName => /^REPL\d+$/.test(fileName) }, nodeInternals: { value: 0, matcher: fileName => /^node:[a-zA-Z\-_]/.test(fileName) }, wdioInternals: { value: 1, matcher: fileName => fileName.includes("/node_modules/webdriverio/") }, projectInternals: { value: 2, matcher: fileName => fileName.includes("/node_modules/") }, userCode: { value: 3, matcher: () => true }, } as const;

KuznetsovRoman · 2024-04-25T00:28:28Z

src/error-snippets/frames.ts

+    } catch (findError) {
+        logger.warn("Unable to find relevant stack frame:", findError);
+
+        return null;
+    }


If error appers, dont throw any error, just returning null with console warn

KuznetsovRoman · 2024-04-25T00:30:28Z

src/error-snippets/index.ts

+const stackFrameResolver = async (stackFrame: SufficientStackFrame): Promise<ResolvedFrame> => {
+    const fileContents = await getSourceCodeFile(stackFrame.fileName);
+    const sourceMaps = await extractSourceMaps(fileContents, stackFrame.fileName);
+
+    return sourceMaps ? resolveWithSourceMap(stackFrame, sourceMaps) : resolveWithStackFrame(stackFrame, fileContents);
+};


getSourceCodeFile extracts source code from file and applies source maps, if it is in http headers: https://docs.google.com/document/d/1U1RGAehQwRypUTovF1KRlpiOFze0b-_2gc6fAH0KY0k/edit#heading=h.lmz475t4mvbx

So in other places we dont have to think "is the file from node:fs or browser:fetch?"

source maps will be evaluated, if exists

KuznetsovRoman · 2024-04-25T00:31:38Z

src/error-snippets/index.ts

+    return sourceMaps ? resolveWithSourceMap(stackFrame, sourceMaps) : resolveWithStackFrame(stackFrame, fileContents);
+};
+
+export const extendWithCodeSnippet = async (err: WithSnippetError): Promise<WithSnippetError> => {


this function does not throw exceptions either.
The worst it can do - log a warning into console.

KuznetsovRoman · 2024-04-25T00:34:56Z

src/error-snippets/source-maps.ts

+            ? fileContents.slice(sourceMapsStartIndex + SOURCE_MAP_URL_COMMENT.length)
+            : fileContents.slice(sourceMapsStartIndex + SOURCE_MAP_URL_COMMENT.length, sourceMapsEndIndex);
+
+    const sourceMaps = await getSourceCodeFile(url.resolve(fileName, sourceMapUrl));


sourceMapUrl could either be relative, absolute or data:application/json;base64,...

https://docs.google.com/document/d/1U1RGAehQwRypUTovF1KRlpiOFze0b-_2gc6fAH0KY0k/edit#heading=h.lmz475t4mvbx

After url.resolve we could get:

absolute path (which we could fs.readFile)

network url (which we could fetch)

embedded (which would be handled by fetch)

KuznetsovRoman · 2024-04-25T00:36:04Z

src/error-snippets/utils.ts

+    linesBelow: number;
+}
+
+export const softFileURLToPath = (fileName: string): string => {


For esm, fileName starts with file://

KuznetsovRoman · 2024-04-25T00:37:30Z

src/error-snippets/utils.ts

+    }
+};
+
+export const formatFileNameHeader = (fileName: string, opts: formatFileNameHeaderOpts): string => {


Formats this kind of header above code snippet:

. | // tests-dir/test.js . | 9 | it.only('only test', async ({browser}) => {

KuznetsovRoman · 2024-04-25T00:38:57Z

src/error-snippets/utils.ts

+    const sourceMapUrl = response.headers.get("SourceMap");
+
+    return responseText + "\n" + SOURCE_MAP_URL_COMMENT + sourceMapUrl;


By specification, source maps could be in HTTP Header SourceMap.

If it is, we could just append our file contents and dont think about it elsewhere

KuznetsovRoman · 2024-04-25T00:39:56Z

src/reporters/flat.js

@@ -37,6 +37,11 @@ module.exports = class FlatReporter extends BaseReporter {
                const icon = testCase.isFailed ? icons.FAIL : icons.RETRY;

                this.informer.log(`   ${testCase.browserId}`);
+                if (testCase.errorSnippet) {
+                    this.informer.log("");
+                    testCase.errorSnippet.split("\n").forEach(line => this.informer.log(line));


align snippet with log prefix

KuznetsovRoman · 2024-04-25T00:40:49Z

src/reporters/utils/helpers.js

+    }
+
+    if (test.err && test.err.snippet) {
+        testInfo.errorSnippet = chalk.supportsColor ? test.err.snippet : stripAnsi(test.err.snippet);


This way we can use colorfull version for console, and remove control characters if user pipes logs into file

src/error-snippets/frames.ts

sipayRT · 2024-04-27T10:39:10Z

src/error-snippets/frames.ts

+const frameRelevance: Record<string, { value: number; matcher: (fileName: string) => boolean }> = {
+    repl: { value: 0, matcher: fileName => /^REPL\d+$/.test(fileName) },
+    nodeNative: { value: 0, matcher: fileName => fileName === "native" },
+    nodeInternals: { value: 0, matcher: fileName => /^node:[a-zA-Z\-_]/.test(fileName) || /^[a-zA-Z\-_]+\.[a-zA-Z]+$/.test(fileName)},


can you, pls, provide an examples for these errors?

I took it from https://nodejs.org/api/errors.html#errorstack

The location information will be one of:

- native, if the frame represents a call internal to V8 (as in [].forEach). - plain-filename.js:line:column, if the frame represents a call internal to Node.js. - /absolute/path/to/file.js:line:column, if the frame represents a call in a user program (using CommonJS module system), or its dependencies. - <transport-protocol>:///url/to/module/file.mjs:line:column, if the frame represents a call in a user program (using ES module system), or its dependencies. The string representing the stack trace is lazily generated when the error.stack property is accessed.

But it looks like they forgot to update it, because internal filename differs from version to version.

In node@8 it was like this:

TypeError: Module.SourceMap is not a constructor at myFunc (/Users/kroman512/appiumProj/stacktrace.js:12:9) at Immediate.setImmediate [as _onImmediate] (/Users/kroman512/appiumProj/stacktrace.js:15:5) at runCallback (timers.js:810:20) at tryOnImmediate (timers.js:768:5) at processImmediate [as _immediateCallback] (timers.js:745:5)

(so timers.js is node internal)

In node@12 it was:

TypeError [ERR_INVALID_ARG_TYPE]: The "payload" argument must be of type object. Received undefined at cloneSourceMapV3 (internal/source_map/source_map.js:315:11) at new SourceMap (internal/source_map/source_map.js:141:21) at myFunc (/Users/kroman512/appiumProj/stacktrace.js:12:9) at Immediate._onImmediate (/Users/kroman512/appiumProj/stacktrace.js:15:5) at processImmediate (internal/timers.js:461:21)

so it starts with internal/. But starting from node@16 its this:

TypeError [ERR_INVALID_ARG_TYPE]: The "payload" argument must be of type object. Received undefined at cloneSourceMapV3 (node:internal/source_map/source_map:318:3) at new SourceMap (node:internal/source_map/source_map:141:21) at myFunc (/Users/kroman512/appiumProj/stacktrace.js:12:9) at Immediate.<anonymous> (/Users/kroman512/appiumProj/stacktrace.js:15:5) at processImmediate (node:internal/timers:466:21)

So it is always node:internal/ in node16+

I will remove second nodeInternals matcher as well as native one (this one can't be produced from javascript and only can appear in node runtime itself. Also turns out, we can't parse it anyways):

/** * @description * Rank values: * * 0: Can't extract code snippet; useless * * 1: WebdriverIO internals: Better than nothing * * 2: Project internals: Better than WebdriverIO internals, but worse, than user code part * * 3: User code: Best choice */ const FRAME_REELVANCE: Record<string, { value: number; matcher: (fileName: string) => boolean }> = { repl: { value: 0, matcher: fileName => /^REPL\d+$/.test(fileName) }, nodeInternals: { value: 0, matcher: fileName => /^node:[a-zA-Z\-_]/.test(fileName) }, wdioInternals: { value: 1, matcher: fileName => fileName.includes("/node_modules/webdriverio/") }, projectInternals: { value: 2, matcher: fileName => fileName.includes("/node_modules/") }, userCode: { value: 3, matcher: () => true }, } as const;

sipayRT · 2024-04-27T10:42:46Z

src/error-snippets/frames.ts

+import type { ResolvedFrame, SufficientStackFrame } from "./types";
+
+// prettier-ignore
+const frameRelevance: Record<string, { value: number; matcher: (fileName: string) => boolean }> = {


Ranking them as

this comment should be in the code

src/error-snippets/frames.ts

test/src/error-snippets/frames.ts

sipayRT · 2024-04-27T11:42:56Z

test/src/error-snippets/frames.ts

+        });
+    });
+
+    it("resolveWithStackFrame", () => {


not the most understandable test name :)

stackFrameResolver -> stackFrameLocationResolver
resolveWithSourceMap -> resolveLocationWithSourceMap
resolveWithStackFrame -> resolveLocationWithStackFrame

sipayRT · 2024-04-27T11:46:32Z

test/src/error-snippets/index.ts

+            assert.calledOnceWith(loggerStub.warn, "Unable to apply code snippet:");
+        });
+
+        it("should return the same Object.is object", async () => {


not sure that the name of the test needs to be specified about Object.is

Why not?
{} and {} are the same objects. But there is a nuance)

test/src/error-snippets/utils.ts

KuznetsovRoman force-pushed the HERMIONE-1517.snippets branch from b823d34 to 54cccb4 Compare April 25, 2024 00:23

KuznetsovRoman commented Apr 25, 2024

View reviewed changes

KuznetsovRoman force-pushed the HERMIONE-1517.snippets branch 6 times, most recently from fe15bf2 to be2a68f Compare April 26, 2024 10:23

sipayRT approved these changes Apr 27, 2024

View reviewed changes

KuznetsovRoman force-pushed the HERMIONE-1517.snippets branch 4 times, most recently from 03b2c0f to 389eb4a Compare April 27, 2024 13:43

feat: add error snippets

a78ba06

KuznetsovRoman force-pushed the HERMIONE-1517.snippets branch from 389eb4a to a78ba06 Compare April 28, 2024 22:32

KuznetsovRoman merged commit ddc3e28 into master May 1, 2024
2 checks passed

KuznetsovRoman deleted the HERMIONE-1517.snippets branch May 1, 2024 15:31

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add error snippets #919

feat: add error snippets #919

KuznetsovRoman commented Apr 25, 2024 •

edited

KuznetsovRoman Apr 25, 2024

KuznetsovRoman Apr 25, 2024

sipayRT Apr 27, 2024

KuznetsovRoman Apr 27, 2024

KuznetsovRoman Apr 25, 2024

KuznetsovRoman Apr 25, 2024

KuznetsovRoman Apr 25, 2024

KuznetsovRoman Apr 25, 2024

KuznetsovRoman Apr 25, 2024 •

edited

KuznetsovRoman Apr 25, 2024

KuznetsovRoman Apr 25, 2024 •

edited

KuznetsovRoman Apr 25, 2024

KuznetsovRoman Apr 25, 2024

KuznetsovRoman Apr 25, 2024

sipayRT Apr 27, 2024

KuznetsovRoman Apr 27, 2024 •

edited

sipayRT Apr 27, 2024

sipayRT Apr 27, 2024

KuznetsovRoman Apr 27, 2024

sipayRT Apr 27, 2024

KuznetsovRoman Apr 27, 2024

		const sourceMapUrl = response.headers.get("SourceMap");

		return responseText + "\n" + SOURCE_MAP_URL_COMMENT + sourceMapUrl;

feat: add error snippets #919

feat: add error snippets #919

Conversation

KuznetsovRoman commented Apr 25, 2024 • edited

What is done

Examples

1

2

3

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

KuznetsovRoman Apr 25, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

KuznetsovRoman Apr 25, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

KuznetsovRoman Apr 27, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

KuznetsovRoman commented Apr 25, 2024 •

edited

KuznetsovRoman Apr 25, 2024 •

edited

KuznetsovRoman Apr 25, 2024 •

edited

KuznetsovRoman Apr 27, 2024 •

edited