Parcel logging improvements

[yamadapc]

Problem statement

Parcel could log more information to aid its developers in debugging issues such as cache corruption, performance issues etc.

In order to have more verbose logs, parcel should ideally:

• Implement better tracing capabilities
• Log data into a file rather to STDOUT, to prevent logs from impacting the user experience
• Rotate log files
• Have a logging strategy that will easily work for both native and node.js components

Current state

Currently parcel logs are implemented via @parcel/logger. The logger itself does not write to any destination, but rather emit events. Within @parcel/core, @parcel/core/src/ReporterRunner.js will subscribe to logger events and forward them to all configured reporters.

The CLI reporter at @parcel/reporter-cli will match logger events and print them to STDOUT and STDERR.

Proposed state

I propose that parcel adds a dependency on popular logging libraries for node.js and rust.

These could be pino and tracing_subscriber + tracing. We would log to a temporary file and configure these libraries to rotate the output after a certain size is reached.

We'd instrument throughout the codebase, verbose structured logs for operations like building of individual files or serialising data between threads; as much as we possibly can without impacting performance.

The reporter based logs, which are "user facing logs", can be kept with the current solution.

@parcel/logger API would be modified to account for user-facing logs, which would be printed to STDOUT and be user-facing ; and internal logs, which would be written to a temporary directory for debugging purposes.

Possibly, this would be split ; making @parcel/logger a different name; or importing the logging libraries directly where they are needed.

[devongovett]

The goal of reporters is to abstract the event handling so multiple UIs can be built - e.g. terminal, browser, vscode extension, etc. Outsourcing logging to another library might make this more difficult. A custom reporter plugin could definitely implement some of the features you propose though, perhaps even using one of those libraries if it is helpful. You could listen for log events and store them wherever you want.

What additional logs are you looking to add? I haven't really thought of Parcel like a typical long-running backend service where logs would need to be stored since it resets on each build so I'm curious what you're looking to learn.

[yamadapc]

I totally understand the use-case for multiple reporters and that can stay for certain types of messages.
However, I think there is missing visibility for development and debugging purposes, rather than user-facing logs.

Currently there are very little logs because they are thought as more user-facing logs.

For example, there's a request system, which could log all requests and their durations (when set at a certain level), which would help debug issues or understand performance, but does not log that information.

What additional logs are you looking to add?

Ideally there would be sufficient logs such that at maximum logging level, any bug is diagnosable. There should never be a case, for example, where we have a build hang, slowness or corrupted cache and can't tell what the source of the problem if given access to verbose logs.

Example verbose / trace log lines:

• All RequestTracker::runRequest calls should log their start & finish times
• All worker calls should log some metadata around what has been serialized & how big the message is
• The number of workers should be logged on start-up as well as the worker back-end
• The size of the request graph should be logged on start-up as well as on every request execution
• The size of the asset and bundle graphs likely could also be logged

I haven't really thought of Parcel like a typical long-running backend service where logs would need to be stored since it resets on each build so I'm curious what you're looking to learn

Consider there's also a case where developers are running parcel as a dev-server, and then eventually hit a problem. Sufficient logging would ideally allow us to then request a log-file from this developer (or automatically already have its contents) and then trace down what went wrong at a high-level.

[yamadapc]

The issue currently with implementing a reporter that will log to a file is that:

• This is significant unnecessary work, since we don't want those log lines to be shown anywhere and their purpose is just for development
• We likely don't want the same logs to be printed on the CLI reporter and to a file, we'd actually want more data to be logged in the background for development purposes than what most users see

[yamadapc]

Current issues that would be aided by extra logging getting added into the tool:

• Hangs on shutdown
• Hangs when querying changes from the watcher
• At least one source of native crashes