Support VS Code problem matchers without custom configuration (i.e. out of the box)

[eamodio]

Refs: #2206 & webpack/webpack#5895

Without any additional configuration, we would like to be able to support VS Code problem matchers for webpack builds (especially watch).

What They Are

VS Code problem matchers scrape terminal output to detect and surface compile/lint/etc issues in VS Code. These problem matchers allow VS Code to provide a richer build experience when using build tasks. These tasks can be (and often are) just package.json scripts run by npm or yarn, or they can be any other type of build system if it outputs usable information to the terminal. For example, many devs configure a default build task, and then when they press F5 to start debugging, the debugger waits until the build task completes successfully before launching. And warnings/errors (say from typescript/eslint/etc) are aggregated into the VS Code Problems view:

See the documentation for more information.

How They Work

A problem matcher is a regex patterns that are run against the terminal output, to capture warnings/errors and provide them to VS Code. See a simple sample here.

But a more advanced problem matcher is needed for background/watch tasks. For that we need to provide two more regex patterns, one to capture the start of the desired output and the end of it. So in with webpack that would be the start of a compilation and when the compilation completes.

What We Need

So to support these we need a reliable output that can be matched to provide a starting and ending point for a build/watch. A start & end message should be output for each webpack configuration that is run and ideally a start & end message for entire "run".

We had these messages before #2206 was merged, with the
Compilation starting... & Compilation finished messages, and the additional Compilation [config-name] starting... & Compilation [config-name] finished when for each config in a multi-config setup

Also, here is the current TypeScript + Webpack Problem Matchers I maintain for VS Code. The relevant patterns are here: https://github.com/eamodio/vscode-tsl-problem-matcher/blob/687f20cd297b0eaa391e5f8776f8819f806f534f/package.json#L188-L196

And to be clear, I really don't mind/care at all what the start/end messages are -- just that they can be identified and used.

/cc @alexander-akait

[alexander-akait]

Do you have ability to modify process.env? We need to find a communication channel between webpack-cli and your extension

[eamodio]

Yeah, we could probably set an env variable.

[alexander-akait]

@eamodio I can implement FORCE_LOG and always output log when you set it

[eamodio]

Do we really want to opt users into all logs? Could we do something more specific to just the types of messages required?

[alexander-akait]

@eamodio yep, what about FORCE_LIFE_CYCLE_LOG (i.e. show only Compilation starting... and Compilation finished), something needed more? Maybe better name, because it is long, any ides?

[eamodio]

Yeah, something like that sounds good... maybe WEBPACK_LOG_LIFE_CYCLE (adding some sort of prefix I think is important -- maybe even WEBPACK_CLI_*)?

[alexander-akait]

What about WEBPACK_CLI_LOG_LIFE_CYCLE?

[eamodio]

Sounds great!

[alexander-akait]

I will try to implement this tomorrow (need finish ES modules config support) and will do release. Want to clarify:

• Do you want to use stderr or stdout?
• Do you need this https://github.com/webpack/webpack-cli/blob/master/packages/webpack-cli/lib/plugins/CLIPlugin.js#L45, https://github.com/webpack/webpack-cli/blob/master/packages/webpack-cli/lib/plugins/CLIPlugin.js#L55 and https://github.com/webpack/webpack-cli/blob/master/packages/webpack-cli/lib/plugins/CLIPlugin.js#L66?

[eamodio]

I think it needs to be stdout.

I'm not sure what the difference between L45 & L55 are, but it looks like those are what I need. Is there a pair that would encapsulate the full watch run, if there were multiple configs?

[alexander-akait]

I think it needs to be stdout.

Why? It breaks stats output, logging is diagnostic information so it should use stderr

I'm not sure what the difference between L45 & L55 are, but it looks like those are what I need. Is there a pair that would encapsulate the full watch run, if there were multiple configs?

Can you clarify?

[eamodio]

Nm, I think stderr is fine.

So for a setup like this:
https://github.com/eamodio/vscode-gitlens/blob/395c61e8678a0a08fa1328de695c56e69a1eba9b/webpack.config.js#L68-L84

Where there are 2 (or more) webpack configs exported. It would be great to be able to classify each via its own start/end pair as well as the entire run that would encapsulate both of them.

[alexander-akait]

If developer have name in configuration we print name in logs, but we can improve this too, so if name was not provide we can print unknown[0], unknown[1] and etc

[eamodio]

Yeah, I believe the name is already used in those life-cycle logs, which is great. I was say that it would be beneficial to have start/end pair that would capture the entire compilation -- for example:

Compilation starting...
Compilation [name0] starting...
Compilation [name0] finished
Compilation [name1] starting...
Compilation [name1] finished
Compilation finished

That way I can tell when each config starts/finishes, as well as when all of them start/finish.

[alexander-akait]

Compilation starting...
Compilation [name0] starting...
Compilation [name0] finished
Compilation [name1] starting...
Compilation [name1] finished
Compilation finished

We can't do it, because each compilation is async, and we can't wait and slow down build

[eamodio]

Do you have to wait? Couldn't you just have a counter of the number of compilations, and then at the end of each decrement the counter, and then do basically what is done here:

webpack-cli/packages/webpack-cli/lib/plugins/CLIPlugin.js

Lines 68 to 72 in 1adc5aa

	process.nextTick(() => {
	if (compiler.watchMode) {
	this.logger.log(`Compiler${getCompilationName()} is watching files for updates...`);
	}
	});

But instead of checking for the watchMode check if the counter is 0, and if so then output the final message?

[alexander-akait]

Do you have to wait?

We don't know when compilation(s) finished so order can be any

But instead of checking for the watchMode check if the counter is 0, and if so then output the final message?

We can't use number, because it can be very big after long usage

[alexander-akait]

Keeping order is impossible, so you can have

Compilation starting...
Compilation [name0] starting...
Compilation [name1] starting...
Compilation [name1] finished
Compilation [name0] finished
Compilation finished

[eamodio]

How can the number get very big? Isn't it always just the number of compilations triggered?

As for order, yeah that is definitely expected and not an issue at all. The only ordering that matters is that the start of any compilation is first, and the final compilation line comes at the end.

Compilation starting...   <--- this is always first
... <--- anything can happen in here
Compilation finished   <--- this is always last

[eamodio]

Basically I'm saying something like at the start of a compilation run, set a counter to the number of individual webpack configs that would each output a Compilation [name0] starting... Compilation [name0] finished pair. Then at the end of each one of those compilations, decrement the counter.

The same could probably work if you can't/don't know the count up front and just have each compliation increment the counter when it starts and decrement at the end. But that theoretically could have a compilation that starts and ends before another one even starts, so you could end up observing 0 before it was really 0.

[alexander-akait]

Sorry, but I'm completely confused what you want

[eamodio]

Ideally I want 2 sets of start & end pairs.

Pair 1:

Compilation [name] starting...
...   <-- compilation stuff/output happens in here
Compilation [name] finished

This covers the start & end of a single compilation.

Pair 2:

Starting compilations...
...   <-- 1 or more individual compilations happen in here
Compilations finished

This covers the start & end of all compilations.

Pair 1, already exists today, while Pair 2 doesn't AFAIK.

I don't really have any preference on what the actual messages of Pair 2 are, I just need to be able to tell the difference between them and Pair 1 (via regex)

[alexander-akait]

What about Starting unknown[0] compilations... and Compilations unknown[0] finished? It means compilerby index0` started

[eamodio]

@alexander-akait Is your suggestion for Pair 2? If so, what is the unknown? If you mean for Pair 1 (assuming the case where a name doesn't exist), then unknown[index] sounds good to me as a fallback.

[alexander-akait]

Is your suggestion for Pair 2?

Yes

If so, what is the unknown?

No name for compiler

[eamodio]

@alexander-akait what would the name be? Pair 2 is to "wrap" all compilations

[alexander-akait]

Sorry, I don't understand your, can you clarify?

[alexander-akait]

Fixed #2566, will be part of the next release, you can use WEBPACK_CLI_START_FINISH_FORCE_LOG env var to enable this