Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Brotli compression support #432

Open
wants to merge 11 commits into
base: master
Choose a base branch
from
47 changes: 25 additions & 22 deletions README.md
Expand Up @@ -46,7 +46,7 @@ This module will help you:
4. Optimize it!

And the best thing is it supports minified bundles! It parses them to get real size of bundled modules.
And it also shows their gzipped sizes!
And it also shows their gzipped or Brotli sizes!

<h2 align="center">Options (for plugin)</h2>

Expand All @@ -61,7 +61,8 @@ new BundleAnalyzerPlugin(options?: object)
|**`analyzerPort`**|`{Number}` or `auto`|Default: `8888`. Port that will be used in `server` mode to start HTTP server.|
|**`reportFilename`**|`{String}`|Default: `report.html`. Path to bundle report file that will be generated in `static` mode. It can be either an absolute path or a path relative to a bundle output directory (which is output.path in webpack config).|
|**`reportTitle`**|`{String\|function}`|Default: function that returns pretty printed current date and time. Content of the HTML `title` element; or a function of the form `() => string` that provides the content.|
|**`defaultSizes`**|One of: `stat`, `parsed`, `gzip`|Default: `parsed`. Module sizes to show in report by default. [Size definitions](#size-definitions) section describes what these values mean.|
|**`defaultSizes`**|One of: `stat`, `parsed`, `compressed`|Default: `parsed`. Module sizes to show in report by default. [Size definitions](#size-definitions) section describes what these values mean.|
|**`compressionAlgorithm`**|One of: `gzip`, `brotli`|Default: `gzip`. Compression type used to calculate the compressed module sizes.|
|**`openAnalyzer`**|`{Boolean}`|Default: `true`. Automatically open report in default browser.|
|**`generateStatsFile`**|`{Boolean}`|Default: `false`. If `true`, webpack stats JSON file will be generated in bundle output directory|
|**`statsFilename`**|`{String}`|Default: `stats.json`. Name of webpack stats JSON file that will be generated if `generateStatsFile` is `true`. It can be either an absolute path or a path relative to a bundle output directory (which is output.path in webpack config).|
Expand Down Expand Up @@ -111,23 +112,25 @@ Directory containing all generated bundles.
### `options`

```
-V, --version output the version number
-m, --mode <mode> Analyzer mode. Should be `server`, `static` or `json`.
In `server` mode analyzer will start HTTP server to show bundle report.
In `static` mode single HTML file with bundle report will be generated.
In `json` mode single JSON file with bundle report will be generated. (default: server)
-h, --host <host> Host that will be used in `server` mode to start HTTP server. (default: 127.0.0.1)
-p, --port <n> Port that will be used in `server` mode to start HTTP server. Should be a number or `auto` (default: 8888)
-r, --report <file> Path to bundle report file that will be generated in `static` mode. (default: report.html)
-t, --title <title> String to use in title element of html report. (default: pretty printed current date)
-s, --default-sizes <type> Module sizes to show in treemap by default.
Possible values: stat, parsed, gzip (default: parsed)
-O, --no-open Don't open report in default browser automatically.
-e, --exclude <regexp> Assets that should be excluded from the report.
Can be specified multiple times.
-l, --log-level <level> Log level.
Possible values: debug, info, warn, error, silent (default: info)
-h, --help output usage information
-V, --version output the version number
-m, --mode <mode> Analyzer mode. Should be `server`, `static` or `json`.
In `server` mode analyzer will start HTTP server to show bundle report.
In `static` mode single HTML file with bundle report will be generated.
In `json` mode single JSON file with bundle report will be generated. (default: server)
-h, --host <host> Host that will be used in `server` mode to start HTTP server. (default: 127.0.0.1)
-p, --port <n> Port that will be used in `server` mode to start HTTP server. Should be a number or `auto` (default: 8888)
-r, --report <file> Path to bundle report file that will be generated in `static` mode. (default: report.html)
-t, --title <title> String to use in title element of html report. (default: pretty printed current date)
-s, --default-sizes <type> Module sizes to show in treemap by default.
Possible values: stat, parsed, compressed (default: parsed)
--compression-algorithm <type> Compression algorithm that will be used to calculate the compressed module sizes.
Possible values: gzip, brotli (default: gzip)
-O, --no-open Don't open report in default browser automatically.
-e, --exclude <regexp> Assets that should be excluded from the report.
Can be specified multiple times.
-l, --log-level <level> Log level.
Possible values: debug, info, warn, error, silent (default: info)
-h, --help output usage information
```

<h2 align="center" id="size-definitions">Size definitions</h2>
Expand All @@ -147,9 +150,9 @@ It is called "stat size" because it's obtained from Webpack's
This is the "output" size of your files. If you're using a Webpack plugin such
as Uglify, then this value will reflect the minified size of your code.

### `gzip`
### `compressed`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we make it so that the added option for defaultSizes would be brotli instead of compressed? That way the current, default gzip usage would not have to be changed.

That way we could keep the gzip-related documentation intact and only worry about adding new documentation related to the brotli options.

I could imagine that the fourth size definition documentation here could look something like this:

brotli

This is the size of running the parsed bundles/modules through Brotli compression. Note: This needs the compression algorithm to be configured to use Brotli.


This is the size of running the parsed bundles/modules through gzip compression.
This is the size of running the parsed bundles/modules through compression.

<h2 align="center">Selecting Which Chunks to Display</h2>

Expand All @@ -169,7 +172,7 @@ The Chunk Context Menu can be opened by right-clicking or `Ctrl`-clicking on a s

<h2 align="center">Troubleshooting</h2>

### I don't see `gzip` or `parsed` sizes, it only shows `stat` size
### I don't see `compressed` or `parsed` sizes, it only shows `stat` size

It happens when `webpack-bundle-analyzer` analyzes files that don't actually exist in your file system, for example when you work with `webpack-dev-server` that keeps all the files in RAM. If you use `webpack-bundle-analyzer` as a plugin you won't get any errors, however if you run it via CLI you get the error message in terminal:
```
Expand Down
17 changes: 10 additions & 7 deletions client/components/ModulesTreemap.jsx
Expand Up @@ -18,11 +18,13 @@ import Search from './Search';
import {store} from '../store';
import ModulesList from './ModulesList';

const SIZE_SWITCH_ITEMS = [
{label: 'Stat', prop: 'statSize'},
{label: 'Parsed', prop: 'parsedSize'},
{label: 'Gzipped', prop: 'gzipSize'}
];
function allSizeSwitchItems() {
return [
{label: 'Stat', prop: 'statSize'},
{label: 'Parsed', prop: 'parsedSize'},
{label: window.compressedSizeLabel, prop: 'gzipSize'}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we change gzipSize to compressedSize?

];
}

@observer
export default class ModulesTreemap extends Component {
Expand Down Expand Up @@ -138,7 +140,7 @@ export default class ModulesTreemap extends Component {
renderModuleSize(module, sizeType) {
const sizeProp = `${sizeType}Size`;
const size = module[sizeProp];
const sizeLabel = SIZE_SWITCH_ITEMS.find(item => item.prop === sizeProp).label;
const sizeLabel = allSizeSwitchItems().find(item => item.prop === sizeProp).label;
const isActive = (store.activeSize === sizeProp);

return (typeof size === 'number') ?
Expand All @@ -162,7 +164,8 @@ export default class ModulesTreemap extends Component {
};

@computed get sizeSwitchItems() {
return store.hasParsedSizes ? SIZE_SWITCH_ITEMS : SIZE_SWITCH_ITEMS.slice(0, 1);
const items = allSizeSwitchItems();
return store.hasParsedSizes ? items : items.slice(0, 1);
}

@computed get activeSizeItem() {
Expand Down
13 changes: 0 additions & 13 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 0 additions & 1 deletion package.json
Expand Up @@ -37,7 +37,6 @@
"acorn-walk": "^8.0.0",
"chalk": "^4.1.0",
"commander": "^6.2.0",
"gzip-size": "^6.0.0",
"lodash": "^4.17.20",
"opener": "^1.5.2",
"sirv": "^1.0.7",
Expand Down
3 changes: 3 additions & 0 deletions src/BundleAnalyzerPlugin.js
Expand Up @@ -104,6 +104,7 @@ class BundleAnalyzerPlugin {
host: this.opts.analyzerHost,
port: this.opts.analyzerPort,
reportTitle: this.opts.reportTitle,
compressionAlgorithm: this.opts.compressionAlgorithm,
bundleDir: this.getBundleDirFromCompiler(),
logger: this.logger,
defaultSizes: this.opts.defaultSizes,
Expand All @@ -115,6 +116,7 @@ class BundleAnalyzerPlugin {
async generateJSONReport(stats) {
await viewer.generateJSONReport(stats, {
reportFilename: path.resolve(this.compiler.outputPath, this.opts.reportFilename || 'report.json'),
compressionAlgorithm: this.opts.compressionAlgorithm,
bundleDir: this.getBundleDirFromCompiler(),
logger: this.logger,
excludeAssets: this.opts.excludeAssets
Expand All @@ -126,6 +128,7 @@ class BundleAnalyzerPlugin {
openBrowser: this.opts.openAnalyzer,
reportFilename: path.resolve(this.compiler.outputPath, this.opts.reportFilename || 'report.html'),
reportTitle: this.opts.reportTitle,
compressionAlgorithm: this.opts.compressionAlgorithm,
bundleDir: this.getBundleDirFromCompiler(),
logger: this.logger,
defaultSizes: this.opts.defaultSizes,
Expand Down
21 changes: 15 additions & 6 deletions src/analyzer.js
Expand Up @@ -2,16 +2,21 @@ const fs = require('fs');
const path = require('path');

const _ = require('lodash');
const gzipSize = require('gzip-size');

const Logger = require('./Logger');
const Folder = require('./tree/Folder').default;
const {parseBundle} = require('./parseUtils');
const {createAssetsFilter} = require('./utils');
const {gzipSize, brotliSize} = require('./sizeUtils');

const FILENAME_QUERY_REGEXP = /\?.*$/u;
const FILENAME_EXTENSIONS = /\.(js|mjs)$/iu;

const COMPRESSED_SIZE = {
gzip: gzipSize,
brotli: brotliSize
};

module.exports = {
getViewerData,
readStatsFromFile
Expand All @@ -20,11 +25,15 @@ module.exports = {
function getViewerData(bundleStats, bundleDir, opts) {
const {
logger = new Logger(),
excludeAssets = null
excludeAssets = null,
compressionAlgorithm
} = opts || {};

const isAssetIncluded = createAssetsFilter(excludeAssets);

const compressedSize = COMPRESSED_SIZE[compressionAlgorithm];
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's better call it getCompressedSize please

if (!compressedSize) throw new Error(`Unsupported compression algorithm: ${compressionAlgorithm}.`);

// Sometimes all the information is located in `children` array (e.g. problem in #10)
if (_.isEmpty(bundleStats.assets) && !_.isEmpty(bundleStats.children)) {
const {children} = bundleStats;
Expand Down Expand Up @@ -102,7 +111,7 @@ function getViewerData(bundleStats, bundleDir, opts) {

if (assetSources) {
asset.parsedSize = Buffer.byteLength(assetSources.src);
asset.gzipSize = gzipSize.sync(assetSources.src);
asset.gzipSize = compressedSize(assetSources.src);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

?

Suggested change
asset.gzipSize = compressedSize(assetSources.src);
asset.compressedSize = compressedSize(assetSources.src);

}

// Picking modules from current bundle script
Expand Down Expand Up @@ -143,7 +152,7 @@ function getViewerData(bundleStats, bundleDir, opts) {
}

asset.modules = assetModules;
asset.tree = createModulesTree(asset.modules);
asset.tree = createModulesTree(asset.modules, {compressedSize});
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
asset.tree = createModulesTree(asset.modules, {compressedSize});
asset.tree = createModulesTree(asset.modules, {getCompressedSize});

return result;
}, {});

Expand Down Expand Up @@ -203,8 +212,8 @@ function isRuntimeModule(statModule) {
return statModule.moduleType === 'runtime';
}

function createModulesTree(modules) {
const root = new Folder('.');
function createModulesTree(modules, opts) {
const root = new Folder('.', opts);

modules.forEach(module => root.addModule(module));
root.mergeNestedFolders();
Expand Down
24 changes: 21 additions & 3 deletions src/bin/analyzer.js
Expand Up @@ -10,7 +10,10 @@ const viewer = require('../viewer');
const Logger = require('../Logger');
const utils = require('../utils');

const SIZES = new Set(['stat', 'parsed', 'gzip']);
const SIZES = new Set(['stat', 'parsed', 'compressed']);
const ACCEPTED_SIZES = new Set([...SIZES, 'gzip']);

const ALGORITHMS = new Set(['gzip', 'brotli']);

const program = commander
.version(require('../../package.json').version)
Expand Down Expand Up @@ -58,6 +61,12 @@ const program = commander
br(`Possible values: ${[...SIZES].join(', ')}`),
'parsed'
)
.option(
'--compression-algorithm <type>',
'Compression algorithm that will be used to calculate the compressed module sizes.' +
br(`Possible values: ${[...ALGORITHMS].join(', ')}`),
'gzip'
)
.option(
'-O, --no-open',
"Don't open report in default browser automatically."
Expand All @@ -83,6 +92,7 @@ let {
report: reportFilename,
title: reportTitle,
defaultSizes,
compressionAlgorithm,
logLevel,
open: openBrowser,
exclude: excludeAssets,
Expand All @@ -104,7 +114,12 @@ if (mode === 'server') {
port = port === 'auto' ? 0 : Number(port);
if (isNaN(port)) showHelp('Invalid port. Should be a number or `auto`');
}
if (!SIZES.has(defaultSizes)) showHelp(`Invalid default sizes option. Possible values are: ${[...SIZES].join(', ')}`);
if (!ACCEPTED_SIZES.has(defaultSizes)) {
showHelp(`Invalid default sizes option. Possible values are: ${[...SIZES].join(', ')}`);
}
if (!ALGORITHMS.has(compressionAlgorithm)) {
showHelp(`Invalid compression algorithm option. Possible values are: ${[...ALGORITHMS].join(', ')}`);
}

bundleStatsFile = resolve(bundleStatsFile);

Expand All @@ -125,6 +140,7 @@ if (mode === 'server') {
port,
host,
defaultSizes,
compressionAlgorithm,
reportTitle,
bundleDir,
excludeAssets,
Expand All @@ -136,13 +152,15 @@ if (mode === 'server') {
reportFilename: resolve(reportFilename || 'report.html'),
reportTitle,
defaultSizes,
compressionAlgorithm,
bundleDir,
excludeAssets,
logger: new Logger(logLevel)
});
} else if (mode === 'json') {
viewer.generateJSONReport(bundleStats, {
reportFilename: resolve(reportFilename || 'report.json'),
compressionAlgorithm,
bundleDir,
excludeAssets,
logger: new Logger(logLevel)
Expand All @@ -156,7 +174,7 @@ function showHelp(error) {
}

function br(str) {
return `\n${' '.repeat(28)}${str}`;
return `\n${' '.repeat(32)}${str}`;
}

function array() {
Expand Down
13 changes: 13 additions & 0 deletions src/sizeUtils.js
@@ -0,0 +1,13 @@
const zlib = require('zlib');

export function gzipSize(input) {
return zlib.gzipSync(input, {level: 9}).length;
}

export function brotliSize(input) {
if (typeof zlib.brotliCompressSync !== 'function') {
throw new Error('Brotli compression requires Node.js v10.16.0 or higher.');
}
dcsaszar marked this conversation as resolved.
Show resolved Hide resolved

return zlib.brotliCompressSync(input).length;
}
3 changes: 2 additions & 1 deletion src/template.js
Expand Up @@ -39,7 +39,7 @@ function getScript(filename, mode) {
}
}

function renderViewer({title, enableWebSocket, chartData, defaultSizes, mode} = {}) {
function renderViewer({title, enableWebSocket, chartData, defaultSizes, compressedSizeLabel, mode} = {}) {
return html`<!DOCTYPE html>
<html>
<head>
Expand All @@ -59,6 +59,7 @@ function renderViewer({title, enableWebSocket, chartData, defaultSizes, mode} =
<script>
window.chartData = ${escapeJson(chartData)};
window.defaultSizes = ${escapeJson(defaultSizes)};
window.compressedSizeLabel = ${escapeJson(compressedSizeLabel)};
</script>
</body>
</html>`;
Expand Down
10 changes: 7 additions & 3 deletions src/tree/Folder.js
@@ -1,5 +1,4 @@
import _ from 'lodash';
import gzipSize from 'gzip-size';

import Module from './Module';
import BaseFolder from './BaseFolder';
Expand All @@ -8,13 +7,18 @@ import {getModulePathParts} from './utils';

export default class Folder extends BaseFolder {

constructor(name, opts) {
super(name);
this.opts = opts;
}

get parsedSize() {
return this.src ? this.src.length : 0;
}

get gzipSize() {
if (!_.has(this, '_gzipSize')) {
this._gzipSize = this.src ? gzipSize.sync(this.src) : 0;
this._gzipSize = this.src ? this.opts.compressedSize(this.src) : 0;
}

return this._gzipSize;
Expand Down Expand Up @@ -42,7 +46,7 @@ export default class Folder extends BaseFolder {
// See `test/stats/with-invalid-dynamic-require.json` as an example.
!(childNode instanceof Folder)
) {
childNode = currentFolder.addChildFolder(new Folder(folderName));
childNode = currentFolder.addChildFolder(new Folder(folderName, this.opts));
}

currentFolder = childNode;
Expand Down