Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Brotli compression support #432

Open
wants to merge 11 commits into
base: master
Choose a base branch
from
45 changes: 26 additions & 19 deletions README.md
Expand Up @@ -45,7 +45,7 @@ This module will help you:
4. Optimize it!

And the best thing is it supports minified bundles! It parses them to get real size of bundled modules.
And it also shows their gzipped sizes!
And it also shows their gzipped or Brotli sizes!

<h2 align="center">Options (for plugin)</h2>

Expand All @@ -61,7 +61,8 @@ new BundleAnalyzerPlugin(options?: object)
|**`analyzerUrl`**|`{Function}` called with `{ listenHost: string, listenHost: string, boundAddress: server.address}`. [server.address comes from Node.js](https://nodejs.org/api/net.html#serveraddress)| Default: `http://${listenHost}:${boundAddress.port}`. The URL printed to console with server mode.|
|**`reportFilename`**|`{String}`|Default: `report.html`. Path to bundle report file that will be generated in `static` mode. It can be either an absolute path or a path relative to a bundle output directory (which is output.path in webpack config).|
|**`reportTitle`**|`{String\|function}`|Default: function that returns pretty printed current date and time. Content of the HTML `title` element; or a function of the form `() => string` that provides the content.|
|**`defaultSizes`**|One of: `stat`, `parsed`, `gzip`|Default: `parsed`. Module sizes to show in report by default. [Size definitions](#size-definitions) section describes what these values mean.|
|**`defaultSizes`**|One of: `stat`, `parsed`, `gzip`, `brotli`|Default: `parsed`. Module sizes to show in report by default. [Size definitions](#size-definitions) section describes what these values mean.|
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this case user will have to provide compression algorithm twice: for compressionAlgorithm and defaultSizes and it opens possibilities for invalid option combinations like defaultSizes: 'gzip', compressionAlgorithm: 'brotli' which don't make sense. I personally would change defaultSizes: 'gzip' to defaultSizes: 'compressed' and deprecate the former, but support it (convert to compressed).

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current implementation handles the mismatch gracefully, so it's more of a cosmetic issue on the user side.

Reintroducing compressed would help here. If used, the config always looks "good", even when switching the algorithm. If we support compressed, fail on mismatches between compressionAlgorithm and defaultSizes?

|**`compressionAlgorithm`**|One of: `gzip`, `brotli`|Default: `gzip`. Compression type used to calculate the compressed module sizes.|
|**`openAnalyzer`**|`{Boolean}`|Default: `true`. Automatically open report in default browser.|
|**`generateStatsFile`**|`{Boolean}`|Default: `false`. If `true`, webpack stats JSON file will be generated in bundle output directory|
|**`statsFilename`**|`{String}`|Default: `stats.json`. Name of webpack stats JSON file that will be generated if `generateStatsFile` is `true`. It can be either an absolute path or a path relative to a bundle output directory (which is output.path in webpack config).|
Expand Down Expand Up @@ -111,23 +112,25 @@ Directory containing all generated bundles.
### `options`

```
-V, --version output the version number
-m, --mode <mode> Analyzer mode. Should be `server`, `static` or `json`.
In `server` mode analyzer will start HTTP server to show bundle report.
In `static` mode single HTML file with bundle report will be generated.
In `json` mode single JSON file with bundle report will be generated. (default: server)
-h, --host <host> Host that will be used in `server` mode to start HTTP server. (default: 127.0.0.1)
-p, --port <n> Port that will be used in `server` mode to start HTTP server. Should be a number or `auto` (default: 8888)
-r, --report <file> Path to bundle report file that will be generated in `static` mode. (default: report.html)
-t, --title <title> String to use in title element of html report. (default: pretty printed current date)
-s, --default-sizes <type> Module sizes to show in treemap by default.
Possible values: stat, parsed, gzip (default: parsed)
-O, --no-open Don't open report in default browser automatically.
-e, --exclude <regexp> Assets that should be excluded from the report.
Can be specified multiple times.
-l, --log-level <level> Log level.
Possible values: debug, info, warn, error, silent (default: info)
-h, --help output usage information
-V, --version output the version number
-m, --mode <mode> Analyzer mode. Should be `server`, `static` or `json`.
In `server` mode analyzer will start HTTP server to show bundle report.
In `static` mode single HTML file with bundle report will be generated.
In `json` mode single JSON file with bundle report will be generated. (default: server)
-h, --host <host> Host that will be used in `server` mode to start HTTP server. (default: 127.0.0.1)
-p, --port <n> Port that will be used in `server` mode to start HTTP server. Should be a number or `auto` (default: 8888)
-r, --report <file> Path to bundle report file that will be generated in `static` mode. (default: report.html)
-t, --title <title> String to use in title element of html report. (default: pretty printed current date)
-s, --default-sizes <type> Module sizes to show in treemap by default.
Possible values: stat, parsed, gzip, brotli (default: parsed)
--compression-algorithm <type> Compression algorithm that will be used to calculate the compressed module sizes.
Possible values: gzip, brotli (default: gzip)
-O, --no-open Don't open report in default browser automatically.
-e, --exclude <regexp> Assets that should be excluded from the report.
Can be specified multiple times.
-l, --log-level <level> Log level.
Possible values: debug, info, warn, error, silent (default: info)
-h, --help output usage information
```

<h2 align="center" id="size-definitions">Size definitions</h2>
Expand All @@ -151,6 +154,10 @@ as Uglify, then this value will reflect the minified size of your code.

This is the size of running the parsed bundles/modules through gzip compression.

### `brotli`

This is the size of running the parsed bundles/modules through Brotli compression.

<h2 align="center">Selecting Which Chunks to Display</h2>

When opened, the report displays all of the Webpack chunks for your project. It's possible to filter to a more specific list of chunks by using the sidebar or the chunk context menu.
Expand Down
23 changes: 15 additions & 8 deletions client/components/ModulesTreemap.jsx
Expand Up @@ -18,11 +18,17 @@ import Search from './Search';
import {store} from '../store';
import ModulesList from './ModulesList';

const SIZE_SWITCH_ITEMS = [
{label: 'Stat', prop: 'statSize'},
{label: 'Parsed', prop: 'parsedSize'},
{label: 'Gzipped', prop: 'gzipSize'}
];
function allSizeSwitchItems() {
const items = [
{label: 'Stat', prop: 'statSize'},
{label: 'Parsed', prop: 'parsedSize'}
];

if (window.compressionAlgorithm === 'gzip') items.push({label: 'Gzipped', prop: 'gzipSize'});
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would change gzipSize/brotliSize to compressedSize but looks like it will be a breaking change.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be nice if we could first do this as a new feature, release in a minor version and then do a major release later to consolidate the API on on plain compressedSize. That has been my opinion during this PR review at least ☺️.

What do you think of this plan?

if (window.compressionAlgorithm === 'brotli') items.push({label: 'Brotli', prop: 'brotliSize'});

return items;
}

@observer
export default class ModulesTreemap extends Component {
Expand Down Expand Up @@ -138,7 +144,7 @@ export default class ModulesTreemap extends Component {
renderModuleSize(module, sizeType) {
const sizeProp = `${sizeType}Size`;
const size = module[sizeProp];
const sizeLabel = SIZE_SWITCH_ITEMS.find(item => item.prop === sizeProp).label;
const sizeLabel = allSizeSwitchItems().find(item => item.prop === sizeProp).label;
const isActive = (store.activeSize === sizeProp);

return (typeof size === 'number') ?
Expand All @@ -162,7 +168,8 @@ export default class ModulesTreemap extends Component {
};

@computed get sizeSwitchItems() {
return store.hasParsedSizes ? SIZE_SWITCH_ITEMS : SIZE_SWITCH_ITEMS.slice(0, 1);
const items = allSizeSwitchItems();
return store.hasParsedSizes ? items : items.slice(0, 1);
}

@computed get activeSizeItem() {
Expand Down Expand Up @@ -316,7 +323,7 @@ export default class ModulesTreemap extends Component {
<br/>
{this.renderModuleSize(module, 'stat')}
{!module.inaccurateSizes && this.renderModuleSize(module, 'parsed')}
{!module.inaccurateSizes && this.renderModuleSize(module, 'gzip')}
{!module.inaccurateSizes && this.renderModuleSize(module, window.compressionAlgorithm)}
{module.path &&
<div>Path: <strong>{module.path}</strong></div>
}
Expand Down
2 changes: 1 addition & 1 deletion client/store.js
Expand Up @@ -4,7 +4,7 @@ import localStorage from './localStorage';

export class Store {
cid = 0;
sizes = new Set(['statSize', 'parsedSize', 'gzipSize']);
sizes = new Set(['statSize', 'parsedSize', 'gzipSize', 'brotliSize']);

@observable.ref allChunks;
@observable.shallow selectedChunks;
Expand Down
13 changes: 0 additions & 13 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 0 additions & 1 deletion package.json
Expand Up @@ -36,7 +36,6 @@
"acorn-walk": "^8.0.0",
"chalk": "^4.1.0",
"commander": "^7.2.0",
"gzip-size": "^6.0.0",
"lodash": "^4.17.20",
"opener": "^1.5.2",
"sirv": "^1.0.7",
Expand Down
4 changes: 4 additions & 0 deletions src/BundleAnalyzerPlugin.js
Expand Up @@ -12,6 +12,7 @@ class BundleAnalyzerPlugin {
this.opts = {
analyzerMode: 'server',
analyzerHost: '127.0.0.1',
compressionAlgorithm: 'gzip',
reportFilename: null,
reportTitle: utils.defaultTitle,
defaultSizes: 'parsed',
Expand Down Expand Up @@ -105,6 +106,7 @@ class BundleAnalyzerPlugin {
host: this.opts.analyzerHost,
port: this.opts.analyzerPort,
reportTitle: this.opts.reportTitle,
compressionAlgorithm: this.opts.compressionAlgorithm,
bundleDir: this.getBundleDirFromCompiler(),
logger: this.logger,
defaultSizes: this.opts.defaultSizes,
Expand All @@ -117,6 +119,7 @@ class BundleAnalyzerPlugin {
async generateJSONReport(stats) {
await viewer.generateJSONReport(stats, {
reportFilename: path.resolve(this.compiler.outputPath, this.opts.reportFilename || 'report.json'),
compressionAlgorithm: this.opts.compressionAlgorithm,
bundleDir: this.getBundleDirFromCompiler(),
logger: this.logger,
excludeAssets: this.opts.excludeAssets
Expand All @@ -128,6 +131,7 @@ class BundleAnalyzerPlugin {
openBrowser: this.opts.openAnalyzer,
reportFilename: path.resolve(this.compiler.outputPath, this.opts.reportFilename || 'report.html'),
reportTitle: this.opts.reportTitle,
compressionAlgorithm: this.opts.compressionAlgorithm,
bundleDir: this.getBundleDirFromCompiler(),
logger: this.logger,
defaultSizes: this.opts.defaultSizes,
Expand Down
14 changes: 8 additions & 6 deletions src/analyzer.js
Expand Up @@ -2,12 +2,12 @@ const fs = require('fs');
const path = require('path');

const _ = require('lodash');
const gzipSize = require('gzip-size');

const Logger = require('./Logger');
const Folder = require('./tree/Folder').default;
const {parseBundle} = require('./parseUtils');
const {createAssetsFilter} = require('./utils');
const {getCompressedSize} = require('./sizeUtils');

const FILENAME_QUERY_REGEXP = /\?.*$/u;
const FILENAME_EXTENSIONS = /\.(js|mjs)$/iu;
Expand All @@ -20,7 +20,8 @@ module.exports = {
function getViewerData(bundleStats, bundleDir, opts) {
const {
logger = new Logger(),
excludeAssets = null
excludeAssets = null,
compressionAlgorithm
} = opts || {};

const isAssetIncluded = createAssetsFilter(excludeAssets);
Expand Down Expand Up @@ -102,7 +103,7 @@ function getViewerData(bundleStats, bundleDir, opts) {

if (assetSources) {
asset.parsedSize = Buffer.byteLength(assetSources.src);
asset.gzipSize = gzipSize.sync(assetSources.src);
asset[`${compressionAlgorithm}Size`] = getCompressedSize(compressionAlgorithm, assetSources.src);
}

// Picking modules from current bundle script
Expand Down Expand Up @@ -143,7 +144,7 @@ function getViewerData(bundleStats, bundleDir, opts) {
}

asset.modules = assetModules;
asset.tree = createModulesTree(asset.modules);
asset.tree = createModulesTree(asset.modules, {compressionAlgorithm});
return result;
}, {});

Expand All @@ -157,6 +158,7 @@ function getViewerData(bundleStats, bundleDir, opts) {
statSize: asset.tree.size || asset.size,
parsedSize: asset.parsedSize,
gzipSize: asset.gzipSize,
brotliSize: asset.brotliSize,
groups: _.invokeMap(asset.tree.children, 'toChartData')
}));
}
Expand Down Expand Up @@ -203,8 +205,8 @@ function isRuntimeModule(statModule) {
return statModule.moduleType === 'runtime';
}

function createModulesTree(modules) {
const root = new Folder('.');
function createModulesTree(modules, opts) {
const root = new Folder('.', opts);

modules.forEach(module => root.addModule(module));
root.mergeNestedFolders();
Expand Down
23 changes: 20 additions & 3 deletions src/bin/analyzer.js
Expand Up @@ -10,7 +10,9 @@ const viewer = require('../viewer');
const Logger = require('../Logger');
const utils = require('../utils');

const SIZES = new Set(['stat', 'parsed', 'gzip']);
const SIZES = new Set(['stat', 'parsed', 'gzip', 'brotli']);

const ALGORITHMS = new Set(['gzip', 'brotli']);

const program = commander
.version(require('../../package.json').version)
Expand Down Expand Up @@ -58,6 +60,12 @@ const program = commander
br(`Possible values: ${[...SIZES].join(', ')}`),
'parsed'
)
.option(
'--compression-algorithm <type>',
'Compression algorithm that will be used to calculate the compressed module sizes.' +
br(`Possible values: ${[...ALGORITHMS].join(', ')}`),
'gzip'
)
.option(
'-O, --no-open',
"Don't open report in default browser automatically."
Expand All @@ -84,6 +92,7 @@ let {
report: reportFilename,
title: reportTitle,
defaultSizes,
compressionAlgorithm,
logLevel,
open: openBrowser,
exclude: excludeAssets
Expand All @@ -104,7 +113,12 @@ if (mode === 'server') {
port = port === 'auto' ? 0 : Number(port);
if (isNaN(port)) showHelp('Invalid port. Should be a number or `auto`');
}
if (!SIZES.has(defaultSizes)) showHelp(`Invalid default sizes option. Possible values are: ${[...SIZES].join(', ')}`);
if (!SIZES.has(defaultSizes)) {
showHelp(`Invalid default sizes option. Possible values are: ${[...SIZES].join(', ')}`);
}
if (!ALGORITHMS.has(compressionAlgorithm)) {
showHelp(`Invalid compression algorithm option. Possible values are: ${[...ALGORITHMS].join(', ')}`);
}

bundleStatsFile = resolve(bundleStatsFile);

Expand All @@ -125,6 +139,7 @@ if (mode === 'server') {
port,
host,
defaultSizes,
compressionAlgorithm,
reportTitle,
bundleDir,
excludeAssets,
Expand All @@ -137,13 +152,15 @@ if (mode === 'server') {
reportFilename: resolve(reportFilename || 'report.html'),
reportTitle,
defaultSizes,
compressionAlgorithm,
bundleDir,
excludeAssets,
logger: new Logger(logLevel)
});
} else if (mode === 'json') {
viewer.generateJSONReport(bundleStats, {
reportFilename: resolve(reportFilename || 'report.json'),
compressionAlgorithm,
bundleDir,
excludeAssets,
logger: new Logger(logLevel)
Expand All @@ -157,7 +174,7 @@ function showHelp(error) {
}

function br(str) {
return `\n${' '.repeat(28)}${str}`;
return `\n${' '.repeat(32)}${str}`;
}

function array() {
Expand Down
24 changes: 24 additions & 0 deletions src/sizeUtils.js
@@ -0,0 +1,24 @@
const zlib = require('zlib');

const COMPRESSED_SIZE = {
gzip: gzipSize,
brotli: brotliSize
};

export function getCompressedSize(compressionAlgorithm, input) {
const fn = COMPRESSED_SIZE[compressionAlgorithm];
if (!fn) throw new Error(`Unsupported compression algorithm: ${compressionAlgorithm}.`);
return fn(input);
}

function gzipSize(input) {
return zlib.gzipSync(input, {level: 9}).length;
}

function brotliSize(input) {
if (typeof zlib.brotliCompressSync !== 'function') {
throw new Error('Brotli compression requires Node.js v10.16.0 or higher.');
}
dcsaszar marked this conversation as resolved.
Show resolved Hide resolved

return zlib.brotliCompressSync(input).length;
}
3 changes: 2 additions & 1 deletion src/template.js
Expand Up @@ -39,7 +39,7 @@ function getScript(filename, mode) {
}
}

function renderViewer({title, enableWebSocket, chartData, defaultSizes, mode} = {}) {
function renderViewer({title, enableWebSocket, chartData, defaultSizes, compressionAlgorithm, mode} = {}) {
return html`<!DOCTYPE html>
<html>
<head>
Expand All @@ -59,6 +59,7 @@ function renderViewer({title, enableWebSocket, chartData, defaultSizes, mode} =
<script>
window.chartData = ${escapeJson(chartData)};
window.defaultSizes = ${escapeJson(defaultSizes)};
window.compressionAlgorithm = ${escapeJson(compressionAlgorithm)};
</script>
</body>
</html>`;
Expand Down
4 changes: 2 additions & 2 deletions src/tree/ConcatenatedModule.js
Expand Up @@ -7,8 +7,8 @@ import {getModulePathParts} from './utils';

export default class ConcatenatedModule extends Module {

constructor(name, data, parent) {
super(name, data, parent);
constructor(name, data, parent, opts) {
super(name, data, parent, opts);
this.name += ' (concatenated)';
this.children = Object.create(null);
this.fillContentModules();
Expand Down