Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add benchmarks #101

Merged
merged 8 commits into from Aug 23, 2019
Merged

Conversation

chrisblossom
Copy link
Contributor

This is not necessarily meant to be merged as is, but meant to try and answer #60.

benchmark.js Outdated Show resolved Hide resolved
benchmark.js Outdated
// https://github.com/bestiejs/benchmark.js/issues/136
createFixtures();

// Async await was giving too many errors. stick with standard promises
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Huh? What does this mean? Async/await shouldn't work any differently.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suite.add({
	name,
	defer: true,
    // if you remove setup() the benchmark works as expected
	setup() {
	},
	async fn(deferred) {
		// Can't use setup because it isn't called after every defer
		// https://github.com/bestiejs/benchmark.js/issues/136
		createFixtures();

		// Async await was giving too many errors. stick with standard promises
		await del(['**/*'], {
			cwd: tempDir,
			concurrency,
			// eslint-disable-next-line promise/prefer-await-to-then
		}).then(removedFiles => {
			if (removedFiles.length !== fixtures.length) {
				const error = new Error(
					`"${name}": files removed: ${removedFiles.length}, expected: ${fixtures.length}`,
				);

				console.error(error);

				del.sync(tempDir, { cwd: tempDir, force: true });

				// eslint-disable-next-line unicorn/no-process-exit
				process.exit(1);
			}

			deferred.resolve();
		});
	},
});

Gives the error:

del on  add-benchmarks [⇡!] is 📦 v5.0.0 via ⬢ v10.16.0 took 6s
➜ node benchmark.js
undefined:9
			await del(['**/*'], {
			^^^^^

SyntaxError: await is only valid in async function
    at Function (<anonymous>)
    at createCompiled (/Users/chris/github/del/node_modules/benchmark/benchmark.js:1725:16)
    at clock (/Users/chris/github/del/node_modules/benchmark/benchmark.js:1608:58)
    at clock (/Users/chris/github/del/node_modules/benchmark/benchmark.js:1818:20)
    at new Deferred (/Users/chris/github/del/node_modules/benchmark/benchmark.js:405:7)
    at Deferred (/Users/chris/github/del/node_modules/benchmark/benchmark.js:402:16)
    at Benchmark.run (/Users/chris/github/del/node_modules/benchmark/benchmark.js:2112:13)
    at execute (/Users/chris/github/del/node_modules/benchmark/benchmark.js:860:74)
    at Timeout._onTimeout (/Users/chris/github/del/node_modules/lodash/lodash.js:2750:43)
    at ontimeout (timers.js:436:11)

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Weird. I already commented on this, but it's gone now... Anyway, I think you're just missing the async keyword for the method. Then it worked for me at least.

@sindresorhus
Copy link
Owner

Results for me on Node.js 12.6.0 and macOS 10.14:

concurrency: 1 x 2.09 ops/sec ±11.43% (15 runs sampled)
concurrency: 3 x 2.86 ops/sec ±8.24% (18 runs sampled)
concurrency: 5 x 3.18 ops/sec ±14.31% (19 runs sampled)
concurrency: 10 x 3.60 ops/sec ±9.86% (22 runs sampled)
concurrency: 15 x 3.77 ops/sec ±7.95% (23 runs sampled)
concurrency: 20 x 4.05 ops/sec ±9.13% (25 runs sampled)
concurrency: 50 x 4.03 ops/sec ±9.58% (25 runs sampled)
concurrency: 100 x 3.67 ops/sec ±10.94% (23 runs sampled)
concurrency: Infinity x 3.84 ops/sec ±6.76% (23 runs sampled)
Fastest is concurrency: 20,concurrency: 50,concurrency: 15,concurrency: 100

How about you?

@chrisblossom
Copy link
Contributor Author

chrisblossom commented Jul 15, 2019

On my desktop:

del on  add-benchmarks is 📦 v5.0.0 via ⬢ v12.6.0 took 16s
➜ npx envinfo --system --binaries
npx: installed 1 in 0.753s

  System:
    OS: macOS 10.14.5
    CPU: (16) x64 Intel(R) Xeon(R) W-2140B CPU @ 3.20GHz
    Memory: 2.31 GB / 32.00 GB
    Shell: 5.7.1 - /usr/local/bin/zsh
  Binaries:
    Node: 12.6.0 - ~/.nvm/versions/node/v12.6.0/bin/node
    Yarn: 1.17.0 - /usr/local/bin/yarn
    npm: 6.9.0 - ~/.nvm/versions/node/v12.6.0/bin/npm


del on  add-benchmarks is 📦 v5.0.0 via ⬢ v12.6.0
➜ node benchmark.js
concurrency: 1 x 3.07 ops/sec ±2.17% (20 runs sampled)
concurrency: 3 x 5.14 ops/sec ±1.15% (29 runs sampled)
concurrency: 5 x 6.10 ops/sec ±0.55% (34 runs sampled)
concurrency: 10 x 5.67 ops/sec ±2.16% (31 runs sampled)
concurrency: 15 x 6.42 ops/sec ±0.78% (35 runs sampled)
concurrency: 20 x 6.45 ops/sec ±0.42% (35 runs sampled)
concurrency: 50 x 6.41 ops/sec ±0.96% (35 runs sampled)
concurrency: 100 x 6.76 ops/sec ±0.70% (37 runs sampled)
concurrency: Infinity x 6.08 ops/sec ±0.51% (34 runs sampled)
Fastest is concurrency: 100

On my laptop:

➜  del git:(add-benchmarks) npx envinfo --system --binaries
npx: installed 1 in 0.981s

  System:
    OS: macOS 10.14.5
    CPU: (12) x64 Intel(R) Core(TM) i9-8950HK CPU @ 2.90GHz
    Memory: 11.44 GB / 32.00 GB
    Shell: 5.7.1 - /usr/local/bin/zsh
  Binaries:
    Node: 12.6.0 - ~/.nvm/versions/node/v12.6.0/bin/node
    Yarn: 1.17.0 - /usr/local/bin/yarn
    npm: 6.9.0 - ~/.nvm/versions/node/v12.6.0/bin/npm

➜  del git:(add-benchmarks) node benchmark.js
concurrency: 1 x 2.09 ops/sec ±3.55% (15 runs sampled)
concurrency: 3 x 3.08 ops/sec ±5.33% (19 runs sampled)
concurrency: 5 x 3.51 ops/sec ±7.06% (22 runs sampled)
concurrency: 10 x 3.97 ops/sec ±3.88% (24 runs sampled)
concurrency: 15 x 4.29 ops/sec ±2.36% (25 runs sampled)
concurrency: 20 x 3.90 ops/sec ±4.75% (24 runs sampled)
concurrency: 50 x 4.11 ops/sec ±6.78% (26 runs sampled)
concurrency: 100 x 4.74 ops/sec ±1.04% (27 runs sampled)
concurrency: Infinity x 4.24 ops/sec ±0.78% (25 runs sampled)
Fastest is concurrency: 100

@chrisblossom
Copy link
Contributor Author

Also, if you have any ideas how to remove createFixtures() from the benchmark I'm all ears. Only way I can think of to do this is to remove benchmark.js and manually calculate each iteration.

@sindresorhus
Copy link
Owner

sindresorhus commented Aug 18, 2019

Seems like 100 would be a good default. Maybe you could add 200, 300, 400, 500, 1000 to the benchmark too? Just to check.

@chrisblossom
Copy link
Contributor Author

chrisblossom commented Aug 22, 2019

Seems like 100 would be a good default. Maybe you could add 200, 300, 400, 500, 1000 to the benchmark too? Just to check.

Added!

From my laptop:

concurrency: 1 x 3.04 ops/sec ±2.91% (19 runs sampled)
concurrency: 3 x 4.84 ops/sec ±0.86% (28 runs sampled)
concurrency: 5 x 5.35 ops/sec ±1.81% (30 runs sampled)
concurrency: 10 x 5.16 ops/sec ±2.28% (29 runs sampled)
concurrency: 15 x 5.30 ops/sec ±2.94% (30 runs sampled)
concurrency: 20 x 5.57 ops/sec ±1.64% (31 runs sampled)
concurrency: 50 x 5.79 ops/sec ±0.86% (32 runs sampled)
concurrency: 100 x 5.87 ops/sec ±1.46% (32 runs sampled)
concurrency: 200 x 6.06 ops/sec ±0.93% (33 runs sampled)
concurrency: 300 x 6.10 ops/sec ±1.94% (34 runs sampled)
concurrency: 400 x 6.11 ops/sec ±0.83% (34 runs sampled)
concurrency: 500 x 6.01 ops/sec ±0.91% (33 runs sampled)
concurrency: 1000 x 5.72 ops/sec ±3.55% (33 runs sampled)
concurrency: Infinity x 5.49 ops/sec ±0.59% (31 runs sampled)
Fastest is concurrency: 400,concurrency: 200,concurrency: 300

benchmark.js Outdated Show resolved Hide resolved
@chrisblossom
Copy link
Contributor Author

I've resolved the merge conflict and updated to async await. I added the line that demonstrates why I was using normal promises. Let me know if you want me to revert the async await change or remove the setup() {} line.

@sindresorhus sindresorhus merged commit 9c72270 into sindresorhus:master Aug 23, 2019
@chrisblossom chrisblossom deleted the add-benchmarks branch August 26, 2019 16:53
@chrisblossom
Copy link
Contributor Author

Are you able to run node benchmark.js without errors?

setup() {}, // This line breaks async await

Is causing the following error for me:

 node benchmark.js
undefined:9
			const removedFiles = await del(['**/*'], {
			                     ^^^^^

SyntaxError: await is only valid in async function
    at Function (<anonymous>)
    at createCompiled (/Users/chris/github/del/node_modules/benchmark/benchmark.js:1725:16)

@sindresorhus
Copy link
Owner

I just forgot to remove the setup method before merging as I had intended to: 728cb7c

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants