Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Require for native add-ons in worker_threads #34

Open
tshemsedinov opened this issue Feb 14, 2019 · 6 comments
Open

Require for native add-ons in worker_threads #34

tshemsedinov opened this issue Feb 14, 2019 · 6 comments
Labels

Comments

@tshemsedinov
Copy link
Member

Some modules, for example argon2 can't be loaded in worker threads.

Try load.js with the following code:

'use strict';

const { Worker } = require('worker_threads');

const argon2 = require('argon2');
console.dir({ argon2 });

new Worker('./thread.js');

and thread.js with the following code:

'use strict';

const argon2 = require('argon2');

console.dir({ argon2 });

we will get exception:

Error: Module did not self-register.
    at Object.Module._extensions..node (internal/modules/cjs/loader.js:775:18)
    at Module.load (internal/modules/cjs/loader.js:626:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:566:12)
    at Function.Module._load (internal/modules/cjs/loader.js:558:3)
    at Module.require (internal/modules/cjs/loader.js:663:17)
    at require (internal/modules/cjs/helpers.js:20:18)
    at bindings (.../node_modules/bindings/bindings.js:112:48)
    at Object.<anonymous> (.../node_modules/argon2/argon2.js:4:37)
    at Module._compile (internal/modules/cjs/loader.js:734:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:745:10)

We need to find solution to use worker thread in impress. @nechaido you can assign this task to developers.

@tshemsedinov
Copy link
Member Author

After resolving this problem we can use solution here: metarhia/impress#1099

@lundibundi
Copy link
Member

lundibundi commented Feb 15, 2019

worker_threads doesn't support native addons as of 11.9.0
"Native addons are not supported yet."
therefore it won't be possible to require('argon2') in them as npm argon2 module is just bindings to the C argon library.

Edit: after additional testing, it looks like you can safely run the above code on node 11.7.0+
Edit2: --//-- not in multiple threads though

@tshemsedinov
Copy link
Member Author

It works in 11.9.0 if you require from a single worker, but fails if you require from master process and 1 worker or from more then 1 workers. We need at least: mdsf, websocket, argon2

@lundibundi
Copy link
Member

Refs: nodejs/node#26175

@tshemsedinov tshemsedinov transferred this issue from metarhia/impress Mar 2, 2019
@tshemsedinov tshemsedinov changed the title Modules load problem in workers Require for native add-ons in worker_threads Mar 2, 2019
@tshemsedinov
Copy link
Member Author

Refs: nodejs/node#23319
Refs: nodejs/node#21481

@lundibundi
Copy link
Member

lundibundi commented Jul 29, 2019

Starting from Node.js 11.11.0 (nodejs/node#26175) it should be possible to use native addons from multiple worker threads as long as they are

  • an N-API addon, or
  • declared as context-aware using NODE_MODULE_INIT() as described above

(https://nodejs.org/api/addons.html#addons_worker_support)

Where the latter basically means that the addon properly manages its global/static state to protect it against race conditions, clean up its data when instance shutdowns, doesn't store persistent JS object references and doesn't access JS objects from the wrong context (not the one they were created in). Therefore using NODE_MODULE_INIT instead of NODE_MODULE tells Node.js that the module respects the given rules.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants