Node-API

[tmikov]

(post by @shirakaba moved to top level from #1072 (comment))

Node-API

Node-API was originally just an API for building native addons for Node.js, but has since been decoupled from Node.js (despite the name), and now serves as a universal standard for interacting with native code from any JS engine that implements Node-API (which is most of them).

Contrast to JSI

It may seem that JSI fulfils this need already. However, Node-API has a much wider feature scope, providing a broad range of low-level APIs much like the V8 APIs. It's so flexible that NodeGUI uses it to create idiomatic JS bindings to the whole of Qt, for example.

It also has a much better ecosystem story than JSI – today, many packages containing Node-API code are distributed through npm. These packages can be run directly by any runtime that implements Node-API, e.g. Node.js, Deno, or Bun. They're also easy to run headless (unlike most JSI libraries, which are only set up to be run via a React Native app), allowing for rapid development (no need to spin up a Android/iOS simulator).

Unlike JSI with its C++-based API, Node-API has an ABI-stable C API, meaning native modules using Node-API could be safely distributed as precompiled binaries, allowing for faster builds. I understand this could be solved by rewriting the JSI internals in C, of course.

Contrast to WebAssembly

This discussion is a bit of a distraction, so feel free to skip it.

See details

Putting aside the fact that WASM isn't supported in Hermes at present (#395 (comment)), let's compare the two.

There is some overlap in use-cases between Node-API and WebAssembly, but Node-API is infinitely more flexible because it is not sandboxed. While WebAssembly can be used to compile a self-contained library (one that you have all the source code for, and no dependencies beyond the C standard library) for use in JS, it cannot access system libraries like Core Foundation, which is my main interest.

An example of a Node-API module

For the curious, here's what Node-API involves. The full docs are here.

See details

Library developers would commonly write a Node-API module in C or C++, though there are other language bindings to it, too. Here's an example of a basic module written in C++:

#include <napi.h>

using namespace Napi;

Napi::String Method(const Napi::CallbackInfo& info) {
  Napi::Env env = info.Env();
  return Napi::String::New(env, "world");
}

Napi::Object Init(Napi::Env env, Napi::Object exports) {
  exports.Set(Napi::String::New(env, "HelloWorld"),
              Napi::Function::New(env, Method));
  return exports;
}

NODE_API_MODULE(addon, Init)

From userland, it is consumed just like any other Node module:

const { HelloWorld } = require('./build/Release/hello-world-native.node');

HelloWorld();

Concrete use-case

We (the NativeScript TSC) would like to bring NativeScript's ability to access native APIs to other JS engines/runtimes. In particular, we'd like to bring it to Hermes so that React Native could access the whole iOS/Android SDK directly from JS.

For an example of how much time it could save compared to writing a native module, here's what it's like to access the iOS batteryLevel API via JavaScript:

// Original Objective-C API:
float batteryLevel = [[UIDevice currentDevice] batteryLevel];

// Equivalent JavaScript API (in NativeScript):
const batteryLevel = UIDevice.currentDevice.batteryLevel;

No need to open Xcode and make a native module, no need to recompile any native code, and no need to write any bindings (all bindings to the iOS SDK are ready-generated and fully typed).

To make NativeScript embeddable into other JS engines/runtimes, we've been rewriting it using Node-API. It's already successfully working using @vmoroz's fork of Hermes for React Native Windows (that adds Node-API support), so if only Hermes could support Node-API officially – whether in its core or via a plugin – we could bring it to React Native iOS/Android as well.

We hope this could become the last native module ever written 🥹

[tmikov]

@shirakaba thank you for posting this detailed and thoughtful question.

I agree with pretty much everything you said and I think that enabling code written for Node-API to work with Hermes would be beneficial for the ecosystem. I have expressed a similar sentiment in our discussions with @vmoroz.

Can Node-API Be Maintained by Hermes

Really, the core question that must be answered first is whether the Node-API implementation can be be maintained by the Hermes team.

We have discussed that question at length internally and our conclusion is that at this time it wouldn't be practical. We have no expertise in Node-API; more importantly, we have no internal usages of it. It is very difficult to maintain something that you don't use. We wouldn't be able to diagnose problems well, answer questions, etc. At the same time, having to keep Node-API working while changing our internal API would be a continuous burden.

Unfortunately it also looks like neither JSI nor Node-API can be implemented efficiently on top of the other one, to save some work. There need to be separate implementations, existing in parallel, simultaneously accessing the same Hermes runtime using the internal APIs.

Sincere question (since I haven't followed this topic closely): are Node-API implementations for other JavaScript engines maintained by the engines themselves, or by separate organizations?

Are Other Solutions Possible

With all of that said, I am hopeful that we can find a satisfactory solution. I see three separate issues that need to be solved:

1. Maintenance. How hard would it be for a separate entity to maintain Hermes/Node-API in the face of constantly changing Hermes?
2. Coexistence. JSI assumes that it is the only API accessing Hermes. If there is another parallel API, JSI will need to be changed.
3. Distribution. How can a Hermes "plugin" accessing internal APIs (which aren't even exported by the shared library) be distributed?

I believe that the first issue is solved by a change in the Hermes release model that we will be making soon. Hermes will transition to a fixed six month release cadence, where the just released version is frozen and stable. This means that a Node-API implementation doesn't need to track a continuously changing target, but instead one changing at a fixed and predictable times.

We are aware of the second issue and I am hoping that we can change JSI can to support that. @neildhar can you comment on this topic?

The last issue is really the most interesting and challenging, particularly the "symbols not exported by the .so" part. It seems like this would have to be an additional Hermes target, build from the Hermes source + the Node API plugin source. I think we perhaps could add extension points for this in the Hermes build scripts? Any ideas on this issue (and not only, obviously) are welcome.

[neildhar]

On the second point, I think the necessary changes would be in the Hermes implementation of JSI rather than in JSI itself. For instance, we would need to make sure that we can distinguish native functions created through JSI and through N-API in Hermes internals. Aside from introducing a new CellKind, a simple solution could be to add a wrapper layer that contains an enum describing where it came from, or to add an internal property providing that information.

[shirakaba]

I'm not an expert, just a cheerleader, but will try to advance the conversation here nonetheless.

Sincere question (since I haven't followed this topic closely): are Node-API implementations for other JavaScript engines maintained by the engines themselves, or by separate organizations?

With the possible exception of the Node-API implementation for Node.js (which of course probably was made by Node.js core members in the first place), I believe all Node-API implementations are maintained by separate organisations rather than the engine-makers themselves.

1. Maintenance. How hard would it be for a separate entity to maintain Hermes/Node-API in the face of constantly changing Hermes?

In other JS engines, I've seen that NAPI is usually implemented as a separate add-on rather than by forking the engine. In this repo, we can see NAPI implementations for ark, jerryscript, quickjs and v8, and they all operate without having to fork the given engine.

This approach may suit the Hermes team by keeping the maintenance burden off them. As long as some key VM APIs remain stable, I imagine the maintenance burden should be low.

However, my impression is that an addon approach is not possible in Hermes today because a few necessary headers are kept private. For reference, here are the headers included by hermes_napi.cpp in @vmoroz's fork of Hermes, which implements NAPI by forking:

#define NAPI_VERSION 8
#define NAPI_EXPERIMENTAL

#include "napi/hermes_napi.h"

#include "hermes/BCGen/HBC/BytecodeProviderFromSrc.h"
#include "hermes/DebuggerAPI.h"
#include "hermes/SourceMap/SourceMapParser.h"
#include "hermes/Support/SimpleDiagHandler.h"
#include "hermes/VM/Callable.h"
#include "hermes/VM/HostModel.h"
#include "hermes/VM/JSArray.h"
#include "hermes/VM/JSArrayBuffer.h"
#include "hermes/VM/JSDataView.h"
#include "hermes/VM/JSDate.h"
#include "hermes/VM/JSError.h"
#include "hermes/VM/JSProxy.h"
#include "hermes/VM/JSTypedArray.h"
#include "hermes/VM/PropertyAccessor.h"
#include "hermes/VM/Runtime.h"
#include "llvh/ADT/SmallVector.h"
#include "llvh/Support/ConvertUTF.h"

#include <algorithm>
#include <atomic>
#include <unordered_map>

Checking the `CMakeLists, most of these are not public:

hermes/API/hermes/CMakeLists.txt

Lines 32 to 33 in 329430e

	file(GLOB api_headers ${CMAKE_CURRENT_SOURCE_DIR}/*.h)
	file(GLOB api_public_headers ${PROJECT_SOURCE_DIR}/public/hermes/Public/*.h)

Currently the only headers in the Public directory are:

Buffer.h
CrashManager.h
CtorConfig.h
DebuggerTypes.h
GCConfig.h
GCTripwireContext.h
HermesExport.h
JSOutOfMemoryError.h
RuntimeConfig.h

I'd need to do a deep-dive to assess what these headers re-export, but for starters, I don't see any re-exports of any of the hermes/VM APIs inside them.

2. Coexistence. JSI assumes that it is the only API accessing Hermes. If there is another parallel API, JSI will need to be changed.

Beyond my expertise, so I defer to @neildhar (above) and @vmoroz here. Though I must say that nothing has broken in my limited proof-of-concept usages of this NAPI fork in React Native. Microsoft are using it in production for React Native Windows, so whatever Vladimir has come up with seems to work.

3. Distribution. How can a Hermes "plugin" accessing internal APIs (which aren't even exported by the shared library) be distributed?

As mentioned, I think we would need the above-used APIs exposed, otherwise it's simply game over. In addition, we will need the plugin to be able to run a block during the HermesExecutorFactory initialisation process because (to my limited understanding) some NAPI libraries need to integrate with the Hermes VM before GC is set up.

Concretely, in our proof-of-concept of embedding NAPI-based NativeScript into React Native (via Hermes), we make the following changes:

#include <hermes/inspector/RuntimeAdapter.h>
#include <hermes/inspector/chrome/Registration.h>

#include "JSITracing.h"

+ // NativeScript START
+ #include <hermes/hermes_api.h>
+
+ extern "C" napi_value objc_bridge_init(napi_env env, napi_value exports);
+
+ #define GUARD(expr) \
+ status = expr; \
+ if (status != napi_ok) { \
+ @throw([NSException exceptionWithName:@"NapiRegistrationFailedException" reason: [NSString stringWithFormat:@"NAPI call returned error: %d", status] userInfo:nil]); \
+ }
+ // NativeScript END

std::unique_ptr<JSExecutor> HermesExecutorFactory::createJSExecutor(
  {
    SystraceSection s("makeHermesRuntime");
    hermesRuntime = hermes::makeHermesRuntime(runtimeConfig_);
+
+   // NativeScript START
+   napi_env env;
+   hermesRuntime->createNapiEnv(&env);
+
+   napi_status status;
+   jsr_napi_env_scope env_scope;
+   GUARD(jsr_open_napi_env_scope(env, &env_scope));
+
+   napi_value exports = nil;
+   GUARD(napi_create_object(env, &exports));
+
+   objc_bridge_init(env, exports);
+   // NativeScript END
  }

If HermesExecutorFactory could expose some hook that would allow libraries to run a code block during createJSExecutor. At its crudest, maybe a static block that by default is a no-op, but could be overwritten by a plugin..? Or alternatively, populated during AppDelegate.

Ideally it would be nice to be able to set up this stuff via a JSI library, but it sounds like JSI is exposed to userland too late for that setup to occur (indeed, I tried and got nowhere).

[tmikov]

As long as some key VM APIs remain stable, I imagine the maintenance burden should be low.

Unfortunately that is a commitment that we cannot make - that's one of the primary reasons why there is a distinction between JSI APIs and VM APIs. However, with the release of Static Hermes we will transition to a 6-month release schedule, so at the very least whatever porting work is necessary will be bounded.

Checking the `CMakeLists, most of these are not public: [...]

The distinction between public and non-public headers is not really meaningful in practice. The headers are right there, on disk. Nothing prevents other code from including them.

The distinction however serves to indicate which APIs can change arbitrarily in non-backwards compatible ways, so in that sense it is useful.

Beyond my expertise, so I defer to @neildhar (above) and @vmoroz here. Though I must say that nothing has broken in my limited proof-of-concept usages of this NAPI fork in React Native. Microsoft are using it in production for React Native Windows, so whatever Vladimir has come up with seems to work.

Hmm. I would wait for @vmoroz to comment here, but I don't see how it could work without non-trivial modifications to JSI. It may appear to work, but it would fail in subtle ways when objects created by one are passed to the other, etc (@neildhar can you comment with examples).

I think it would work reliably if it was using n-api and JSI over n-api (which is slower), but definitely not when using "regular" JSI and n-api.

The fact of the matter is, adding n-api requires modifying JSI for correctness. I am not opposed to that in principle, and perhaps we can figure elegant ways to do it, but it is something that must be kept in mind, and is, unfortunately a complication.

Exporting Internal APIs

On the topic if exporting internal APIs from the shared library. We can't really do that - that's why these APIs are internal. Exporting every single internal API only because some "plugin" might potentially use it is not a very rational strategy and, worse of all, it prevents optimizations and it increases the size of the binary with unnecessary strings.

I think a different model might make more sense and solve all of these problems. Before Hermes is turned into a shared library, it exists as a regular static library (.a). We could introduce the concept of a plugin that is also a static library, and is introduced at that stage of the process. This plugin then gets linked together with libhermes_a.a into the resulting shared library. So, it has access to all private APIs without forcing them to become public.

In this way n-api can be "injected" into the Hermes build process.

The other point you raised - a hook during initialization - is not a problem. I have no objection to adding something like that.

Really, the only real problem is the modification of JSI to co-exist with different APIs. This is non-trivial and if it means slowing down JSI it could turn into a significant blocker.

[neildhar]

NAPI is usually implemented as a separate add-on rather than by forking the engine

Note that it should be possible to implement much of NAPI on top of JSI, which is Hermes' public API. Or if ABI stability is required, the translation layer can be written on top of the Hermes C-API that is currently being worked on. That is closer in spirit to how NAPI is implemented in other engines, since it is built on top of their public API.

The performance will be worse than implementing NAPI directly on Hermes internals, but it would be stable and can be used to support NAPI extensions. It would be interesting to characterise the performance impact of such an approach, since it would resolve a lot of these issues, including seamlessly allowing NAPI and JSI to operate in parallel.

can you comment with examples

If we have both NAPI and JSI built directly on top of Hermes internals, there are a few internal assumptions that would break. These mostly relate to the representation of things like HostObject, HostFunction, and NativeState in the JSI implementation for Hermes.

For instance, the implementations of isHostFunction and getHostFunction assume that any vm::FinalizableNativeFunction originates in our JSI layer and can be used to obtain a jsi::HostFunction. If both NAPI and JSI coexist, we will need to introduce some other mechanism to distinguish their origin.

[tmikov]

After much deliberation, we now think we should expose it directly from Hermes. So, we can change our minds on things like this, just give us time!

Ideally we would like to reuse and import @vmoroz's implementation. Some modifications will be necessary to avoid the problems @neildhar was describing above, but we think they will be relatively minor and we can take care of them.