Best way to dynamically link wasm modules?

[alianse777]

Summary

Lets say I have module A which requires some_util_fn() and arbitrary module B which exports some_util_fn().
Whats the best way to load and link B with A during instantiation so A can call B's some_util_fn() or other exports, without knowing exact names and signatures during development?

[Michael-F-Bryan]

It's a bit clunky, but you should be able to do something like this...

First, we instantiate module A (a.wasm) like normal.

let store = Store::default();
let module_a = Module::new(&store, std::fs::read("a.wasm")?)?;
let instance_a = Instance::new(&module_a, &ImportObject::new())?;

We also want to compile b.wasm, but can't instantiate it yet.

let module_b = Module::new(&store, std::fs::read("b.wasm")?)?;

Now, to make things simpler we'll assume that the items exported by a.wasm satisfy a single namespace as imported by b.wasm. That assumption lets us group B's imports by name.

let mut imports_by_namespace: HashMap<String, HashMap<String, ExternType>> = HashMap::new();

for import in module_b.imports() {
    imports_by_namespace
        .entry(import.name().to_string())
        .or_default()
        .insert(import.name().to_string(), import.ty().clone());
}

Now, we need to figure out which namespace a.wasm should provide. In theory, the functions from a.wasm should provide a superset of the namespace's functions.

let (namespace, _) = imports_by_namespace
    .iter()
    .find(|(_, imported_items)| is_satisfied_by_exports(imported_items, &instance_a.exports))
    .expect("Unable to figure out which namespace \"a.wasm\" would satisfy");

fn is_satisfied_by_exports(
    required_items: &HashMap<String, ExternType>,
    exports: &Exports,
) -> bool {
    for (name, import_ty) in required_items {
        let export_ty = match exports.get_extern(name) {
            Some(item) => item.ty(),
            None => return false,
        };

        if export_ty != *import_ty {
            // I would probably treat matching name with mismatched signatures
            // as an error because it often indicates a subtle bug (i.e.
            // "a.wasm" provides `add: fn(i32, i32) -> i32` while "b.wasm"
            // provides `add: fn(f32, f32) -> f32`), but that's up to you.
            return false;
        }
    }

    true
}

We now know the name of the namespace we're providing so we can set up B's imports object and instantiate it.

let mut imports = ImportObject::new();
imports.register(namespace, instance_a.exports.clone());

let instance_b = Instance::new(&module_b, &imports)?;

I'm used to working with multiple modules where each module provides its own namespace, but you should be able to adjust that code to suit your setup. For example, if you actually had modules X, Y, and Z all contributing functions to the env namespace¹ as imported by module B, you would skip the grouping stuff and is_satisfied_by_exports(), and do something like this:

let available_dependencies = vec![instantiate("x.wasm"), instantiate("y.wasm"), instantiate("z.wasm")];
let module_b = Module::new(&store, std::fs::read("b.wasm")?)?;

let mut env: HashMap<String, ExternType> = HashMap::new();

for import in module_b.imports() {
  let name = import.name();
  let export = available_dependencies.iter()
    .find_map(|dep| dep.exports.get_extern(name).cloned())
    .expect("TODO: handle unsatisfied imports");
  env.insert(name.to_string(), export);
}

let imports = wasmer::imports! { "env" => env };
let instance_b = Instance::new(&module_b, &imports)?;

I haven't actually tried to run this code, but it compiles and should hopefully give you enough ideas that you can adapt it your use case.

Footnotes

1. this is what you get in Rust and C/C++ when you don't explicitly the #[link(wasm_import_module = "foo")] attribute ↩

[zicklag]

Thanks for a great example!

This is essentially exactly what I need to do, but I need to be able to write the libraries in Rust.

Do you know whether or not it's possible to create the X, Y, and Z modules like in your example with Rust, or would we run into stack/data corruption because we'd have to compile the WASM modules as dynamic libraries?

I haven't figured out yet, if it's possible to compile shared WASM libraries from Rust. This section of the WASM tool conventions seems to indicate that you can create shared libraries with LLVM, but in the LLD Docs it says that lld can't make WASM shared libraries.