Weird error occurs when targeting wasm32-unknown-emscripten

[km19809]

I am trying to compile my simple pyo3 library to the wasm32-unknown-emscripten target.
However, it fails like this:

error: Unknown target OS family for cross-compilation: "unix,wasm".

As I know, CARGO_CFG_TARGET_FAMILY is one of "unix", "windows", "wasm".
And I also set my environment variable to "wasm".
However, it throws the error. What should I do?

+
I modified the source of pyo3-build-config to fix above error, and now it throws:

error: failed to run custom build command for `pyo3 v0.15.1`

Caused by:
  process didn't exit successfully: `/home/minsoo/hobby/pyxel/lib/target/release/build/pyo3-9d61ebea8a0ee2e2/build-script-build` (exit status: 101)
  --- stdout
  cargo:rerun-if-env-changed=PYO3_CROSS
  cargo:rerun-if-env-changed=PYO3_CROSS_LIB_DIR
  cargo:rerun-if-env-changed=PYO3_CROSS_PYTHON_VERSION
  cargo:rerun-if-env-changed=_PYTHON_SYSCONFIGDATA_NAME
  cargo:rerun-if-env-changed=PYO3_PYTHON
  cargo:rerun-if-env-changed=VIRTUAL_ENV
  cargo:rerun-if-env-changed=CONDA_PREFIX
  cargo:rerun-if-env-changed=PATH
  cargo:rerun-if-env-changed=PYO3_PRINT_CONFIG

  -- PYO3_PRINT_CONFIG=1 is set, printing configuration and halting compile --
  implementation=CPython
  version=3.7
  shared=false
  abi3=true
  lib_name=python3.9
  lib_dir=/src/cpython/installs/python-3.9.5/lib
  pointer_width=32
  build_flags=WITH_THREAD
  suppress_build_script_link_lines=false
warning: build failed, waiting for other jobs to finish...
error: build failed

++ edit
What I modified is a single line of pyo3-build-config/src/impl_.rs

fn load_cross_compile_config(
    cross_compile_config: CrossCompileConfig,
) -> Result<InterpreterConfig> {
    match cargo_env_var("CARGO_CFG_TARGET_FAMILY") {
        // Configure for unix platforms using the sysconfigdata file
        Some(os) if os == "unix" => load_cross_compile_from_sysconfigdata(cross_compile_config),
        // Use hardcoded interpreter config when targeting Windows
        Some(os) if os == "windows" => windows_hardcoded_cross_compile(cross_compile_config),
        // sysconfigdata works fine on wasm/wasi
        Some(os) if os == "wasm" => load_cross_compile_from_sysconfigdata(cross_compile_config),
       // The modified line
	 Some(os) if os == "unix,wasm" => load_cross_compile_from_sysconfigdata(cross_compile_config),
        // Waiting for users to tell us what they expect on their target platform
        Some(os) => bail!(
            "Unknown target OS family for cross-compilation: {:?}.\n\
            \n\
            Please set the PYO3_CONFIG_FILE environment variable to a config suitable for your \
            target interpreter.",
            os
        ),
        // Unknown os family - try to do something useful
        None => load_cross_compile_from_sysconfigdata(cross_compile_config),
    }
}

[davidhewitt]

@km19809 try ensuring PYO3_PRINT_CONFIG env var is unset?

It's hard to know what to suggest, given you say you modified the source code.

[km19809]

Thank you for the answer.
I unset the variable, then the message disappeared.

But there are link errors now.
such as:

undefined symbol: _Py_TrueStruct
undefined symbol: _Py_FalseStruct

I enabled ["extension-module"] because there is no wasm libpython to link.
Could you give me any tips about those link error?

[davidhewitt]

I'm afraid I don't know much about emscripten linking. Can you share the full output?

[km19809]

On second thought, I think that this is not from the PyO3.
But I'll share the full text here.
This is so long and messy, sorry.

Full text:

error: linking with `emcc` failed: exit status: 1
  |
  = note: "emcc" "-s" "EXPORTED_FUNCTIONS=[\"_PyInit_py_cyril\",\"_rust_eh_personality\"]" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.157yx5tyvfm179w.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.198dvcc8thb7yq47.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.1dc2njuwlg3hmwh.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.1eq5c1zskb1ahyis.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.1hbswffypcpvzg8w.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.1huwav4kv5irwk23.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.1ptfdc45sml7cdhj.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.1qi2fjvts4k6nvwh.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.1r37kza0kkg2nlgj.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.1wpzrcphrzb1im37.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.22dstf961hemynj3.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.22njblvc0fu9o3dn.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.261xks7707asy41x.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.29pukj7pegkwklyc.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.2f9l9gpbyc6qeonh.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.2jc2a0rexsncztz2.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.2k7foon144x6ub86.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.2lk3468bnhzfh4s6.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.2q6ymhw82q3rfiuy.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.2qdwk7tf2u4tszpl.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.2tgukbbshqqpebcd.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.2uoh2ox026x0p5vt.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.2vxuuzuq4vo0vv13.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.2yk29yruw7tna99y.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.33gbidts8s0ytuc8.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.36bopb8ujezl95xv.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.3cer3khc92uuvzsj.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.3deaq4hzbk3soavn.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.3dnxsof8vj9x2ejn.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.3eheo0m8vj3ih3e3.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.3fc57pe7ia33ws71.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.3gnn6qjgfyzycvzp.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.3oih20nldskka5i9.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.3p8ucru8kuu256kq.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.3qt1xr9ysd78rhcy.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.3tnj5wrtilb7u409.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.3y0u8e7h3q2k5jgp.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.3zqipi6b29637jao.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.464e66nvl0iqtljd.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.4a43k4wjv21w9bpf.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.4bqn0a7hruj3105c.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.4d9sj7bguzfww8dn.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.4e1wr1yhzs3zsuv.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.4h3irag0jummxwdd.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.4py9y9xxe6ujgw0k.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.4v6x37fk63r7xiwr.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.4xnvzrezkuj36se7.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.50a3qil6o3tfysko.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.51v05cx4krowy851.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.52hbeq3ltd215y0j.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.549soadci7wzxruf.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.55dx1mr4vkmqoehp.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.58jgmf5mhmlo8cpv.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.5epy3g5f2i5kb3to.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.ad4dcy9v03yfstq.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.cg1hxf6rbkqjixi.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.osvu3zmp2bg0w5p.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.tjd2zygebidlz9x.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.u6gq7t0xde9k11p.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.z5z52wn31zmpva4.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.zyl3j1033izjrd3.rcgu.o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.1pledkoxgmsbst2p.rcgu.o" "-L" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps" "-L" "/home/minsoo/hobby/py_cyril/target/debug/deps" "-L" "/home/minsoo/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/libpyo3-3d1f6a9e6e0af0dc.rlib" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/libparking_lot-14ceb2df0eaab2bb.rlib" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/libparking_lot_core-c2f4b309e394cfeb.rlib" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/libsmallvec-d8b3fca218f794cb.rlib" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/liblock_api-efb2c2715207af5a.rlib" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/libscopeguard-a53b0fc22bea738f.rlib" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/libinstant-6163d6d51301788e.rlib" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/libcfg_if-70042c3bba752a60.rlib" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/liblibc-79778cb05005485b.rlib" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/libunindent-5ad3b3d227a21b69.rlib" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/libpaste-59bd44b2b8ab61cd.rlib" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/libindoc-d20075316f11db07.rlib" "/home/minsoo/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/libstd-e42ff3047517e183.rlib" "/home/minsoo/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/libpanic_unwind-31fb4e2ba1dc06a5.rlib" "/home/minsoo/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/libstd_detect-9399f03198074a22.rlib" "/home/minsoo/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/librustc_demangle-ba54c9474735c2d0.rlib" "/home/minsoo/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/libhashbrown-1001ff7f8317fc1f.rlib" "/home/minsoo/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/librustc_std_workspace_alloc-90d5f6d94b90c3f1.rlib" "/home/minsoo/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/libunwind-3b48a08bd63267eb.rlib" "/home/minsoo/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/libcfg_if-235f316e999b88a8.rlib" "/home/minsoo/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/liblibc-fe35ba1b405099f5.rlib" "/home/minsoo/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/liballoc-20deb84cf1e1ba54.rlib" "/home/minsoo/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/librustc_std_workspace_core-aa74bcfc52d2a9a4.rlib" "/home/minsoo/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/libcore-b34a127052c6c454.rlib" "/home/minsoo/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/libcompiler_builtins-c234d542c931ddff.rlib" "-l" "c" "-s" "DISABLE_EXCEPTION_CATCHING=0" "-L" "/home/minsoo/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib" "-L" "/home/minsoo/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/self-contained" "-o" "/home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/py_cyril.wasm" "-O0" "--memory-init-file" "0" "-g4" "-s" "DEFAULT_LIBRARY_FUNCS_TO_INCLUDE=[]" "emscripten/gxx_personality_v0_stub.o" "-s" "ASYNCIFY" "-s" "ALLOW_MEMORY_GROWTH=1" "--no-entry" "-s" "ERROR_ON_UNDEFINED_SYMBOLS=1" "-s" "ASSERTIONS=1" "-s" "ABORTING_MALLOC=0" "-Wl,--fatal-warnings"
  = note: emcc: warning: please replace -g4 with -gsource-map [-Wdeprecated]
          wasm-ld: error: /home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/libpyo3-3d1f6a9e6e0af0dc.rlib(pyo3-3d1f6a9e6e0af0dc.pyo3.2b037835-cgu.0.rcgu.o): undefined symbol: PyExc_AttributeError
          wasm-ld: error: /home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/libpyo3-3d1f6a9e6e0af0dc.rlib(pyo3-3d1f6a9e6e0af0dc.pyo3.2b037835-cgu.0.rcgu.o): undefined symbol: PyExc_BaseException
          wasm-ld: error: /home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/libpyo3-3d1f6a9e6e0af0dc.rlib(pyo3-3d1f6a9e6e0af0dc.pyo3.2b037835-cgu.0.rcgu.o): undefined symbol: PyExc_ValueError
          wasm-ld: error: /home/minsoo/hobby/py_cyril/target/wasm32-unknown-emscripten/debug/deps/libpyo3-3d1f6a9e6e0af0dc.rlib(pyo3-3d1f6a9e6e0af0dc.pyo3.2b037835-cgu.0.rcgu.o): undefined symbol: PyExc_TypeError
          emcc: error: '/home/minsoo/emsdk/upstream/bin/wasm-ld @/tmp/emscripten_kw6egf8y.rsp.utf-8' failed (returned 1)
          

error: could not compile `py-cyril` due to previous error

[davidhewitt]

Hmm, I have no experience with emscripten, but this bit looks relevant:

"-s" "ERROR_ON_UNDEFINED_SYMBOLS=1"

If you can find a way to configure the build to turn that option off, it'll probably complete fine.

Otherwise, you will presumably need to compile libpython for emscripten and link to it.

[km19809]

I'll ty it. Thank you for your help!