Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add intial support for wasm32-unknown-wasi #1307

Merged
merged 3 commits into from Mar 28, 2019
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
69 changes: 69 additions & 0 deletions ci/docker/wasm32-unknown-wasi/Dockerfile
@@ -0,0 +1,69 @@
FROM ubuntu:18.04 as reference-sysroot

RUN apt-get update && \
apt-get install -y --no-install-recommends \
ca-certificates \
clang \
cmake \
curl \
g++ \
git \
libc6-dev \
libclang-dev \
make \
ssh \
xz-utils

RUN curl http://releases.llvm.org/8.0.0/clang+llvm-8.0.0-x86_64-linux-gnu-ubuntu-18.04.tar.xz | tar xJf -
alexcrichton marked this conversation as resolved.
Show resolved Hide resolved
RUN mv /clang+llvm-8.0.0-x86_64-linux-gnu-ubuntu-18.04 /wasmcc

RUN git clone https://github.com/CraneStation/reference-sysroot-wasi && \
cd reference-sysroot-wasi && \
git reset --hard d5a609fe63926533e1054e539ba5f2693d51bdf5
alexcrichton marked this conversation as resolved.
Show resolved Hide resolved
RUN make -C reference-sysroot-wasi install -j $(nproc) WASM_CC=/wasmcc/bin/clang INSTALL_DIR=/wasm-sysroot
COPY docker/wasm32-unknown-wasi/clang.sh /wasm-sysroot/bin/clang
alexcrichton marked this conversation as resolved.
Show resolved Hide resolved

FROM ubuntu:18.04 as wasmtime

RUN apt-get update && \
apt-get install -y --no-install-recommends \
ca-certificates \
clang \
cmake \
curl \
g++ \
git \
libclang-dev \
make \
ssh

RUN curl -sSf https://sh.rustup.rs | sh -s -- -y
ENV PATH=/root/.cargo/bin:$PATH

RUN apt-get install -y --no-install-recommends python
RUN git clone https://github.com/CraneStation/wasmtime-wasi wasmtime && \
cd wasmtime && \
git reset --hard a7ac05df74759a7536b2b1e30adc6ff4867e36c3

# Install wasmtime in /usr/bin, but make sure to remove rust afterwards because
# we don't want it conflicting with the main Rust we're using to compile
# `libc`.
alexcrichton marked this conversation as resolved.
Show resolved Hide resolved
RUN cargo build --release --manifest-path wasmtime/Cargo.toml

FROM ubuntu:18.04

RUN apt-get update && \
apt-get install -y --no-install-recommends \
gcc \
libc6-dev \
libxml2

COPY --from=reference-sysroot /wasmcc /wasmcc/
alexcrichton marked this conversation as resolved.
Show resolved Hide resolved
COPY --from=reference-sysroot /wasm-sysroot/ /wasm-sysroot/
COPY --from=wasmtime /wasmtime/target/release/wasmtime /usr/bin/

ENV CARGO_TARGET_WASM32_UNKNOWN_WASI_RUNNER=wasmtime \
CARGO_TARGET_WASM32_UNKNOWN_WASI_LINKER=/wasm-sysroot/bin/clang \
CC_wasm32_unknown_wasi=/wasm-sysroot/bin/clang \
PATH=$PATH:/rust/bin \
RUSTFLAGS=-Ctarget-feature=-crt-static
2 changes: 2 additions & 0 deletions ci/docker/wasm32-unknown-wasi/clang.sh
@@ -0,0 +1,2 @@
#!/bin/sh
alexcrichton marked this conversation as resolved.
Show resolved Hide resolved
exec /wasmcc/bin/clang --target=wasm32-unknown-wasi --sysroot /wasm-sysroot "$@"
73 changes: 69 additions & 4 deletions libc-test/build.rs
Expand Up @@ -5,12 +5,12 @@ extern crate ctest;

use std::env;

#[cfg(unix)]
fn do_cc() {
cc::Build::new().file("src/cmsg.c").compile("cmsg");
let target = env::var("TARGET").unwrap();
if cfg!(unix) && !target.contains("wasi") {
cc::Build::new().file("src/cmsg.c").compile("cmsg");
}
}
#[cfg(not(unix))]
fn do_cc() {}

fn do_ctest() {
let target = env::var("TARGET").unwrap();
Expand Down Expand Up @@ -38,6 +38,7 @@ fn do_ctest() {
t if t.contains("solaris") => return test_solaris(t),
t if t.contains("netbsd") => return test_netbsd(t),
t if t.contains("dragonfly") => return test_dragonflybsd(t),
t if t.contains("wasi") => return test_wasi(t),
_ => (),
}

Expand Down Expand Up @@ -1866,3 +1867,67 @@ fn test_dragonflybsd(target: &str) {

cfg.generate("../src/lib.rs", "main.rs");
}

fn test_wasi(target: &str) {
assert!(target.contains("wasi"));

let mut cfg = ctest::TestGenerator::new();
cfg.define("_GNU_SOURCE", None);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this really necessary?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC it was to get some more of the libc APIs to show up, but I can play around with it once we turn this verification on


headers! { cfg:
"errno.h",
"fcntl.h",
"limits.h",
"locale.h",
"malloc.h",
"stddef.h",
"stdint.h",
"stdio.h",
"stdlib.h",
"sys/stat.h",
"sys/types.h",
"time.h",
"unistd.h",
"wasi/core.h",
"wasi/libc.h",
"wchar.h",
}

cfg.type_name(move |ty, is_struct, is_union| match ty {
"FILE" => ty.to_string(),
t if is_union => format!("union {}", t),
t if t.starts_with("__wasi") && t.ends_with("_u") => format!("union {}", t),
t if t.starts_with("__wasi") && is_struct => format!("struct {}", t),
t if t.ends_with("_t") => t.to_string(),
t if is_struct => format!("struct {}", t),
t => t.to_string(),
});

// This is an opaque struct but we go through shenanigans to define values
// for it
cfg.skip_struct(move |ty| ty == "__clockid");
alexcrichton marked this conversation as resolved.
Show resolved Hide resolved

cfg.field_name(move |_struct, field| {
match field {
// deal with fields as rust keywords
alexcrichton marked this conversation as resolved.
Show resolved Hide resolved
"type_" => "type".to_string(),
s => s.to_string(),
}
});

cfg.skip_static(move |name| {
match name {
// wasi shenanigans for defining CLOCK_REALTIME and such
s if s.starts_with("__CLOCK") => true,
alexcrichton marked this conversation as resolved.
Show resolved Hide resolved
_ => false,
}
});

// Looks like LLD doesn't merge duplicate imports, so if the Rust
// code imports from a module and the C code also imports from a
// module we end up with two imports of function pointers which
// improt the same thing bug have different function pointers
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo: "bug"/"but"
typo: "improt"/"import"

So IIUC the issue here is that "libc" (wasi/the sysroot) is linked statically into both the Rust code (due to "-C target-feature=+crt-static"), and the C code, and that when these two are linked, two copies of the "libc" are present in the final binary. This is super-unfortunate.

Have you tried whether -C lto=fat solves the issue ? That might be too brittle. Alternatively, maybe for this target it might make sense to not link the C code (or the Rust code) to the sysroot when compiling, so that when Rust and C are linked together, there is only one sysroot.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nah this is actually a different issue that's somewhat deeper in LLD and unrelated to linkage/LTO/etc. It has to do with an LLVM module that looks like:

define void @foo() {
    call void @bar1()
    call void @bar2()
}

declare void @bar1() #1 
declare void @bar2() #2 

attributes #1 = { "wasm-import-module"="foo" }
attributes #1 = { "wasm-import-module"="foo" "wasm-import-name"="bar1" }

Both bar1 and bar2 reference the same function, but due to how LLD is implemented these show up as two different functions in the final binary, where the same name is imported twice (giving the same function two different function indices).

It's not really an issue, just a minor thing that will likely get fixed in LLD at some point.

cfg.skip_fn_ptrcheck(|f| f.starts_with("__wasi"));

cfg.generate("../src/lib.rs", "main.rs");
}
5 changes: 4 additions & 1 deletion src/lib.rs
Expand Up @@ -112,7 +112,10 @@ cfg_if! {
} else if #[cfg(all(target_env = "sgx", target_vendor = "fortanix"))] {
mod sgx;
pub use sgx::*;
} else {
} else if #[cfg(target_env = "wasi")] {
mod wasi;
pub use wasi::*;
} else {
// non-supported targets: empty...
}
}