Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add intial support for wasm32-unknown-wasi #1307

Merged
merged 3 commits into from Mar 28, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
92 changes: 92 additions & 0 deletions ci/docker/wasm32-unknown-wasi/Dockerfile
@@ -0,0 +1,92 @@
# In the first container we want to assemble the `wasi-sysroot` by compiling it
# from source. This requires a clang 8.0+ compiler with enough wasm support and
# then we're just running a standard `make` inside of what we clone.
FROM ubuntu:18.04 as wasi-sysroot

RUN apt-get update && \
apt-get install -y --no-install-recommends \
ca-certificates \
clang \
cmake \
curl \
g++ \
git \
libc6-dev \
libclang-dev \
make \
ssh \
xz-utils

# Fetch clang 8.0+ which is used to compile the wasi target and link our
# programs together.
RUN curl http://releases.llvm.org/8.0.0/clang+llvm-8.0.0-x86_64-linux-gnu-ubuntu-18.04.tar.xz | tar xJf -
alexcrichton marked this conversation as resolved.
Show resolved Hide resolved
RUN mv /clang+llvm-8.0.0-x86_64-linux-gnu-ubuntu-18.04 /wasmcc

# Note that we're using `git reset --hard` to pin to a specific commit for
# verification for now. The sysroot is currently in somewhat of a state of flux
# and is expected to have breaking changes, so this is an attempt to mitigate
# those breaking changes on `libc`'s own CI
RUN git clone https://github.com/CraneStation/wasi-sysroot && \
cd wasi-sysroot && \
git reset --hard 320054e84f8f2440def3b1c8700cedb8fd697bf8
RUN make -C wasi-sysroot install -j $(nproc) WASM_CC=/wasmcc/bin/clang INSTALL_DIR=/wasi-sysroot

# This is a small wrapper script which executes the actual clang binary in
# `/wasmcc` and then is sure to pass the right `--sysroot` argument which we
# just built above.
COPY docker/wasm32-unknown-wasi/clang.sh /wasi-sysroot/bin/clang

# In the second container we're going to build the `wasmtime` binary which is
# used to execute wasi executables. This is a standard Rust project so we're
# just checking out a known revision (which pairs with the sysroot one we
# downlaoded above) and then we're building it with Cargo
FROM ubuntu:18.04 as wasmtime

RUN apt-get update && \
apt-get install -y --no-install-recommends \
ca-certificates \
clang \
cmake \
curl \
g++ \
git \
libclang-dev \
make \
ssh

RUN curl -sSf https://sh.rustup.rs | sh -s -- -y
ENV PATH=/root/.cargo/bin:$PATH

RUN apt-get install -y --no-install-recommends python
RUN git clone https://github.com/CraneStation/wasmtime-wasi wasmtime && \
cd wasmtime && \
git reset --hard 4fe2d6084e5b5cc74e69a26860f12750df51d339
RUN cargo build --release --manifest-path wasmtime/Cargo.toml

# And finally in the last image we're going to assemble everything together.
# We'll install things needed at runtime for now and then copy over the
# sysroot/wasmtime artifacts into their final location.
FROM ubuntu:18.04

RUN apt-get update && \
apt-get install -y --no-install-recommends \
gcc \
libc6-dev \
libxml2

# Copy over clang we downloaded to link executables ...
COPY --from=reference-sysroot /wasmcc /wasmcc/
alexcrichton marked this conversation as resolved.
Show resolved Hide resolved
# ... and the sysroot we built to link executables against ...
COPY --from=reference-sysroot /wasi-sysroot/ /wasi-sysroot/
# ... and finally wasmtime to actually execute binaries
COPY --from=wasmtime /wasmtime/target/release/wasmtime /usr/bin/

# Of note here is our clang wrapper which just executes a normal clang
# executable with the right sysroot, and then we're sure to turn off the
# crt-static feature to ensure that the CRT that we're specifying with `clang`
# is used.
ENV CARGO_TARGET_WASM32_UNKNOWN_WASI_RUNNER=wasmtime \
CARGO_TARGET_WASM32_UNKNOWN_WASI_LINKER=/wasi-sysroot/bin/clang \
CC_wasm32_unknown_wasi=/wasi-sysroot/bin/clang \
PATH=$PATH:/rust/bin \
RUSTFLAGS=-Ctarget-feature=-crt-static
2 changes: 2 additions & 0 deletions ci/docker/wasm32-unknown-wasi/clang.sh
@@ -0,0 +1,2 @@
#!/usr/bin/env sh
exec /wasmcc/bin/clang --target=wasm32-unknown-wasi --sysroot /wasi-sysroot "$@"
3 changes: 3 additions & 0 deletions ci/style.rs
Expand Up @@ -144,6 +144,9 @@ fn check_style(file: &str, path: &Path, err: &mut Errors) {
let line = if is_pub {&line[4..]} else {line};

let line_state = if line.starts_with("use ") {
if line.contains("c_void") {
continue;
}
if is_pub {
State::Modules
} else {
Expand Down
63 changes: 59 additions & 4 deletions libc-test/build.rs
Expand Up @@ -5,12 +5,12 @@ extern crate ctest;

use std::env;

#[cfg(unix)]
fn do_cc() {
cc::Build::new().file("src/cmsg.c").compile("cmsg");
let target = env::var("TARGET").unwrap();
if cfg!(unix) && !target.contains("wasi") {
cc::Build::new().file("src/cmsg.c").compile("cmsg");
}
}
#[cfg(not(unix))]
fn do_cc() {}

fn do_ctest() {
let target = env::var("TARGET").unwrap();
Expand Down Expand Up @@ -38,6 +38,7 @@ fn do_ctest() {
t if t.contains("solaris") => return test_solaris(t),
t if t.contains("netbsd") => return test_netbsd(t),
t if t.contains("dragonfly") => return test_dragonflybsd(t),
t if t.contains("wasi") => return test_wasi(t),
_ => (),
}

Expand Down Expand Up @@ -1866,3 +1867,57 @@ fn test_dragonflybsd(target: &str) {

cfg.generate("../src/lib.rs", "main.rs");
}

fn test_wasi(target: &str) {
assert!(target.contains("wasi"));

let mut cfg = ctest::TestGenerator::new();
cfg.define("_GNU_SOURCE", None);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this really necessary?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC it was to get some more of the libc APIs to show up, but I can play around with it once we turn this verification on


headers! { cfg:
"errno.h",
"fcntl.h",
"limits.h",
"locale.h",
"malloc.h",
"stddef.h",
"stdint.h",
"stdio.h",
"stdlib.h",
"sys/stat.h",
"sys/types.h",
"time.h",
"unistd.h",
"wasi/core.h",
"wasi/libc.h",
"wchar.h",
}

cfg.type_name(move |ty, is_struct, is_union| match ty {
"FILE" => ty.to_string(),
t if is_union => format!("union {}", t),
t if t.starts_with("__wasi") && t.ends_with("_u") => {
format!("union {}", t)
}
t if t.starts_with("__wasi") && is_struct => format!("struct {}", t),
t if t.ends_with("_t") => t.to_string(),
t if is_struct => format!("struct {}", t),
t => t.to_string(),
});

cfg.field_name(move |_struct, field| {
match field {
// deal with fields as rust keywords
"type_" => "type".to_string(),
s => s.to_string(),
}
});

// Looks like LLD doesn't merge duplicate imports, so if the Rust
// code imports from a module and the C code also imports from a
// module we end up with two imports of function pointers which
// import the same thing but have different function pointers
cfg.skip_fn_ptrcheck(|f| f.starts_with("__wasi"));

cfg.generate("../src/lib.rs", "main.rs");
}
5 changes: 4 additions & 1 deletion src/lib.rs
Expand Up @@ -112,7 +112,10 @@ cfg_if! {
} else if #[cfg(all(target_env = "sgx", target_vendor = "fortanix"))] {
mod sgx;
pub use sgx::*;
} else {
} else if #[cfg(target_env = "wasi")] {
mod wasi;
pub use wasi::*;
} else {
// non-supported targets: empty...
}
}