Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Relative file path loads on Windows not resolving as expected for Rust projects #360

Open
temeddix opened this issue Jan 26, 2024 · 10 comments
Labels
bug Something isn't working

Comments

@temeddix
Copy link

temeddix commented Jan 26, 2024

Hi. First, I would like to thank the developers here for all the great ongoing efforts to make wasm32-wasi work on multiple platforms. Also congratulations for the 1.0.0 release.

Since the Rinf (Rust in Flutter) project is planning to move from wasm32-unknown-unknown to wasm32-wasi in the future, I've been testeing jco version 1.0.0 on my personal repository, and found out most of the things were broken, at least for me.

I'm using...

  • Windows 11
  • Node.js 20
  • Chrome 120

You can take a look at my repo, but the Rust logic is rather simple. It just tests various I/O functionalities.

pub fn start() {
    println!("YAHOOOO");

    // Current directory
    let current_dir = env::current_dir().unwrap();
    println!("{current_dir:?}");

    // Time IO
    let now = std::time::Instant::now();
    println!("{now:?}");

    // Network IO
    let url = "http://jsonplaceholder.typicode.com/todos/1";
    let response_body = request_web(url);
    println!("{response_body}");

    // Threads
    std::thread::spawn(|| println!("YAHOO"));
    "Hello, World!".to_string();

    // File IO
    let test_text = std::fs::read("nodejs/file_test.txt");
    println!("{test_text:?}");
}

When I build and test the above repo for node.js, I get the following error: (Not only the errors, but the current directory is somewhat wrong)

YAHOOOO
"/"
Instant(260450.7532439s)
No valid response (4)
thread '<unnamed>' panicked at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/thread/mod.rs:686:29:
failed to spawn thread: Error { kind: Unsupported, message: "operation not supported on this platform" }
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
wasm://wasm/00b5c766:1


RuntimeError: unreachable
    at __rust_start_panic (wasm://wasm/00b5c766:wasm-function[458]:0x1b731)
    at rust_panic (wasm://wasm/00b5c766:wasm-function[446]:0x1b327)
    at _ZN3std9panicking20rust_panic_with_hook17hc93abff18edee779E (wasm://wasm/00b5c766:wasm-function[445]:0x1b304)
    at _ZN3std9panicking19begin_panic_handler28_$u7b$$u7b$closure$u7d$$u7d$17h922bcdd9c6fdedfbE (wasm://wasm/00b5c766:wasm-function[432]:0x1a46f)
    at _ZN3std10sys_common9backtrace26__rust_end_short_backtrace17h2597d6ecb1d3419eE (wasm://wasm/00b5c766:wasm-function[431]:0x1a39c)
    at rust_begin_unwind (wasm://wasm/00b5c766:wasm-function[440]:0x1ac1f)
    at _ZN4core9panicking9panic_fmt17h35d9e7e9c02f9eb5E (wasm://wasm/00b5c766:wasm-function[543]:0x20db9)
    at _ZN4core6result13unwrap_failed17hdced1445f29366ebE (wasm://wasm/00b5c766:wasm-function[569]:0x23172)
    at _ZN3std6thread5spawn17h62eb3b40d7dc13b2E (wasm://wasm/00b5c766:wasm-function[153]:0x8520)
    at _ZN8jco_test6common5start17h6565574157033c34E (wasm://wasm/00b5c766:wasm-function[162]:0x980e)

Node.js v20.11.0

When I build and test the above repo for browsers, I get the following error:

jco_test.js:4 Uncaught SyntaxError: The requested module '@bytecodealliance/preview2-shim/io' does not provide an export named 'error' (at jco_test.js:4:10)
image

I know that support for these platforms are not finalized, but it would be nice to know which parts are supported and which parts are not yet, since jco have reached version 1.0.0. I've been digging in the README and docs, but couldn't find any relevant information.

If I've missed anything, please let me know :)

@temeddix temeddix changed the title Most of the basic Broken pieces of 1.0.0 Jan 26, 2024
@guybedford
Copy link
Collaborator

guybedford commented Jan 26, 2024

Hi thanks for posting! You've hit two features that are currently unsupported in JCO:

  1. Rust threads are not supported
  2. Browser WASI is not supported and is marked as experimental (https://github.com/bytecodealliance/jco/tree/main/packages/preview2-shim#preview2-shim).

We're actively working on techniques for (2) in this project, including through the WASI-Virt project for virtualizing WASI in the browser. It won't be overnight, but this experience will improve very soon now that Node.js support is stable.

For (1), new threading specifications for Wasm will solve this in due course, but I can't offer any roadmap currently unfortunately.

@temeddix
Copy link
Author

temeddix commented Jan 26, 2024

Thanks for the fast reply, @guybedford . I am aware of WASI threads proposal hasn't reached phase 4, but some questions seemed to be unsolved yet :) please let me clarify...

  • Shouldn't network I/O work on node.js?
  • Shouldn't file I/O work on node.js? It was working before, but now it's broken in 1.0.0.

@guybedford
Copy link
Collaborator

Both network IO and file IO should work on Node.js. If you have any example that is failing I would be happy to take a look as that would be a bug.

@temeddix
Copy link
Author

temeddix commented Jan 26, 2024

It's this repository right here:

After cloning the repo, all the commands are written in README.md file, so the problem should be easy to reproduce.

@guybedford
Copy link
Collaborator

It is using threads though - https://github.com/temeddix/jco-test/blob/main/src/common.rs#L24 ?

I thought you said the problem was something other than the threads support? Do you have an example that doesn't use threads to verify that?

@temeddix
Copy link
Author

temeddix commented Jan 26, 2024

Even if I remove the thread code like this

pub fn start() {
    println!("YAHOOOO");

    // Current directory
    let current_dir = env::current_dir().unwrap();
    println!("{current_dir:?}");

    // Time IO
    let now = std::time::Instant::now();
    println!("{now:?}");

    // Network IO
    let url = "http://jsonplaceholder.typicode.com/todos/1";
    let response_body = request_web(url);
    println!("{response_body}");

    // File IO
    let test_text = std::fs::read("nodejs/file_test.txt");
    println!("{test_text:?}");
}

It gives results like this on Node.js

YAHOOOO
"/"
Instant(272895.6400198s)
No valid response (4)
Error: UNKNOWN: unknown error, open '\\nodejs\file_test.txt\'
    at openSync (node:fs:581:18)
    at Descriptor.openAt (file:///C:/Users/temed/Documents/GitHub/jco-test/node_modules/@bytecodealliance/preview2-shim/lib/nodejs/filesystem.js:364:18)
    at trampoline19 (file:///C:/Users/temed/Documents/GitHub/jco-test/nodejs/out_dir/jco_test.js:1289:57)
    at wit-component:shim.indirect-wasi:filesystem/types@0.2.0-[method]descriptor.open-at (wasm://wasm/wit-component:shim-1ca20d1e:wasm-function[7]:0x190)
    at wit-component:adapter:wasi_snapshot_preview1._ZN22wasi_snapshot_preview18bindings4wasi10filesystem5types10Descriptor7open_at17hc77490b982af5d76E (wasm://wasm/wit-component:adapter:wasi_snapshot_preview1-0001c22a:wasm-function[56]:0x385b)
    at wit-component:adapter:wasi_snapshot_preview1.path_open (wasm://wasm/wit-component:adapter:wasi_snapshot_preview1-0001c22a:wasm-function[52]:0x31d0)
    at wit-component:shim.adapt-wasi_snapshot_preview1-path_open (wasm://wasm/wit-component:shim-1ca20d1e:wasm-function[24]:0x278)
    at _ZN4wasi13lib_generated9path_open17h2a15a04eb366fdd2E (wasm://wasm/00a885a6:wasm-function[373]:0x169e3)
    at _ZN3std3sys4wasi2fs7open_at17h082474d937053e28E (wasm://wasm/00a885a6:wasm-function[352]:0x163dd)
    at _ZN3std3sys4wasi2fs4File4open17h247fdc4a63060476E (wasm://wasm/00a885a6:wasm-function[298]:0x12f37) {
  errno: -4094,
  code: 'UNKNOWN',
  syscall: 'open',
  path: '\\\\nodejs\\file_test.txt\\'
}
file:///C:/Users/temed/Documents/GitHub/jco-test/nodejs/out_dir/jco_test.js:1465
          throw new TypeError(`"${val7}" is not one of the cases of error-code`);
                ^

TypeError: "Error: UNKNOWN: unknown error, open '\\nodejs\file_test.txt\'" is not one of the cases of error-code
    at trampoline19 (file:///C:/Users/temed/Documents/GitHub/jco-test/nodejs/out_dir/jco_test.js:1465:17)
    at wit-component:shim.indirect-wasi:filesystem/types@0.2.0-[method]descriptor.open-at (wasm://wasm/wit-component:shim-1ca20d1e:wasm-function[7]:0x190)
    at wit-component:adapter:wasi_snapshot_preview1._ZN22wasi_snapshot_preview18bindings4wasi10filesystem5types10Descriptor7open_at17hc77490b982af5d76E (wasm://wasm/wit-component:adapter:wasi_snapshot_preview1-0001c22a:wasm-function[56]:0x385b)
    at wit-component:adapter:wasi_snapshot_preview1.path_open (wasm://wasm/wit-component:adapter:wasi_snapshot_preview1-0001c22a:wasm-function[52]:0x31d0)
    at wit-component:shim.adapt-wasi_snapshot_preview1-path_open (wasm://wasm/wit-component:shim-1ca20d1e:wasm-function[24]:0x278)
    at _ZN4wasi13lib_generated9path_open17h2a15a04eb366fdd2E (wasm://wasm/00a885a6:wasm-function[373]:0x169e3)
    at _ZN3std3sys4wasi2fs7open_at17h082474d937053e28E (wasm://wasm/00a885a6:wasm-function[352]:0x163dd)
    at _ZN3std3sys4wasi2fs4File4open17h247fdc4a63060476E (wasm://wasm/00a885a6:wasm-function[298]:0x12f37)
    at _ZN3std2fs4read5inner17h53a32882e372104bE (wasm://wasm/00a885a6:wasm-function[297]:0x12ca5)
    at _ZN3std2fs4read17ha81f41ca4a935112E (wasm://wasm/00a885a6:wasm-function[21]:0xc43)

Node.js v20.11.0

I'm pretty sure that things were working well about a couple of months ago. Yesterday I tried jco 1.0.0 and it was broken like the output above. The build process still works well, though.

@guybedford
Copy link
Collaborator

Thanks, this is a Windows-specific pathing bug, I'll take a look.

@guybedford guybedford changed the title Broken pieces of 1.0.0 Non-relative file loads on Windows not resolving as expected for Rust projects Jan 26, 2024
@guybedford guybedford added the bug Something isn't working label Jan 26, 2024
@guybedford guybedford changed the title Non-relative file loads on Windows not resolving as expected for Rust projects Relative file path loads on Windows not resolving as expected for Rust projects Jan 26, 2024
@temeddix
Copy link
Author

Let me note that Network I/O is not working as well, like the output below

No valid response (4)

which derives from this Rust code

fn request_web(url: &str) -> String {
    // Parse the URL to extract the domain and path
    let (domain, path) = parse_url(url).expect("Invalid URL");

    // Establish a TCP connection to the domain
    if let Ok(mut stream) = TcpStream::connect(format!("{}:80", domain)) {
        // Prepare the HTTP request
        let request = format!(
            "GET {} HTTP/1.1\r\n\
             Host: {}\r\n\
             Connection: close\r\n\
             \r\n",
            path, domain
        );

        // Send the HTTP request
        if stream.write(request.as_bytes()).is_ok() {
            // Read the response
            let mut response = String::new();
            let mut reader = BufReader::new(&stream);
            if reader.read_to_string(&mut response).is_ok() {
                // Print the response
                let response_parts: Vec<&str> = response.split("\r\n\r\n").collect();
                if response_parts.len() >= 2 {
                    let body = response_parts[1];
                    return body.to_owned();
                } else {
                    return String::from("No valid response (1)");
                }
            } else {
                return String::from("No valid response (2)");
            }
        } else {
            return String::from("No valid response (3)");
        }
    } else {
        return String::from("No valid response (4)");
    }
}

@pchickey
Copy link
Collaborator

pchickey commented Jan 26, 2024

There is probably more information about the error in the Err returned from TcpStream::connect, and other places - you might want to change that program to return Result<String, anyhow::Error> and use the debug print of the Err on failure.

But additionally, the rust std TcpStream is not yet connected to preview 2 wasi-sockets, right now you have to use wasi-sockets bindings directly. Work is underway in wasi-libc that will lead to support for wasi-sockets in std.

@temeddix
Copy link
Author

temeddix commented Jan 26, 2024

I've checked the Result, and the message is as follows.

fn request_web(url: &str) -> String {
    // Parse the URL to extract the domain and path
    let (domain, path) = parse_url(url).expect("Invalid URL");

    // Establish a TCP connection to the domain
    let result = TcpStream::connect(format!("{}:80", domain));
    println!("{result:?}");
}
Err(Error { kind: Unsupported, message: "operation not supported on this platform" })

But I see that TcpStream is not directly supported yet. Thanks for the explanation :) Maybe it would be a good idea to add clear guides about supported APIs, in my humble opinion.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants