ARM support #485

vctibor · 2019-08-27T11:04:13Z

I tried to build on Raspberry Pi. It fails compiling raw-cpuid. It seems to me this crate is specific for x86 arch.

I wonder if ARM support is planned / possible?

sophiajt · 2019-08-27T16:40:10Z

We're planning on making some of the plugins that use platform-specific code optional. For now, you can comment out the sys binary in the Cargo.toml file and see if that helps get further.

vctibor · 2019-08-29T00:19:08Z

Hi and thanks for reply!

I have removed sys binary from Cargo.toml, removed sys.rs file, deleted Cargo.lock and target, but Rust still attempts to compile raw-cpuid crate and fails. Not sure how else I can make it forget this dependency.

Also, I can only get so far with debug builds. If I attempt cargo build --release, it fails immediately, unable to compile libc, proc-macro2, byteorder and semver-parser, all of them failing with invalid memory reference.

I will try to investigate further.

sophiajt · 2019-08-30T07:46:29Z

@vctibor - you may also need to remove the dependency on heim after you remove sys. It looks like heim is the one that pulls in raw-cpuid

theAkito · 2019-08-30T13:05:53Z

If I find the time, I might look into this as well, since I am also interested in stuff working on the Rpi.

vctibor · 2019-08-31T00:36:18Z

Now it fails on compilation of decimal crate. What I don't understand is that I can compile this crate by itself, but being build as dependency of Nu it fails. In both cases it's version 2.0.4.

Fenex · 2019-10-22T09:57:05Z

I have built nu on arm arch sucessfully.

Compile

To compile on ARM arch, you need clone my fork of the repository and compile arm branch.

Run commands step by step:

git clone https://github.com/Fenex/nushell.git # or you can add new remote path to my fork inside your local repo (see `git remote --help`)
cd nushell
git checkout arm
cargo build

If you use a device with 1GB RAM or less (e.g. Raspberry Pi 3b+ and older), you can need run compiling with single thread only:

cargo build --jobs=1

And yes, full compile taken 50min on my RPi 3b+ 😆

What is the cause of... and where is a PR?

A code in this repository compiling on ARM successfully unlike some depending crates.

The trouble crates are:

heim: (github | crates.io) includes incorrect cfg-directives for linking x86-specific cpuraw-id.
Already fixed, but not published on crates.io yet, PR: Fix building on non-x86 compatible archs heim-rs/heim#175
bson-rs: (github | crates.io) includes broken decimal crate.
Not fixed, unknown lifetime for fixing bug. You can know more for bug's reasons here: Fix building on ARM arch mongodb/bson-rust#132. Also you can read bson-rs's author answer for him plan to fix that.

After update heim to development branch I had to update futures-preview from 0.3.0-alpha.18 to 0.3.0-alpha.19 version. The change began to require update surf to http-rs/surf#74 .

So, as you can see, now Cargo.toml has a 3 git-depending crates, and one of them, bson-rs, also includes one another git-depending crate: decimal.
I didn't create a PR here because if I correct understand, it is impossible to publish a crate with git-dependencies on crates.io. Thus until bson-rs's authors fixed their crate and heim will update (author promised do it ~~soon~~after release Rust 1.39), we cannot merge the changes into master for release. However, you can make a new arm branch and then I could create a PR.

❤️

Fenex · 2019-10-22T10:09:59Z

Also, I can only get so far with debug builds. If I attempt cargo build --release, it fails immediately, unable to compile libc, proc-macro2, byteorder and semver-parser, all of them failing with invalid memory reference.

This is a fail of rustc, not crates. See issue rust-lang/rust#62896, PR rust-lang/rust#65302.

svartalf · 2019-10-22T10:11:59Z

@Fenex it seems that async-await branch of heim (your PR were merged into it as we discussed) will not be released any time soon (that would require stable Rust 1.39), so I can backport your PR into master branch and publish it as a 0.8.1 version, so it can be used here. That would also get rid of the futures-preview = "0.3.0-alpha.18"/-alpha.19 issue, as the 0.8.x versions are using -alpha.18 dependency.

Also, as I mentioned already, that does not mean that heim will provide any guarantees for ARM platforms so far, but I can add them into CI build, at least we can be sure that it can be compiled properly (what ARM targets should be checked?).

Fenex · 2019-10-22T10:22:21Z

@svartalf
I tested on below toolchain:

nightly-armv7-unknown-linux-gnueabihf (default)
rustc 1.40.0-nightly (6767d9b90 2019-10-11)

Anyway, your crate is not main problem, at the first we need to wait for a fix in bson-rs. So, no need publish fixed heim version right now, at least for nu project.

svartalf · 2019-10-22T16:20:03Z

I added a bunch of ARM targets to the heim CI, builds are green; @Fenex, you can try to switch to the master branch of heim now.
Let me know when you will need a release with your changes.

https://crates.io/crates/nu/0.7.1 `aarch64-linux` is currently not supported as several crates fail to compile on ARM archs (nushell/nushell#485).

L-as · 2020-08-07T13:31:23Z

I tried compiling on aarch64 today, and I get this:

error[E0308]: mismatched types
  --> /build/nushell-0.17.0-vendor.tar.gz/rawkey/src/linux.rs:44:46
   |
44 |             xlib::XQueryKeymap(self.display, keymap);
   |                                              ^^^^^^ expected `u8`, found `i8`
   |
   = note: expected raw pointer `*mut u8`
              found raw pointer `*mut i8`

error: aborting due to previous error

For more information about this error, try `rustc --explain E0308`.
error: could not compile `rawkey`.

To learn more, run the command again with --verbose.
warning: build failed, waiting for other jobs to finish...
error: build failed

I am wondering if a patch to libstd that just makes c_char and i8 on all platforms would fix this. It might break some FFI, but I honestly doubt it. In any case, this is hardly a nushell problem but a problem for the entire rust ecosystem that happened because of bad (IMO) design choices in libstd.

L-as · 2020-08-09T07:12:43Z

FWIW disabling the bson and binaryview features makes it work fine without any problems.

sophiajt · 2020-08-09T07:23:41Z

@L-as that's great news! If you want, you could add a PR that lists the supported features for arm support. We do this for default, stable, and are working on a set for wasm.

L-as · 2020-08-09T08:14:57Z

@jonathandturner where would I list that? I looked a bit inside the repo but I can't find any such place.

sophiajt · 2020-08-09T09:14:54Z

@L-as here is where the main feature groups start: https://github.com/nushell/nushell/blob/main/Cargo.toml line 65

colemickens · 2020-09-05T21:31:59Z

Per NixOS/nixpkgs#97262, this seems to just work now in 0.19.0, maybe the default feature set changed? I'm currently running it on my pinebook pro.

Walther · 2020-12-22T03:22:14Z

Anecdata:

Raspberry Pi 4 Model B Rev 1.4 (4-core 1.5GHz, 8GB RAM)
Debian GNU/Linux 10 (buster) aarch64

uname -a
Linux raspberrypi 5.4.79-v8+ #1373 SMP PREEMPT Mon Nov 23 13:32:41 GMT 2020 aarch64 GNU/Linux

   Compiling nu v0.24.1
    Finished release [optimized] target(s) in 36m 38s
  Installing /home/walther/.cargo/bin/nu
  Installing /home/walther/.cargo/bin/nu_plugin_core_fetch
  Installing /home/walther/.cargo/bin/nu_plugin_core_inc
  Installing /home/walther/.cargo/bin/nu_plugin_core_match
  Installing /home/walther/.cargo/bin/nu_plugin_core_post
  Installing /home/walther/.cargo/bin/nu_plugin_core_ps
  Installing /home/walther/.cargo/bin/nu_plugin_core_sys
  Installing /home/walther/.cargo/bin/nu_plugin_core_textview
   Installed package `nu v0.24.1` (executables `nu`, `nu_plugin_core_fetch`, `nu_plugin_core_inc`, `nu_plugin_core_match`, `nu_plugin_core_post`, `nu_plugin_core_ps`, `nu_plugin_core_sys`, `nu_plugin_core_textview`)
cargo install nu  5726.86s user 227.63s system 270% cpu 36:39.45 total

The install might not have been the fastest, but it works!

folliehiyuki · 2021-02-16T17:53:51Z

0.26.0 builds and works on my Pi 3B (aarch64) with --features=extra (without bson)

Type1J · 2021-03-16T01:45:09Z

FYI, if anybody needs an ARM64 to test compilation, the AWS m6g.nano is about $0.0057 per hour (a little over half a cent per hour). There is an image for Ubuntu 20.04 that seems to work well.

I tried to compile nu with --all-features, and it almost made it:

error: failed to compile `nu v0.28.0`, intermediate artifacts can be found at `/tmp/cargo-install1du9h6`

Caused by:
  could not compile `nu`

Caused by:
  process didn't exit successfully: `rustc --crate-name nu --edition=2018 '/home/admin@blah.local/.cargo/registry/src/github.com-1ecc6299db9ec823/nu-0.28.0/src/main.rs' --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C opt-level=z -C lto -C codegen-units=1 --cfg 'feature="binaryview"' --cfg 'feature="bson"' --cfg 'feature="chart"' --cfg 'feature="clipboard-cli"' --cfg 'feature="ctrlc"' --cfg 'feature="ctrlc-support"' --cfg 'feature="default"' --cfg 'feature="directories-support"' --cfg 'feature="extra"' --cfg 'feature="fetch"' --cfg 'feature="inc"' --cfg 'feature="match"' --cfg 'feature="nu_plugin_binaryview"' --cfg 'feature="nu_plugin_chart"' --cfg 'feature="nu_plugin_fetch"' --cfg 'feature="nu_plugin_from_bson"' --cfg 'feature="nu_plugin_from_sqlite"' --cfg 'feature="nu_plugin_inc"' --cfg 'feature="nu_plugin_match"' --cfg 'feature="nu_plugin_post"' --cfg 'feature="nu_plugin_ps"' --cfg 'feature="nu_plugin_s3"' --cfg 'feature="nu_plugin_selector"' --cfg 'feature="nu_plugin_start"' --cfg 'feature="nu_plugin_sys"' --cfg 'feature="nu_plugin_textview"' --cfg 'feature="nu_plugin_to_bson"' --cfg 'feature="nu_plugin_to_sqlite"' --cfg 'feature="nu_plugin_tree"' --cfg 'feature="nu_plugin_xpath"' --cfg 'feature="post"' --cfg 'feature="ps"' --cfg 'feature="ptree-support"' --cfg 'feature="rustyline-support"' --cfg 'feature="s3"' --cfg 'feature="selector"' --cfg 'feature="sqlite"' --cfg 'feature="stable"' --cfg 'feature="start"' --cfg 'feature="sys"' --cfg 'feature="table-pager"' --cfg 'feature="term-support"' --cfg 'feature="textview"' --cfg 'feature="trace"' --cfg 'feature="trash-support"' --cfg 'feature="tree"' --cfg 'feature="uuid-support"' --cfg 'feature="wasi"' --cfg 'feature="which-support"' --cfg 'feature="xpath"' --cfg 'feature="zip-support"' -C metadata=59496f08d36b462d -C extra-filename=-59496f08d36b462d --out-dir /tmp/cargo-install1du9h6/release/deps -L dependency=/tmp/cargo-install1du9h6/release/deps --extern clap=/tmp/cargo-install1du9h6/release/deps/libclap-69a9c0015783bd01.rlib --extern ctrlc=/tmp/cargo-install1du9h6/release/deps/libctrlc-bb1995756d2fe138.rlib --extern futures=/tmp/cargo-install1du9h6/release/deps/libfutures-3b204c2a844fcb7e.rlib --extern itertools=/tmp/cargo-install1du9h6/release/deps/libitertools-0998fd9dbc3a3527.rlib --extern log=/tmp/cargo-install1du9h6/release/deps/liblog-c8cc0585c825a1f0.rlib --extern nu_cli=/tmp/cargo-install1du9h6/release/deps/libnu_cli-0a299e8e01c2845e.rlib --extern nu_command=/tmp/cargo-install1du9h6/release/deps/libnu_command-0bd630308c534689.rlib --extern nu_data=/tmp/cargo-install1du9h6/release/deps/libnu_data-d49d2fe5bf0b933f.rlib --extern nu_engine=/tmp/cargo-install1du9h6/release/deps/libnu_engine-f858b6bfa7eed308.rlib --extern nu_errors=/tmp/cargo-install1du9h6/release/deps/libnu_errors-534dce9f452b69f0.rlib --extern nu_parser=/tmp/cargo-install1du9h6/release/deps/libnu_parser-5836522f97bc7bd8.rlib --extern nu_plugin=/tmp/cargo-install1du9h6/release/deps/libnu_plugin-88aa2e39693d5217.rlib --extern nu_protocol=/tmp/cargo-install1du9h6/release/deps/libnu_protocol-d5d102de41d228d9.rlib --extern nu_source=/tmp/cargo-install1du9h6/release/deps/libnu_source-018f2760a293b9fc.rlib --extern nu_value_ext=/tmp/cargo-install1du9h6/release/deps/libnu_value_ext-68f603dae725dceb.rlib --extern nu_plugin_binaryview=/tmp/cargo-install1du9h6/release/deps/libnu_plugin_binaryview-f0052ea4c7478e48.rlib --extern nu_plugin_chart=/tmp/cargo-install1du9h6/release/deps/libnu_plugin_chart-124a5341bc33a9a1.rlib --extern nu_plugin_fetch=/tmp/cargo-install1du9h6/release/deps/libnu_plugin_fetch-322b3f3f5b4e7ccd.rlib --extern nu_plugin_from_bson=/tmp/cargo-install1du9h6/release/deps/libnu_plugin_from_bson-0247f8f857b7e429.rlib --extern nu_plugin_from_sqlite=/tmp/cargo-install1du9h6/release/deps/libnu_plugin_from_sqlite-8c63169a5a65b363.rlib --extern nu_plugin_inc=/tmp/cargo-install1du9h6/release/deps/libnu_plugin_inc-b7da0e408ae2c29c.rlib --extern nu_plugin_match=/tmp/cargo-install1du9h6/release/deps/libnu_plugin_match-c4c9c611913487c8.rlib --extern nu_plugin_post=/tmp/cargo-install1du9h6/release/deps/libnu_plugin_post-be4ab443dc68e11a.rlib --extern nu_plugin_ps=/tmp/cargo-install1du9h6/release/deps/libnu_plugin_ps-d51233ac3631f49d.rlib --extern nu_plugin_s3=/tmp/cargo-install1du9h6/release/deps/libnu_plugin_s3-264775c228774ae4.rlib --extern nu_plugin_selector=/tmp/cargo-install1du9h6/release/deps/libnu_plugin_selector-30cf71d6fd62001a.rlib --extern nu_plugin_start=/tmp/cargo-install1du9h6/release/deps/libnu_plugin_start-c3837e2fa8a78926.rlib --extern nu_plugin_sys=/tmp/cargo-install1du9h6/release/deps/libnu_plugin_sys-f10e0e05cfc228ad.rlib --extern nu_plugin_textview=/tmp/cargo-install1du9h6/release/deps/libnu_plugin_textview-de18bc549c319597.rlib --extern nu_plugin_to_bson=/tmp/cargo-install1du9h6/release/deps/libnu_plugin_to_bson-8960ad05555f79fa.rlib --extern nu_plugin_to_sqlite=/tmp/cargo-install1du9h6/release/deps/libnu_plugin_to_sqlite-59dd4840596ce96c.rlib --extern nu_plugin_tree=/tmp/cargo-install1du9h6/release/deps/libnu_plugin_tree-a9f3b232c6abd050.rlib --extern nu_plugin_xpath=/tmp/cargo-install1du9h6/release/deps/libnu_plugin_xpath-ac8bd7531ae4a391.rlib --extern pretty_env_logger=/tmp/cargo-install1du9h6/release/deps/libpretty_env_logger-ca282ede4dc02cb7.rlib --cap-lints allow -L native=/tmp/cargo-install1du9h6/release/build/bzip2-sys-a6bf01fe9b2060a8/out/lib -L native=/tmp/cargo-install1du9h6/release/build/libgit2-sys-1d1f77fe29ca7642/out/build -L native=/tmp/cargo-install1du9h6/release/build/libz-sys-10a7464aa9b7f0bf/out/build -L native=/usr/lib/aarch64-linux-gnu -L native=/tmp/cargo-install1du9h6/release/build/ring-7e357bbfb1418e3e/out -L native=/tmp/cargo-install1du9h6/release/build/curl-sys-10baf863d6b3bf6e/out/build -L native=/tmp/cargo-install1du9h6/release/build/libnghttp2-sys-457d51496713609a/out/i/lib -L native=/tmp/cargo-install1du9h6/release/build/decimal-ff62654e9ee237b0/out -L native=/tmp/cargo-install1du9h6/release/build/libsqlite3-sys-9057a0ae5ee0d81a/out -L native=/tmp/cargo-install1du9h6/release/build/rust-crypto-511520ae53169a0b/out -L native=/tmp/cargo-install1du9h6/release/build/ansi_colours-8d4b8c992d16f5b2/out -L native=/tmp/cargo-install1du9h6/release/build/onig_sys-e8b392dcaebbefd1/out` (signal: 9, SIGKILL: kill)

I didn't send a SIGKILL, so I'm guessing it was cargo or a thread in rustc. The same operation hard locks my PineBook Pro (an ARM64 laptop based on Rock64) running Manjaro.

I tried without the --all-features flag, and it doesn't compile (same error).

colemickens · 2021-03-16T03:35:35Z

It's probably being killed because that aws instance is too micro. It does build on aarch64 successfully (I've built on aws arm instances, packet instances, PBP, etc)

Type1J · 2021-03-16T11:46:12Z

@colemickens That may be the case. I'll try it with a larger instance later today.

Type1J · 2021-03-16T20:21:27Z

I had the same issue on a c6g.large. This is the size of node that I use in most of the K8s clusters that I use, and I know that this size of machine can take some pretty heavy loads.

One thing to note is that this is using bson. How does one specify that they don't want bson with cargo install nu ...? Do I need to checkout the source and remove the bson plugins from the Cargo.toml, or is the bson problem fixed, now?

folliehiyuki · 2021-03-16T20:31:54Z

@Type1J bson is considering the update for decimal here.
And yes currently to not build nushell with bson plugin just delete it from Cargo.toml.

github-actions · 2021-06-24T20:04:28Z

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

Type1J · 2021-06-25T21:27:00Z

bson has merged the update. I haven't checked lately, but this might be no longer an issue.

sophiajt · 2021-06-25T21:56:47Z

Possibly related - I saw that gentoo supports ARM64 for nushell: https://packages.gentoo.org/packages/app-shells/nushell

folliehiyuki · 2021-06-25T23:53:18Z

We can already cross compile nushell for ARM on Void https://voidlinux.org/packages/?arch=aarch64&q=nushell

Type1J · 2021-06-27T01:08:33Z

@vctibor can you please confirm that the latest released version compiles on your Raspberry Pi, and close this issue?

filaretov · 2021-08-19T12:03:31Z

I just compiled the latest commit on the main branch on a Raspberry Pi 4 with 4GB of RAM running a recent installation of Raspberry OS (32-bit).

I first installed the Ubuntu/Debian dependencies listed in the book.

cargo install --features extra --path . successfully built the binary and I could run it.

cargo test successfully passed all tests, but I first encountered an error during compilation:

  = note: collect2: fatal error: ld terminated with signal 9 [Killed]
          compilation terminated.

Apparently, this might be caused by limited memory (I only had 100MB of swap). I reran with cargo test --jobs 1 and it completed successfully.

Walther · 2021-08-20T08:29:22Z

Can confirm again also on aarch64 version, Raspberry Pi 4 with 8GB and running the 64bit debian build. Installed with cargo install nu from the crates repo, works - although it takes a bit of time 😄

cargo install nu  14205.79s user 300.60s system 389% cpu 1:02:08.28 total

fdncred · 2022-02-18T00:32:21Z

looks like this is solved.

* Fix checkout step for feed plugin (nushell#485) * Revert "Fix checkout step for feed plugin (nushell#485)" (nushell#486) This reverts commit b301fd777b9b8351c3bfe5bd2a78ca81db516d28. * added doc for new config file login.nu Co-authored-by: Justin Ma <hustcer@outlook.com>

sophiajt added the upstream problem with upstream dependency label Aug 27, 2019

wycats added the platform-specific platform-specific issues label Aug 27, 2019

Ma27 added a commit to NixOS/nixpkgs that referenced this issue Jan 2, 2020

nushell: 0.7.0 -> 0.7.1

2ff742e

https://crates.io/crates/nu/0.7.1 `aarch64-linux` is currently not supported as several crates fail to compile on ARM archs (nushell/nushell#485).

folliehiyuki mentioned this issue Feb 4, 2021

nushell: update to 0.26.0 void-linux/void-packages#28341

Merged

3 tasks

github-actions bot added the Stale used for marking issues and prs as stale label Jun 24, 2021

github-actions bot removed the Stale used for marking issues and prs as stale label Jul 7, 2021

elferherrera pushed a commit to elferherrera/nushell that referenced this issue Feb 7, 2022

Better filepath completions (nushell#485)

906c0e6

kubouch pushed a commit that referenced this issue Feb 7, 2022

Better filepath completions (#485)

2100bb3

fdncred closed this as completed Feb 18, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ARM support #485

ARM support #485

vctibor commented Aug 27, 2019

sophiajt commented Aug 27, 2019

vctibor commented Aug 29, 2019

sophiajt commented Aug 30, 2019

theAkito commented Aug 30, 2019

vctibor commented Aug 31, 2019

Fenex commented Oct 22, 2019 •

edited

Fenex commented Oct 22, 2019

svartalf commented Oct 22, 2019

Fenex commented Oct 22, 2019

svartalf commented Oct 22, 2019

L-as commented Aug 7, 2020

L-as commented Aug 9, 2020

sophiajt commented Aug 9, 2020

L-as commented Aug 9, 2020

sophiajt commented Aug 9, 2020

colemickens commented Sep 5, 2020

Walther commented Dec 22, 2020 •

edited

folliehiyuki commented Feb 16, 2021

Type1J commented Mar 16, 2021

colemickens commented Mar 16, 2021

Type1J commented Mar 16, 2021

Type1J commented Mar 16, 2021

folliehiyuki commented Mar 16, 2021

github-actions bot commented Jun 24, 2021

Type1J commented Jun 25, 2021

sophiajt commented Jun 25, 2021

folliehiyuki commented Jun 25, 2021

Type1J commented Jun 27, 2021

filaretov commented Aug 19, 2021

Walther commented Aug 20, 2021 •

edited

fdncred commented Feb 18, 2022

ARM support #485

ARM support #485

Comments

vctibor commented Aug 27, 2019

sophiajt commented Aug 27, 2019

vctibor commented Aug 29, 2019

sophiajt commented Aug 30, 2019

theAkito commented Aug 30, 2019

vctibor commented Aug 31, 2019

Fenex commented Oct 22, 2019 • edited

Compile

What is the cause of... and where is a PR?

Fenex commented Oct 22, 2019

svartalf commented Oct 22, 2019

Fenex commented Oct 22, 2019

svartalf commented Oct 22, 2019

L-as commented Aug 7, 2020

L-as commented Aug 9, 2020

sophiajt commented Aug 9, 2020

L-as commented Aug 9, 2020

sophiajt commented Aug 9, 2020

colemickens commented Sep 5, 2020

Walther commented Dec 22, 2020 • edited

folliehiyuki commented Feb 16, 2021

Type1J commented Mar 16, 2021

colemickens commented Mar 16, 2021

Type1J commented Mar 16, 2021

Type1J commented Mar 16, 2021

folliehiyuki commented Mar 16, 2021

github-actions bot commented Jun 24, 2021

Type1J commented Jun 25, 2021

sophiajt commented Jun 25, 2021

folliehiyuki commented Jun 25, 2021

Type1J commented Jun 27, 2021

filaretov commented Aug 19, 2021

Walther commented Aug 20, 2021 • edited

fdncred commented Feb 18, 2022

Fenex commented Oct 22, 2019 •

edited

Walther commented Dec 22, 2020 •

edited

Walther commented Aug 20, 2021 •

edited