Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fails to build on Nix #184

Open
knarkzel opened this issue Mar 20, 2023 · 8 comments
Open

Fails to build on Nix #184

knarkzel opened this issue Mar 20, 2023 · 8 comments

Comments

@knarkzel
Copy link

I have everything setup except for the esp-idf which this crate should handle. However, when attempting to build, I get following:

esp32-hello-world via (nix-shell-env) ❯ cargo build
    Updating crates.io index
   Compiling esp-idf-sys v0.32.1
error: failed to run custom build command for `esp-idf-sys v0.32.1`

Caused by:
  process didn't exit successfully: `/home/odd/source/nix/temp/esp32-hello-world/target/debug/build/esp-idf-sys-abdf284afcdfa8df/build-script-build` (exit status: 1)
  --- stdout
  cargo:rerun-if-env-changed=ESP_IDF_TOOLS_INSTALL_DIR
  cargo:rerun-if-env-changed=ESP_IDF_SDKCONFIG
  cargo:rerun-if-env-changed=ESP_IDF_SDKCONFIG_DEFAULTS
  cargo:rerun-if-env-changed=MCU
  cargo:rerun-if-env-changed=ESP_IDF_SYS_ROOT_CRATE
  cargo:rerun-if-env-changed=ESP_IDF_VERSION
  cargo:rerun-if-env-changed=ESP_IDF_REPOSITORY
  cargo:rerun-if-env-changed=ESP_IDF_CMAKE_GENERATOR
  cargo:rerun-if-env-changed=IDF_PATH
  cargo:rerun-if-env-changed=EXTRA-COMPONENTS
  cargo:rerun-if-env-changed=ESP_IDF_COMPONENTS
  PATH=/home/odd/source/nix/temp/esp32-hello-world/.embuild/espressif/esp-idf/release-v4.4/tools:$PATH

  --- stderr
  Build configuration: BuildConfig {
      esp_idf_tools_install_dir: None,
      esp_idf_sdkconfig: None,
      esp_idf_sdkconfig_defaults: None,
      mcu: None,
      native: NativeConfig {
          esp_idf_version: Some(
              Branch(
                  "release/v4.4",
              ),
          ),
          esp_idf_repository: None,
          esp_idf_cmake_generator: None,
          idf_path: None,
          extra_components: [],
          esp_idf_components: None,
      },
      esp_idf_sys_root_crate: None,
  }
  Using managed esp-idf repository: RemoteSdk { repo_url: None, git_ref: Branch("release/v4.4") }
  fatal: No names found, cannot describe anything.
  fatal: No names found, cannot describe anything.
  WARNING: Git describe was unsuccessful: b''
  ERROR: This script was called from a virtual environment, can not create a virtual environment again
  Error: Could not install esp-idf

  Caused by:
      command '"python3" "/home/odd/source/nix/temp/esp32-hello-world/.embuild/espressif/esp-idf/release-v4.4/tools/idf_tools.py" "--idf-path" "/home/odd/source/nix/temp/esp32-hello-world/.embuild/espressif/esp-idf/release-v4.4" "--non-interactive" "install-python-env"' exited with non-zero status code 1

All the code is located here.

@N3xed
Copy link
Collaborator

N3xed commented Mar 20, 2023

  ERROR: This script was called from a virtual environment, can not create a virtual environment again
  Error: Could not install esp-idf

  Caused by:
      command '"python3" "/home/odd/source/nix/temp/esp32-hello-world/.embuild/espressif/esp-idf/release-v4.4/tools/idf_tools.py" "--idf-path" "/home/odd/source/nix/temp/esp32-hello-world/.embuild/espressif/esp-idf/release-v4.4" "--non-interactive" "install-python-env"' exited with non-zero status code 1

Make sure that python3 doesn't come from a virtual env (see Determine if Python is running inside virtualenv for more info).
The esp-idf-sys will try to install the needed tools which includes creating a python virtualenv.

@knarkzel
Copy link
Author

knarkzel commented Oct 13, 2023

Ok, I've changed the shell now:

{
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
    esp-dev = {
      url = "github:thiskappaisgrey/nixpkgs-esp-dev-rust";
      inputs.nixpkgs.follows = "nixpkgs";
    };
  };

  outputs = {
    self,
    nixpkgs,
    esp-dev,
  }: let
    pkgs = import nixpkgs {
      system = "x86_64-linux";
      overlays = [ esp-dev.overlay ];
    };
  in {
    devShells.x86_64-linux.default = pkgs.mkShell {
      buildInputs = with pkgs; [
        gcc-riscv32-esp32c3-elf-bin
        openocd-esp32-bin

        # Tools required to use ESP-IDF.
        git
        wget
        gnumake

        flex
        bison
        gperf
        pkg-config

        cmake
        ninja

        ncurses5

        python3
        python3Packages.pip
        python3Packages.virtualenv
      ];

      shellHook = ''
        export ESP_IDF_VERSION="v4.4"
        export LD_LIBRARY_PATH="${pkgs.lib.makeLibraryPath [ pkgs.libxml2 pkgs.zlib pkgs.stdenv.cc.cc.lib ]}"
        export LIBCLANG_PATH=${pkgs.libclang.lib}/lib
      '';
    };
  };
}

Now I get this:

  [939/941] Linking C executable libespidf.elf
  [940/941] Generating binary image from built executable
  esptool.py v3.2-dev
  Merged 1 ELF section
  Generated /home/odd/source/nix/hello-esp32c3/target/riscv32imc-esp-espidf/debug/build/esp-idf-sys-3444d1415f0a5f40/out/build/libespidf.bin
  [941/941] cd /home/odd/source/nix/hello-esp32c3/target/riscv32imc-esp-espidf/debug/build/esp-idf-sys-3444d1415f0a5f40/out/build/esp-idf/esptool_py && /home/odd/source/nix/hello-esp32c3/.embuild/espressif/python_env/idf4.4_py3.10_env/bin/python /home/odd/source/nix/hello-esp32c3/.embuild/espressif/esp-idf/v4.4/components/partition_table/check_sizes.py --offset 0x8000 partition --type app /home/odd/source/nix/hello-esp32c3/target/riscv32imc-esp-espidf/debug/build/esp-idf-sys-3444d1415f0a5f40/out/build/partition_table/partition-table.bin /home/odd/source/nix/hello-esp32c3/target/riscv32imc-esp-espidf/debug/build/esp-idf-sys-3444d1415f0a5f40/out/build/libespidf.bin
  libespidf.bin binary size 0x21db0 bytes. Smallest app partition is 0x100000 bytes. 0xde250 bytes (87%) free.
  cargo:root=/home/odd/source/nix/hello-esp32c3/target/riscv32imc-esp-espidf/debug/build/esp-idf-sys-3444d1415f0a5f40/out
  cargo:rerun-if-changed=/home/odd/.cargo/registry/src/index.crates.io-6f17d22bba15001f/esp-idf-sys-0.33.2/src/include/esp-idf/bindings.h
  cargo:rustc-env=EMBUILD_GENERATED_BINDINGS_FILE=/home/odd/source/nix/hello-esp32c3/target/riscv32imc-esp-espidf/debug/build/esp-idf-sys-3444d1415f0a5f40/out/bindings.rs

  --- stderr
  Build configuration: BuildConfig {
      esp_idf_tools_install_dir: None,
      esp_idf_sdkconfig: None,
      esp_idf_sdkconfig_defaults: None,
      mcu: None,
      native: NativeConfig {
          esp_idf_version: Some(
              Tag(
                  "v4.4",
              ),
          ),
          esp_idf_repository: None,
          esp_idf_cmake_generator: None,
          idf_path: None,
          extra_components: [],
          esp_idf_components: None,
      },
      esp_idf_sys_root_crate: None,
  }
  Using managed esp-idf repository: RemoteSdk { repo_url: None, git_ref: Tag("v4.4") }
  Using esp-idf v4.4.0 at '/home/odd/source/nix/hello-esp32c3/.embuild/espressif/esp-idf/v4.4'
  WARNING: directory for tool riscv32-esp-elf version esp-2021r2-8.4.0 is present, but tool was not found
  WARNING: destination path already exists, removing
  WARNING: directory for tool cmake version 3.20.3 is present, but tool was not found
  WARNING: destination path already exists, removing
  WARNING: directory for tool ninja version 1.10.2 is present, but tool was not found
  WARNING: destination path already exists, removing
  WARNING: directory for tool riscv32-esp-elf version esp-12.2.0_20230208 is present, but tool was not found
  WARNING: destination path already exists, removing
  /home/odd/source/nix/hello-esp32c3/.embuild/espressif/esp-idf/v4.4/tools/check_python_dependencies.py:12: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
    import pkg_resources
  CMake Warning:
    Manually-specified variables were not used by the project:

      CMAKE_TOOLCHAIN_FILE


  Built components: esp_ringbuf, efuse, esp_ipc, driver, esp_pm, mbedtls, bootloader, esptool_py, partition_table, app_update, bootloader_support, spi_flash, nvs_flash, pthread, esp_gdbstub, espcoredump, esp_phy, esp_system, esp_rom, hal, vfs, esp_eth, tcpip_adapter, esp_netif, esp_event, wpa_supplicant, esp_wifi, ieee802154, console, openthread, lwip, log, heap, soc, esp_hw_support, riscv, esp32c3, esp_common, esp_timer, freertos, newlib, cxx, app_trace, asio, bt, cbor, unity, cmock, coap, nghttp, esp-tls, esp_adc_cal, esp_hid, tcp_transport, esp_http_client, esp_http_server, esp_https_ota, esp_https_server, esp_lcd, protobuf-c, protocomm, mdns, esp_local_ctrl, sdmmc, esp_serial_slave_link, esp_websocket_client, expat, wear_levelling, fatfs, freemodbus, idf_test, jsmn, json, libsodium, mqtt, openssl, spiffs, usb, tinyusb, wifi_provisioning
  /nix/store/y8wfrgk7br5rfz4221lfb9v8w3n0cnyd-glibc-2.37-8-dev/include/limits.h:145:5: error: function-like macro '__GLIBC_USE' is not defined
  /nix/store/y8wfrgk7br5rfz4221lfb9v8w3n0cnyd-glibc-2.37-8-dev/include/limits.h:184:5: error: function-like macro '__GLIBC_USE' is not defined
  /home/odd/source/nix/hello-esp32c3/.embuild/espressif/esp-idf/v4.4/components/freertos/port/riscv/include/freertos/portmacro.h:439:20: warning: section does not match previous declaration [-Wsection]
  /home/odd/source/nix/hello-esp32c3/.embuild/espressif/esp-idf/v4.4/components/freertos/port/riscv/include/freertos/portmacro.h:271:20: note: previous attribute is here
  /home/odd/source/nix/hello-esp32c3/.embuild/espressif/esp-idf/v4.4/components/esp_hw_support/include/soc/esp_spiram.h:1:2: warning: esp_spiram.h has been replaced by esp32/spiram.h, please include esp32/spiram.h instead [-W#warnings]
  clang diag: /home/odd/source/nix/hello-esp32c3/.embuild/espressif/esp-idf/v4.4/components/freertos/port/riscv/include/freertos/portmacro.h:439:20: warning: section does not match previous declaration [-Wsection]
  clang diag: /home/odd/source/nix/hello-esp32c3/.embuild/espressif/esp-idf/v4.4/components/esp_hw_support/include/soc/esp_spiram.h:1:2: warning: esp_spiram.h has been replaced by esp32/spiram.h, please include esp32/spiram.h instead [-W#warnings]
  Error: failed to generate bindings in file '/home/odd/source/nix/hello-esp32c3/target/riscv32imc-esp-espidf/debug/build/esp-idf-sys-3444d1415f0a5f40/out/bindings.rs'

  Caused by:
      clang diagnosed error: /nix/store/y8wfrgk7br5rfz4221lfb9v8w3n0cnyd-glibc-2.37-8-dev/include/limits.h:145:5: error: function-like macro '__GLIBC_USE' is not defined
      /nix/store/y8wfrgk7br5rfz4221lfb9v8w3n0cnyd-glibc-2.37-8-dev/include/limits.h:184:5: error: function-like macro '__GLIBC_USE' is not defined

@yuri91
Copy link

yuri91 commented Oct 27, 2023

@knarkzel I am trying to make esp-idf for rust work on nix, same as you.

I took a step back and side-stepped the issue of the idf source not being a git repo, by unsetting IDF_PATH and letting esp-idf-sys download it at runtime (in a nix develop session).

Then, I stumbled on this __GLIBC_USE issue.

I think it has to do with nix wrapping clang and adding include paths to it. But esp-idf does not want to pull in host system headers, which are being mixed up with headers from the cross compiler.

So I removed clang from the shell inputs (but kept libclang), and it worked!

It would still be nice to be able to use the esp toolchain from the nix store though.

@shymega
Copy link

shymega commented Nov 24, 2023

This is something I'm trying to get working too.

I've bumped a Flake I forked with the ESP-IDF, but encountering a Git repo error.

I think this will take some work.....

@ChocolateLoverRaj
Copy link

Does anyone have a working flake? I'm also getting the __GLIBC_USE issue.

@auroraanna
Copy link

@ChocolateLoverRaj i do with the dependencies of the flake above (actually without openocd-esp32-bin, flex, bison and gperf) but i put them in an FHS env with nix because the build process tried to execute it's own non-nix executables.

@Vollbrecht
Copy link
Collaborator

Its a two part process:

The first part is us installing essential stuff like the python virt_env + cmake + ninja + either riscv or xtensa gcc compiler needed for the specific ESP_IDF version, That could be more or less statically provided under the assumption that it follows the same layout ESP_IDF expacts.

The second part and the bigger problem is that ESP_IDF also uses pip and a bunch of python dependancys e.g kconfig wrapper and esp-idf-component manager written in python, that depends on other stuff it pulls in via pip. Keep in mind that upstream has this deeply coupled inside there system so just providing something similar is not enough, it will just fail on certain test that are run inline when the cmake buildsystem is executed.

So a complete solution would need to tackle the second part, the first part is more trivial.

@ChocolateLoverRaj
Copy link

@auroraanna can u provide your flake.nix? This is what I'm using atm: https://github.com/ChocolateLoverRaj/rust-esp32c3-examples/blob/main/flake.nix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Todo
Development

No branches or pull requests

7 participants