Can't use skbuild-based package from another package that uses a "throwaway" build venv

[kleinesfilmroellchen]

My setup is as follows:

• A main package, which uses hatchling for building, meaning that all builds are isolated in their own virtual environment that is created anew for each build. (This is done by build and pip in the background automatically, and should also apply to setuptools-based builds.)
• A dependency of this main package, which uses skbuild for a CMake-based extension. In my case the dependency is hdlConvertor, in a locally modified version as a subdirectory of the main package.

Whenever hatchling needs to rebuild the dependencies, for example when dependencies change, skbuild fails while trying to run CMake: apparently cython is not found:

Full relevant output

  -- Building python hdlConvertor module with libhdlConvertor linked in
      CMake Warning (dev) at /tmp/pip-build-env-fjm7q5jf/overlay/lib/python3.9/site-packages/skbuild/resources/cmake/FindPythonExtensions.cmake:245 (find_package):
        Policy CMP0148 is not set: The FindPythonInterp and FindPythonLibs modules
        are removed.  Run "cmake --help-policy CMP0148" for policy details.  Use
        the cmake_policy command to set the policy and suppress this warning.
      
      Call Stack (most recent call first):
        CMakeLists.txt:17 (find_package)
      This warning is for project developers.  Use -Wno-dev to suppress it.
      
      CMake Warning (dev) at /tmp/pip-build-env-fjm7q5jf/overlay/lib/python3.9/site-packages/skbuild/resources/cmake/FindPythonExtensions.cmake:252 (find_package):
        Policy CMP0148 is not set: The FindPythonInterp and FindPythonLibs modules
        are removed.  Run "cmake --help-policy CMP0148" for policy details.  Use
        the cmake_policy command to set the policy and suppress this warning.
      
      Call Stack (most recent call first):
        CMakeLists.txt:17 (find_package)
      This warning is for project developers.  Use -Wno-dev to suppress it.
      
      CMake Warning (dev) at CMakeLists.txt:18 (find_package):
        Policy CMP0148 is not set: The FindPythonInterp and FindPythonLibs modules
        are removed.  Run "cmake --help-policy CMP0148" for policy details.  Use
        the cmake_policy command to set the policy and suppress this warning.
      
      This warning is for project developers.  Use -Wno-dev to suppress it.
      
      CMake Error at /tmp/pip-build-env-fjm7q5jf/overlay/lib/python3.9/site-packages/skbuild/resources/cmake/FindCython.cmake:71 (message):
        Command "/tmp/pip-build-env-vwsv0gbj/overlay/bin/cython;--version" failed
        with output:
      
      Call Stack (most recent call first):
        CMakeLists.txt:19 (find_package)
      
      
      -- using antlr4-complete.jar found in $HOME/.local/lib/antlr4-complete.jar
      -- Antlr4 cpp runtime root set to: $HOME/.local/share/antlr4/usr/local/
      -- ANTLR4CPP_INCLUDE_DIRS=$HOME/.local/share/antlr4/usr/local/include/antlr4-runtime
      -- ANTLR4CPP_LIBRARIES=$HOME/.local/share/antlr4/usr/local/lib/libantlr4-runtime.so
      _modinit_prefix:PyInit_
      -- Configuring incomplete, errors occurred!
      Traceback (most recent call last):
        File "/tmp/pip-build-env-fjm7q5jf/overlay/lib/python3.9/site-packages/skbuild/setuptools_wrap.py", line 666, in setup
          env = cmkr.configure(
        File "/tmp/pip-build-env-fjm7q5jf/overlay/lib/python3.9/site-packages/skbuild/cmaker.py", line 357, in configure
          raise SKBuildError(msg)
      
      An error occurred while configuring with CMake.

Indeed, manually checking /tmp/pip-build-env-vwsv0gbj/overlay/bin/cython reveals that this is long gone; probably the build environment that was used last time. It looks like CMake caches this path; since the build directory (_skbuild) is never removed between runs it just gets reused.

The workaround currently therefore is to remove the _skbuild directory, and then everything works correctly once. Until the next package update of course.

I'm not sure what a good solution would be here. Maybe skbuild can remove its build directory and retry building if it fails the first time, which wouldn't completely slash incremental builds.