Unable to Install Nix within Runner

[gthomson31]

Checks

• I've already read https://github.com/actions/actions-runner-controller/blob/master/TROUBLESHOOTING.md and I'm sure my issue is not covered in the troubleshooting guide.
• I'm not using a custom entrypoint in my runner image

Controller Version

0.22.0

Helm Chart Version

0.22.0

CertManager Version

v1.9.2

Deployment Method

Helm

cert-manager installation

Yes installed and working

Checks

• This isn't a question or user support case (For Q&A and community support, go to Discussions. It might also be a good idea to contract with any of contributors and maintainers if your business is so critical and therefore you need priority support
• I've read releasenotes before submitting this issue and I'm sure it's not due to any recently-introduced backward-incompatible changes
• My actions-runner-controller version (v0.x.y) does support the feature
• I've already upgraded ARC (including the CRDs, see charts/actions-runner-controller/docs/UPGRADING.md for details) to the latest and it didn't fix the issue
• I've migrated to the workflow job webhook event (if you using webhook driven scaling)

Resource Definitions

apiVersion: actions.summerwind.dev/v1alpha1
kind: Runner
metadata:
  annotations:
    sync-time: "2023-02-15T09:48:34Z"
  creationTimestamp: "2023-02-15T09:48:34Z"
  finalizers:
  - runner.actions.summerwind.dev
  generateName: github-runner-deployment-dp52d-
  generation: 1
  labels:
    runner-deployment-name: github-runner-deployment
    runner-template-hash: 7f7d579795
  name: github-runner-deployment-dp52d-89b4v
  namespace: github-actions-runners-prod01
  ownerReferences:
  - apiVersion: actions.summerwind.dev/v1alpha1
    blockOwnerDeletion: true
    controller: true
    kind: RunnerReplicaSet
    name: github-runner-deployment-dp52d
    uid: 189f2f79-6570-4a52-84c2-1df367ad23bf
  resourceVersion: "110119449"
  uid: 50b45db9-9080-4bce-b59e-39f4ec326243
spec:
  dockerdContainerResources: {}
  env:
  - name: USER
    value: runner
  image: summerwind/actions-runner-dind:ubuntu-22.04
  imagePullPolicy: IfNotPresent
  labels:
  - nix
  organization: org-name
  resources: {}
status:
  phase: Running
  ready: true
  registration:
    expiresAt: "2023-02-15T10:24:49Z"
    labels:
    - nix
    organization: org-name
    token: <REDACTED>

To Reproduce

Deploy runner using the summerwind images and try to run the install-nix github action as part of a workflow.


name: Hello World
on:
  push:
    paths:
      - ".github/workflows/hello-world.yml"


jobs:
  example-job:
    runs-on: nix
    steps:
      - name: Install Nix
        uses: cachix/install-nix-action@v19

### Describe the bug

For the Runners we are  building our own images built on top off the image below but issue also occurs on standard image

`https://hub.docker.com/r/summerwind/actions-runner/tags`

Have tested using images built from the following versions 
-  [ubuntu-20.04/Latest](https://hub.docker.com/layers/summerwind/actions-runner/latest/images/sha256-2f2c8ca0b5764d267f9463a344a1ba96ee3c89f9f0dc59427b975e4a7c324b86?context=explore)
- [ubuntu-22.04](https://hub.docker.com/layers/summerwind/actions-runner/ubuntu-22.04/images/sha256-44c885990d8151d27c8078f21d25d673c8c0b3d9d7119787de94f7afad30eeb0?context=explore)


## Dockerfile for Image 

FROM summerwind/actions-runner:latest

USER root

Hashistack installation , etc

USER runner

## Error Message 

Run cachix/install-nix-action@v19
Run /runner/_work/_actions/cachix/install-nix-action/v19/install-nix.sh
/runner/_work/_actions/cachix/install-nix-action/v19/install-nix.sh
shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
env:
INPUT_EXTRA_NIX_CONFIG:
INPUT_GITHUB_ACCESS_TOKEN:
INPUT_INSTALL_OPTIONS:
INPUT_INSTALL_URL:
INPUT_NIX_PATH:
GITHUB_TOKEN: ***
Installing Nix
/runner/_work/_actions/cachix/install-nix-action/v19/install-nix.sh: line 23: USER: unbound variable
Error: Process completed with exit code 1.

After then setting the USER to Runner we are presented with this error.

Run sh <(curl -L https://nixos.org/nix/install) --no-daemon --no-channel-add
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed

0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0

0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 4052 100 4052 0 0 16635 0 --:--:-- --:--:-- --:--:-- 565k
downloading Nix 2.13.2 binary tarball for x86_64-linux from 'https://releases.nixos.org/nix/nix-2.13.2/nix-2.13.2-x86_64-linux.tar.xz' to '/tmp/nix-binary-tarball-unpack.p6cjhhcO1H'...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed

0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 20.0M 100 20.0M 0 0 72.7M 0 --:--:-- --:--:-- --:--:-- 72.9M
Note: a multi-user installation is possible. See https://nixos.org/manual/nix/stable/installation/installing-binary.html#multi-user-installation
performing a single-user installation of Nix...
directory /nix does not exist; creating it by running 'mkdir -m 0755 /nix && chown runner /nix' using sudo

copying Nix to /nix/store...
installing 'nix-2.13.2'
error: mounting /proc: Operation not permitted
(use '--show-trace' to show detailed location information)
/tmp/nix-binary-tarball-unpack.p6cjhhcO1H/unpack/nix-2.13.2-x86_64-linux/install: unable to install Nix into your default profile
Error: Process completed with exit code 1.

### Describe the expected behavior

Expect to install Nix within the runner via the action.

### Whole Controller Logs

```shell
N/A

Whole Runner Pod Logs

N/A

Additional Context

Associated Issues
cachix/install-nix-action#122
NixOS/nix#4107 (comment)

[mumoshu]

Hey @gthomson31! Isn't this an issue in nix? They seem to provide their own docker image https://nixos.org/manual/nix/stable/installation/installing-docker.html. Perhaps building a custom runner image based on the nix image would be a more viable way to go? 🤔

I've read the linked issues, and at least unmounting parts of /proc didn't make sense to me, as I don't think it's possible within a container, and also, it won't be a good idea to let it unmount the host OS' /proc (if we added a host volume mount there...).

[gthomson31]

Thanks for the feedback nix I was new to me but we have managed a workaround in just installing onto the runner as part of the actions workflow for the time being

[mumoshu]

Hey @gthomson31! Awesome to hear you found a workaround.
Would you mind sharing what worked for reference? Thanks!

[gthomson31]

Yeah no problem @mumoshu
I wasnt involved in this workflow creation but this setup allowed Nix to function on the Self Hosted Runner

  steps:
      - name: Checkout Code
        uses: actions/checkout@v3
        with:
          submodules: true
          ssh-key: ${{ secrets.GLOBAL_SSH_KEY }}

      - name: Setup SSH Keys and known_hosts
        env:
          SSH_AUTH_SOCK: /tmp/ssh_agent.sock
        run: |
          ssh-agent -a $SSH_AUTH_SOCK > /dev/null
          ssh-add - <<< "${{ secrets.GLOBAL_SSH_KEY }}"
          ssh-keyscan -t rsa github.com >> /home/runner/.ssh/known_hosts

      - name: Install nix
        run: |
          sudo mkdir -p /nix /etc/nix
          sudo chmod a+rwx /nix
          sudo sh -c "echo 'sandbox = false' > /etc/nix/nix.conf"
          sh <(curl -L https://nixos.org/nix/install) --no-daemon
          mkdir -p /home/runner/.config/nix/
          echo 'experimental-features = nix-command flakes' > /home/runner/.config/nix/nix.conf

      - name: Run e2e-test seeds
        env:
          SSH_AUTH_SOCK: /tmp/ssh_agent.sock
        run: |
          . /home/runner/.nix-profile/etc/profile.d/nix.sh
          export PATH="/home/runner/.nix-profile/bin:/nix/var/nix/profiles/default/bin:$PATH"
          nix develop -c task-github-actions-e2e-seed 

      - name: Run e2e-test
        env:
          SSH_AUTH_SOCK: /tmp/ssh_agent.sock
        run: |
          . /home/runner/.nix-profile/etc/profile.d/nix.sh
          export PATH="/home/runner/.nix-profile/bin:/nix/var/nix/profiles/default/bin:$PATH"
          nix develop -c task-go-test-e2e