worktree.add corrupts repository index

[tiandrey]

Subject of the issue

Working with git worktree via ruby-git corrupts repository index: previously clean original branch has files deleted/modified and reverse changes unstaged, new worktree has the same problems.

Your environment

git version 2.38.1
git (1.18.0)
ruby 2.7.7p221 (2022-11-24 revision 168ec2b1e5) [x86_64-linux]

Steps to reproduce

For demonstration I've created new repository with two branches, second 1 commit ahead of first, and from that second branch I've added worktree with first branch in a new directory - after that both worktrees became dirty.

user@localhost [/tmp] $ D=`mktemp -d`; echo $D; cd $D
/tmp/tmp.pwtopGh59g
user@localhost [/tmp/tmp.pwtopGh59g] $ git init
Initialized empty Git repository in /tmp/tmp.pwtopGh59g/.git/
user@localhost [/tmp/tmp.pwtopGh59g] $ echo "1.0" > VERSION; git add VERSION ; git commit -m "init commit"
[master (root-commit) a85add1] init commit
 1 file changed, 1 insertion(+)
 create mode 100644 VERSION
user@localhost [/tmp/tmp.pwtopGh59g] $ git checkout -b new-version
Switched to a new branch 'new-version'
user@localhost [/tmp/tmp.pwtopGh59g] $ echo "2.0" > VERSION; git add VERSION; git commit -m "new version"
[new-version 87dbf0e] new version
 1 file changed, 1 insertion(+), 1 deletion(-)
user@localhost [/tmp/tmp.pwtopGh59g] $ D=`mktemp -d`; echo $D
/tmp/tmp.HbaM98rzYd
user@localhost [/tmp/tmp.pwtopGh59g] $ irb
2.7.7 :001 > require 'git'
 => true 
2.7.7 :002 > g = Git.open('.')
 => 
#<Git::Base:0x000056325dcc6880                                                        
...                                                                                   
2.7.7 :003 > g.worktree('/tmp/tmp.HbaM98rzYd', 'master').add
 => "Preparing worktree (checking out 'master')\nHEAD is now at a85add1 init commit" 
2.7.7 :004 > exit
user@localhost [/tmp/tmp.pwtopGh59g] $ git status                                                            
On branch new-version                                                                                        
Changes to be committed:                                                                                     
  (use "git restore --staged <file>..." to unstage)                                                          
        modified:   VERSION                                                                                  
                                                                                                             
Changes not staged for commit:                                                                               
  (use "git add <file>..." to update what will be committed)                                                 
  (use "git restore <file>..." to discard changes in working directory)                                      
        modified:   VERSION                                                                                  
                                                                                                             
user@localhost [/tmp/tmp.pwtopGh59g] $ git diff
diff --git a/VERSION b/VERSION
index d3827e7..cd5ac03 100644
--- a/VERSION
+++ b/VERSION
@@ -1 +1 @@
-1.0
+2.0
user@localhost [/tmp/tmp.pwtopGh59g] $ git diff HEAD
user@localhost [/tmp/tmp.pwtopGh59g] $ cd /tmp/tmp.HbaM98rzYd
user@localhost [/tmp/tmp.HbaM98rzYd] $ git status                                                            
On branch master
Changes to be committed:
  (use "git restore --staged <file>..." to unstage)
	deleted:    VERSION

Untracked files:
  (use "git add <file>..." to include in what will be committed)
	VERSION

user@localhost [/tmp/tmp.HbaM98rzYd] $ git diff
user@localhost [/tmp/tmp.HbaM98rzYd] $ git diff HEAD
diff --git a/VERSION b/VERSION
deleted file mode 100644
index d3827e7..0000000
--- a/VERSION
+++ /dev/null
@@ -1 +0,0 @@
-1.0

Expected behaviour

Original and new worktrees are clean

Actual behaviour

Original and new worktrees are dirty, although files were not actually changed, only its status in index

Investigation

This case almost blew my brain out, I was debugging with strace to see which git commands ruby-git executes, and when I ran the same commands by hand - everything worked as expected, i.e. both worktrees were clean. But after thorough examination I found out that ruby-git adds three environment variables: GIT_DIR, GIT_WORK_TREE and GIT_INDEX_FILE. The first two were duplicated by command-line options --git-dir and --work-tree, but the last one was the root of the problem: it points to index file in the original worktree while new worktree should use its own index.

I'm unsure whether that environment variable is needed at all, because GIT_DIR should be enough. But for sure it should be treated separately when working with git worktree.

[jcouball]

Thank you for the report, @tiandrey ! I'll look into this as time allows.

[jcouball]

I was able to duplicate the problem outside of this library using the script below. The script will output SUCCESS if the index is clean or FAILED if the index has been corrupted.

Comment out the GIT_INDEX_FILE line and then git worktree add not corrupt the index. Leave that line in, and the index is corrupted.

Is this a defect in git itself? Regardless, I will search for a workaround such as not setting the GIT_INDEX_FILE environment variable if an index was not specified. When I tried this, some other tests failed. I am still looking into it.

Examples of why you might want to set the index to something other than the default value are covered in this article by Linus Torvalds. Examples along those lines can be found in test_tree_ops.rb. Search for calls to .with_temp_index and .with_index.

#!/bin/bash

BASEDIR=`mktemp -d`
cd "${BASEDIR}"

mkdir main_worktree
cd main_worktree

git init --initial-branch=main

echo '1.0' > VERSION
git add .
git commit -m 'init commit'

git checkout -b new_branch
echo '2.0' > VERSION
git add .
git commit -m 'new version'

# Comment out the GIT_INDEX_FILE line and then `git worktree add` will succeed

GIT_INDEX_FILE="${BASEDIR}/main_worktree/.git/index" \
GIT_DIR="${BASEDIR}/main_worktree/.git" \
GIT_WORK_TREE="${BASEDIR}/main_worktree" \
git --git-dir="${BASEDIR}/main_worktree/.git" --work-tree="${BASEDIR}/main_worktree" '-c' 'core.quotePath=true' '-c' 'color.ui=false' 'worktree' 'add' "${BASEDIR}/linked_worktree" 'main'

echo
if git status | grep -q 'nothing to commit, working tree clean'; then
  echo 'SUCCESS: Index is clean'
else
  echo 'FAILED: Index has been corrupted'
fi