fatal: fetch-pack: invalid index-pack output (malloc failed)

[liam-baker-sm]

• I was not able to find an open or closed issue matching what I'm seeing

Two issues closed without any information or resolution.

Setup

• Which version of Git for Windows are you using? Is it 32-bit or 64-bit?

$ git --version --build-options

C:\Users\Administrator>git --version --build-options
git version 2.44.0.windows.1
cpu: x86_64
built from commit: ad0bbfffa543db6979717be96df630d3e5741331
sizeof-long: 4
sizeof-size_t: 8
shell-path: /bin/sh
feature: fsmonitor--daemon

• Which version of Windows are you running? Vista, 7, 8, 10? Is it 32-bit or 64-bit?
  Windows Server 2022 Datacenter 21H2 (64-bit)

$ cmd.exe /c ver

Microsoft Windows [Version 10.0.20348.2322]

• What options did you set as part of the installation? Or did you choose the
  defaults?

# One of the following:
> type "C:\Program Files\Git\etc\install-options.txt"
> type "C:\Program Files (x86)\Git\etc\install-options.txt"
> type "%USERPROFILE%\AppData\Local\Programs\Git\etc\install-options.txt"
> type "$env:USERPROFILE\AppData\Local\Programs\Git\etc\install-options.txt"
$ cat /etc/install-options.txt

C:\Users\Administrator>type "C:\Program Files\Git\etc\install-options.txt"
Editor Option: VIM
Custom Editor Path:
Default Branch Option:
Path Option: Cmd
SSH Option: OpenSSH
Tortoise Option: false
CURL Option: OpenSSL
CRLF Option: CRLFAlways
Bash Terminal Option: MinTTY
Git Pull Behavior Option: Merge
Use Credential Manager: Enabled
Performance Tweaks FSCache: Enabled
Enable Symlinks: Disabled
Enable Pseudo Console Support: Disabled
Enable FSMonitor: Disabled

• Any other interesting things about your environment that might be related
  to the issue you're seeing?

We consistently see this error on Server 2022 in AWS, but not on in-house Server 2012 or on Windows 10 Professional machines, despite running the same command on the same repository.

It is likely that network bandwidth on this machine (an AWS cloud Virtual Machine instance) is greater than disk bandwidth, for in-house build servers this is not the case.

Details

• Which terminal/shell are you running Git from? e.g Bash/CMD/PowerShell/other
  CMD Powershell via ansible
  ansible.windows.win_command: git fetch --prune --all

• What commands did you run to trigger this issue? If you can provide a
  Minimal, Complete, and Verifiable example
  this will help us understand the issue.

git fetch --prune --all

• What did you expect to occur after running these commands?
  Git should fetch all branches (creating a reference repository)

• What actually happened instead?

The following error is thrown

fatal: Out of memory, malloc failed (tried to allocate 2117377 bytes)
fatal: fetch-pack: invalid index-pack output

This is despite the system having 12GBytes of available memory (on a server Virtual Machine with 16GB RAM) in Windows Resource monitor. The detailed memory view shows "standby" 12740MB, "Free" 0MB. No errors are seen in Windows Event Viewer

• If the problem was occurring with a specific repository, can you provide the
  URL to that repository to help us with testing?

This is a large internal repository hosted on Github Enterprize.
The Server is on an internal network, the URL cannot be shared.

[liam-baker-sm]

From another machine with Windows 10 Professional, these same commands on the same repository do not cause an error.

`c:\engineering>git clone --mirror git@company.domain:engineering/large_repo.git --branch master --single-branch
Cloning into bare repository 'large_repo.git'...
remote: Enumerating objects: 1344769, done.
remote: Counting objects: 100% (291121/291121), done.
remote: Compressing objects: 100% (32225/32225), done.
remote: Total 1344769 (delta 280050), reused 258946 (delta 258894), pack-reused 1053648
Receiving objects: 100% (1344769/1344769), 3.16 GiB | 40.05 MiB/s, done.

Resolving deltas: 100% (1008885/1008885), done.

c:\engineering\large_repo.git>git fetch --prune --all
remote: Enumerating objects: 2722232, done.
remote: Counting objects: 100% (591028/591028), done.
remote: Compressing objects: 100% (3321/3321), done.
remote: Total 2722232 (delta 589007), reused 588753 (delta 587618), pack-reused 2131204
Receiving objects: 100% (2722232/2722232), 9.64 GiB | 44.06 MiB/s, done.

Resolving deltas: 100% (2131156/2131156), completed with 102172 local objects.`
... (branch names omitted, completed successfully)

[dscho]

Hmm. Maybe experiment with something like this?

[core]
  packedGitLimit = 128m
  packedGitWindowSize = 128m

[pack]
  deltaCacheSize = 128m
  packSizeLimit = 128m
  windowMemory = 128m

[liam-baker-sm]

Thanks.
For now it appears to have been resolved by adding --quiet to the command

git fetch --prune --all --quiet

There is a large amount of output generated (few MB worth of text), possible we are triggering an msys2 bug, as output does not appear to be flushed by ansible until the end of the command.
If we see this issue again, we will try modifying the .gitconfig as you suggested.

[dscho]

possible we are triggering an msys2 bug, as output does not appear to be flushed by ansible until the end of the command.

I think that is unlikely because there is no MSYS2 involved in fetch-pack, and the error message is clearly printed by git.exe (which is distinctly not an MSYS program).

It's strange that the --quiet option should work around the issue. Maybe there's a memory leak somewhere in the code path that is concerned with displaying the information that --quiet suppresses.

[liam-baker-sm]

I now suspect the root cause is a quota set by default in winrm https://learn.microsoft.com/en-us/windows/win32/winrm/quotas
The process is likely right on the edge of the limit for the output to affect it.