ValueError: unexpected delta opcode 0

[scosenza]

Summary

While using Aider, which uses gitdb==4.0.10 (I also see the same issue when upgrading to 4.0.11) the following fatal failure occurs:

aider
...
Use /help to see in-chat commands, run with --help to see cmd line args
Traceback (most recent call last):
  File "/opt/homebrew/bin/aider", line 33, in <module>
    sys.exit(load_entry_point('aider-chat', 'console_scripts', 'aider')())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/steve_cosenza/workspace/aider/aider/main.py", line 576, in main
    coder.run()
  File "/Users/steve_cosenza/workspace/aider/aider/coders/base_coder.py", line 319, in run
    new_user_message = self.run_loop()
                       ^^^^^^^^^^^^^^^
  File "/Users/steve_cosenza/workspace/aider/aider/coders/base_coder.py", line 390, in run_loop
    self.get_addable_relative_files(),
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/steve_cosenza/workspace/aider/aider/coders/base_coder.py", line 714, in get_addable_relative_files
    return set(self.get_all_relative_files()) - set(self.get_inchat_relative_files())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/steve_cosenza/workspace/aider/aider/coders/base_coder.py", line 696, in get_all_relative_files
    files = self.repo.get_tracked_files()
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/steve_cosenza/workspace/aider/aider/repo.py", line 187, in get_tracked_files
    for blob in commit.tree.traverse():
  File "/opt/homebrew/lib/python3.11/site-packages/git/objects/util.py", line 526, in _traverse
    addToStack(stack, item, branch_first, nd)
  File "/opt/homebrew/lib/python3.11/site-packages/git/objects/util.py", line 488, in addToStack
    lst = self._get_intermediate_items(item)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/git/objects/tree.py", line 239, in _get_intermediate_items
    return tuple(index_object._iter_convert_to_object(index_object._cache))
                                                      ^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/gitdb/util.py", line 253, in __getattr__
    self._set_cache_(attr)
  File "/opt/homebrew/lib/python3.11/site-packages/git/objects/tree.py", line 246, in _set_cache_
    self._cache: List[TreeCacheTup] = tree_entries_from_data(ostream.read())
                                                             ^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/gitdb/base.py", line 138, in read
    return self[3].read(size)
           ^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/gitdb/stream.py", line 487, in read
    bl = self._size - self._br      # bytes left
         ^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/gitdb/util.py", line 253, in __getattr__
    self._set_cache_(attr)
  File "/opt/homebrew/lib/python3.11/site-packages/gitdb/stream.py", line 461, in _set_cache_brute_
    apply_delta_data(bbuf, src_size, ddata, len(ddata), tbuf.write)
  File "/opt/homebrew/lib/python3.11/site-packages/gitdb/fun.py", line 673, in apply_delta_data
    raise ValueError("unexpected delta opcode 0")
ValueError: unexpected delta opcode 0

Repro Details

• The git repo being used is a large Github Enterprise hosted monorepo
• I'm running Aider/gitdb on a Mac M2 running Sonoma 14.0
• The following git client version is installed: 2.40.0

[Byron]

I can only say that I am very surprised this pure-python implementation is used at all, and sorry for the issues it will cause.

It's a crazy-complicated 'streaming' implementation for the delta-decompression to decode packed objects, which makes it slow and wasteful. It's probably also riddled with bugs that show up like this.

If it must be pure python, I recommend using dulwich instead which I'd hope has a simpler, and better, implementation, as nothing in gitdb will be fixed - it's something I want to get rid of also so that GitPython won't offer it as backend anymore.

Thanks for your understanding.