From b892964f7ebb151b697563fecbb2c5e4032af249 Mon Sep 17 00:00:00 2001 From: Johannes Schindelin Date: Wed, 5 Aug 2026 00:18:44 +0200 Subject: [PATCH] packfile: fix perf regression with many packs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Since 589127caa730 (packfile: move list of packs into the packfile store, 2025-10-30), there is a performance regression when many packfiles need to be loaded: `packfile_store_add_pack()` now calls `packfile_list_remove_internal()` to detect whether the packfile was _already_ in the list, and if so, move it to the end of the list. This function linearly scans the existing list before every insertion. Newly loading N packs therefore has complexity O(N²). In one reported use case (https://github.com/microsoft/git/issues/970), N equals 37,815 and caused a slow-down of a simple `git rev-parse --short HEAD` (which is regularly executed as part of `GIT_PS1`) from 0.4s to 4.5s. Let's fix this by establishing a fast path for known-new packfiles. The keen reader will note that there is currently only a single, "known-new" caller of the `packfile_list_append()` function, and wonder why not simply remove this check whether the packfile already exists in the list? Originally, when above-mentioned commit introduced that logic, there was a second caller in `prepare_midx()`, which would have required that check, but that caller was removed in 6aff1f25a046 (packfile: always add packfiles to MRU when adding a pack, 2025-10-30). Still, the function is declared in a header file, and to avoid any problems with in-flight or downstream callers, it is safer to extend the signature to be explicit whether or not to skip that check. Signed-off-by: Johannes Schindelin --- packfile-list.c | 5 +++-- packfile-list.h | 3 ++- packfile.c | 2 +- t/perf/p5303-many-packs.sh | 4 ++++ 4 files changed, 10 insertions(+), 4 deletions(-) diff --git a/packfile-list.c b/packfile-list.c index 01fb913abf78fc..d6d411823c34a9 100644 --- a/packfile-list.c +++ b/packfile-list.c @@ -57,11 +57,12 @@ void packfile_list_prepend(struct packfile_list *list, struct packed_git *pack) list->tail = entry; } -void packfile_list_append(struct packfile_list *list, struct packed_git *pack) +void packfile_list_append(struct packfile_list *list, struct packed_git *pack, + int skip_dup_check) { struct packfile_list_entry *entry; - entry = packfile_list_remove_internal(list, pack); + entry = skip_dup_check ? NULL : packfile_list_remove_internal(list, pack); if (!entry) { entry = xmalloc(sizeof(*entry)); entry->pack = pack; diff --git a/packfile-list.h b/packfile-list.h index 1b05e2aa36de08..2b4b98b22673a4 100644 --- a/packfile-list.h +++ b/packfile-list.h @@ -15,7 +15,8 @@ struct packfile_list_entry { void packfile_list_clear(struct packfile_list *list); void packfile_list_remove(struct packfile_list *list, struct packed_git *pack); void packfile_list_prepend(struct packfile_list *list, struct packed_git *pack); -void packfile_list_append(struct packfile_list *list, struct packed_git *pack); +void packfile_list_append(struct packfile_list *list, struct packed_git *pack, + int skip_dup_check); /* * Find the pack within the "packs" list whose index contains the object diff --git a/packfile.c b/packfile.c index 0eee45055f833e..f80f05a1fec8ea 100644 --- a/packfile.c +++ b/packfile.c @@ -781,7 +781,7 @@ void packfile_store_add_pack(struct odb_source_packed *store, if (pack->pack_fd != -1) pack_open_fds++; - packfile_list_append(&store->packs, pack); + packfile_list_append(&store->packs, pack, 1); strmap_put(&store->packs_by_path, pack->pack_name, pack); } diff --git a/t/perf/p5303-many-packs.sh b/t/perf/p5303-many-packs.sh index af173a7b73e398..4221f9dd706243 100755 --- a/t/perf/p5303-many-packs.sh +++ b/t/perf/p5303-many-packs.sh @@ -141,4 +141,8 @@ test_perf "load 10,000 packs" ' git rev-parse --verify "HEAD^{commit}" ' +test_perf "abbreviate with 10,000 packs" ' + git rev-parse --short HEAD +' + test_done