commit-reach: remove get_reachable_subset()#2144
Open
spkrka wants to merge 1 commit into
Open
Conversation
Author
|
/submit |
|
Submitted as pull.2144.git.1781033285419.gitgitgadget@gmail.com To fetch this version into To fetch this version to local tag |
|
Junio C Hamano wrote on the Git mailing list (how to reply to this email): "Kristofer Karlsson via GitGitGadget" <gitgitgadget@gmail.com>
writes:
> get_reachable_subset() was introduced in fcb2c0769d (2018-11-02)
> for add_missing_tags() in remote.c. tips_reachable_from_bases()
> was added in cbfe360b14 (2023-03-20) as part of the ahead-behind
> series. The two were never consolidated.
Good finding. It is curious to see that these were from the same
author.
> ... Without generation numbers, some edge cases
> may be slower with DFS instead of BFS since the date-ordered
> prio_queue naturally stays near the top of the graph, but this
> should not matter in practice
"should not matter in practice" because...?
> -- worst case both visit the full
> graph down from the bases.
And of course the worst case scenario is by definition not a typical
case that appear in practice, so it does not make a good explanation
for "should not matter in practice".
> The flag in remote.c changes from 1 (bit 0) to TMP_MARK (bit 4)
> because tips_reachable_from_bases() uses SEEN (bit 0) internally.
> TMP_MARK is already used for deduplication earlier in the same
> function and is cleared before the reachability check.
And tips_reachable_from_bases() clears SEEN at the end as expected.
> commit-reach.c | 73 -------------------------------------------
> commit-reach.h | 13 --------
> remote.c | 19 ++++++-----
> t/helper/test-reach.c | 39 +++++++++++------------
> t/t6600-test-reach.sh | 18 +++++------
> 5 files changed, 36 insertions(+), 126 deletions(-)
Yay, a lot of deletions ;-)
> diff --git a/t/t6600-test-reach.sh b/t/t6600-test-reach.sh
> index b5b314e570..51b140a539 100755
> --- a/t/t6600-test-reach.sh
> +++ b/t/t6600-test-reach.sh
> @@ -391,7 +391,7 @@ test_expect_success 'rev-list: symmetric difference topo-order' '
> run_all_modes git rev-list --topo-order commit-3-8...commit-6-6
> '
>
> -test_expect_success 'get_reachable_subset:all' '
> +test_expect_success 'tips_reachable_from_bases:all' '
> cat >input <<-\EOF &&
> X:commit-9-1
> X:commit-8-3
> @@ -403,15 +403,15 @@ test_expect_success 'get_reachable_subset:all' '
> Y:commit-5-6
> EOF
> (
> - echo "get_reachable_subset(X,Y)" &&
> + echo "tips_reachable_from_bases(X,Y)" &&
> git rev-parse commit-3-3 \
> commit-1-7 \
> commit-5-6 | sort
> ) >expect &&
> - test_all_modes get_reachable_subset
> + test_all_modes tips_reachable_from_bases
> '
>
> -test_expect_success 'get_reachable_subset:some' '
> +test_expect_success 'tips_reachable_from_bases:some' '
> cat >input <<-\EOF &&
> X:commit-9-1
> X:commit-8-3
> @@ -422,14 +422,14 @@ test_expect_success 'get_reachable_subset:some' '
> Y:commit-5-6
> EOF
> (
> - echo "get_reachable_subset(X,Y)" &&
> + echo "tips_reachable_from_bases(X,Y)" &&
> git rev-parse commit-3-3 \
> commit-1-7 | sort
> ) >expect &&
> - test_all_modes get_reachable_subset
> + test_all_modes tips_reachable_from_bases
> '
>
> -test_expect_success 'get_reachable_subset:none' '
> +test_expect_success 'tips_reachable_from_bases:none' '
> cat >input <<-\EOF &&
> X:commit-9-1
> X:commit-8-3
> @@ -439,8 +439,8 @@ test_expect_success 'get_reachable_subset:none' '
> Y:commit-7-6
> Y:commit-2-8
> EOF
> - echo "get_reachable_subset(X,Y)" >expect &&
> - test_all_modes get_reachable_subset
> + echo "tips_reachable_from_bases(X,Y)" >expect &&
> + test_all_modes tips_reachable_from_bases
> '
>
> test_expect_success 'for-each-ref ahead-behind:linear' '
>
> base-commit: 600fe743028cbfb640855f659e9851522214bc0b
Initially I feared that changes to the test script were a sign of
need to adjuist to behaviour changes, but as the proposed log
message explained, all of the above changes are about the name of
the function being used and tested, which makes sense.
Thanks. |
spkrka
force-pushed
the
krka/remove-get-reachable-subset-v2
branch
3 times, most recently
from
28bca5c to
6ff5968
Compare
|
Kristofer Karlsson wrote on the Git mailing list (how to reply to this email): Junio C Hamano <gitster@pobox.com> writes:
> "should not matter in practice" because...?
>
> And of course the worst case scenario is by definition not a typical
> case that appear in practice, so it does not make a good explanation
> for "should not matter in practice".
You are right, that was hand-wavy. I started with writing a somewhat
long analysis but after finding a clean way of supporting both DFS
and priority queue modes it feels somewhat unnecessary - I will
still include it here, but the short summary is that it's fixable.
Since the prio_queue struct supports both LIFO and heap mode,
it's actually quite easy to plug this into
tips_reachable_from_bases. I just need to switch away from using
the stack structure and pass a mode to choose between LIFO
and heap. This preserves the old behavior while still reducing
the code size and unifying the code more.
I will submit a v2 of the patch shortly.
I will also include the original analysis I wrote before finding
the simple fix.
Thanks for the review!
Kristofer
---
I will refer to the prio_queue approach in get_reachable_subset
as PQ for brevity, and the DFS in tips_reachable_from_bases as
simply DFS.
tips_reachable_from_bases() was designed for --merged queries where
the targets (branches/tags) can be deep ancestors of the base. DFS
is a natural fit there: it dives deep along first-parent quickly,
and with generation numbers the dynamic floor raising prunes
aggressively.
add_missing_tags() has the opposite shape: the bases are branch
tips being pushed (near the top) and the targets are tag commits
the remote does not have yet, which tend to be relatively close
to those tips. PQ ordered by commit date is a better fit here
because it sweeps down from the top and finds nearby targets early,
while DFS might take a long detour down a side branch before
coming back.
With a commit-graph this difference mostly disappears since the
generation floor keeps DFS from going too far off track. Without a
commit-graph, neither approach prunes anything (generation is
GENERATION_NUMBER_INFINITY for all commits) so the traversal order
is the only thing that matters, and PQ has the edge for shallow
targets.
So the current code actually has each caller matched to the
traversal strategy that fits its typical workload. My patch traded
that away for code reduction.
That said, in practice the difference is limited: repositories
large enough for this to matter typically have a commit-graph,
and small repositories are fast either way.
--- |
|
User |
|
Derrick Stolee wrote on the Git mailing list (how to reply to this email): On 6/10/2026 11:48 AM, Junio C Hamano wrote:
> "Kristofer Karlsson via GitGitGadget" <gitgitgadget@gmail.com>
> writes:
>
>> get_reachable_subset() was introduced in fcb2c0769d (2018-11-02)
>> for add_missing_tags() in remote.c. tips_reachable_from_bases()
>> was added in cbfe360b14 (2023-03-20) as part of the ahead-behind
>> series. The two were never consolidated.
>
> Good finding. It is curious to see that these were from the same
> author.
I agree. In my defense, these commits are five years apart. I still
should have looked for similar code that could be reused instead of
rolling new code. (But the new code is better when a commit-graph
exists.)
The other thing that I should have done in the later commit was add
the method to the test-tool, which you do here.
>> ... Without generation numbers, some edge cases
>> may be slower with DFS instead of BFS since the date-ordered
>> prio_queue naturally stays near the top of the graph, but this
>> should not matter in practice
>
> "should not matter in practice" because...?
>
>> -- worst case both visit the full
>> graph down from the bases.
>
> And of course the worst case scenario is by definition not a typical
> case that appear in practice, so it does not make a good explanation
> for "should not matter in practice".
It's important to recognize the use cases that call each method and
to understand if it is appropriate to take these performance changes.
Both methods terminate in the case that all potential targets are
found. And that's the only case that matters, as we will walk all
reachable commits in the case of any one commit not being reachable.
Both methods avoid walking below the "minimum generation" among the
target commits.
The key opportunity here is that tips_reachable_from_bases() will
"increase" the minimum generation when it finds the current-minimum
target commit. That's a big reason why the DFS approach wins: it
has the opportunity to find those lower commits without needing to
walk _every_ commit with higher generation.
The one downside to this approach is that the DFS approach does not
take into account the commit date as a fallback when there is no
commit-graph file with computed generation numbers. When there is
no commit-graph file, then the fallback to commit date to break ties
among "generation number infinity" commits can't be used to help the
BFS search in get_reachable_subset().
And perhaps that is the critical reason for the different algorithms:
in 2018 we didn't have the commit-graph for very long so it wasn't a
reasonable expectation that we'd have one, even for large repositories.
Now? The feature is quite stable and it's easy for users to create
and maintain one. All servers are expected to use it for performance
needs. It's probably reasonable to expect that any repos where this
would matter would have one.
Thanks,
-Stolee
|
|
This branch is now known as |
|
This patch series was integrated into seen via git@ed96572. |
get_reachable_subset() and tips_reachable_from_bases() both answer the same reachability question but use different traversal strategies: priority queue vs depth-first search. Consolidate them into tips_reachable_from_bases() with a mode parameter to select between DFS and PQ traversal, preserving the preferred strategy for each caller. This works cleanly because prio_queue already supports LIFO mode (when compare is NULL), so a single prio_queue acts as either a stack or a heap depending on the mode. The unified traversal pushes all unseen parents at once rather than peeking and pushing one parent at a time. This eliminates merge commit revisits entirely: a 2-parent merge now requires 1 visit instead of 3. For DFS (LIFO) mode, the first parent is pushed last so it ends up on top of the stack, preserving first-parent traversal order. Parsing is deferred to pop time for DFS since parent objects carry valid flags without a full repo_parse_commit() call. PQ mode parses before push so the heap can order by generation number. Add exhaustive reachability tests that use every commit in the grid as a tip, protecting against subtle traversal bugs such as wrong parent ordering or premature pruning. The existing tests are also extended to exercise both DFS and PQ modes. The flag in remote.c changes from 1 (bit 0) to TMP_MARK (bit 4) because tips_reachable_from_bases() uses SEEN (bit 0) internally. TMP_MARK is already used for deduplication earlier in the same function and is cleared before the reachability check. Signed-off-by: Kristofer Karlsson <krka@spotify.com>
spkrka
force-pushed
the
krka/remove-get-reachable-subset-v2
branch
from
6ff5968 to
da01032
Compare
Author
|
/submit |
|
Submitted as pull.2144.v2.git.1781178567862.gitgitgadget@gmail.com To fetch this version into To fetch this version to local tag |
|
Derrick Stolee wrote on the Git mailing list (how to reply to this email): On 6/11/2026 7:49 AM, Kristofer Karlsson via GitGitGadget wrote:
> From: Kristofer Karlsson <krka@spotify.com>
> * Added PQ mode to the existing test-reach tests so both DFS and PQ
> paths are exercised by the test suite.
This is a substantial change that I don't think is merited. I
think that this makes the point of your change moot: we essentially
have two implementations in one complicated method instead of two
implementations that have different performance characteristics.
I'd rather leave the code as-is than take this complication. I don't
think your commit message justifies the merging of these
implementations, either.
Moreover, I thought the previous patch was fine, it just needed better
awareness of the performance implications of the change. Specifically,
it could be a regression for large repos without a commit-graph file
while simultaneously potentially being a performance boost for large
repos _with_ a commit-graph file.
_If_ we were to go this direction, then it should be two patches, with
the first introducing the new mode and testing it. The second patch
could change the callers of get_reachable_subset() and delete that
code.
Finally, a commentary: You seem to have a habit of responding to
review feedback only through new patch versions, but I'd rather see
some thoughts in the discussion thread as direct replies to the review,
especially if you think you will change direction like this. Saying
something like "Maybe I should update the method to have two walk modes"
in a reply would have given me an opportunity to respond and perhaps
avoided a new version that went in this direction.
Thanks,
-Stolee
|
|
Kristofer Karlsson wrote on the Git mailing list (how to reply to this email): On Thu, 11 Jun 2026 at 14:57, Derrick Stolee <stolee@gmail.com> wrote:
> Finally, a commentary: You seem to have a habit of responding to
> review feedback only through new patch versions, but I'd rather see
> some thoughts in the discussion thread as direct replies to the review,
> especially if you think you will change direction like this. Saying
> something like "Maybe I should update the method to have two walk modes"
> in a reply would have given me an opportunity to respond and perhaps
> avoided a new version that went in this direction.
That's fair, I apologize both for jumping ahead too quickly with a new
patch and also for evolving it into multiple logical changes
(both code removal and complex refactoring).
I will be more mindful going forward about letting the discussion
settle more before submitting followup patches.
I have no strong opinion on how to proceed - either park/abandon this
or go with v1 as-is. I think you're right that having two modes within
tips_reachable_from_bases is reducing the win here unless the mode is
truly seamless but the abstraction does leak through a bit.
Thanks,
Kristofer |
|
Derrick Stolee wrote on the Git mailing list (how to reply to this email): On 6/11/2026 9:52 AM, Kristofer Karlsson wrote:
> On Thu, 11 Jun 2026 at 14:57, Derrick Stolee <stolee@gmail.com> wrote:
>> Finally, a commentary: You seem to have a habit of responding to
>> review feedback only through new patch versions, but I'd rather see
>> some thoughts in the discussion thread as direct replies to the review,
>> especially if you think you will change direction like this. Saying
>> something like "Maybe I should update the method to have two walk modes"
>> in a reply would have given me an opportunity to respond and perhaps
>> avoided a new version that went in this direction.
>
> That's fair, I apologize both for jumping ahead too quickly with a new
> patch and also for evolving it into multiple logical changes
> (both code removal and complex refactoring).
>
> I will be more mindful going forward about letting the discussion
> settle more before submitting followup patches.
>
> I have no strong opinion on how to proceed - either park/abandon this
> or go with v1 as-is. I think you're right that having two modes within
> tips_reachable_from_bases is reducing the win here unless the mode is
> truly seamless but the abstraction does leak through a bit.
I think that my biggest issue is that callers are needing to know
something about the performance characteristics of each solution. It
may be better to keep the behavior completely internal: have the
method decide which approach is better based on the information it
has. For instance: is the minimum generation number "infinite"? Then
the BFS is going to be better than the DFS approach. Such an approach
would make it clear why there is the complexity _and_ would improve
both callers in the right scenarios.
You were correct to find two methods that claimed to do the same
thing, but we need to take time to consider the solutions based on
all factors.
Thanks,
-Stolee
|
|
This patch series is no longer integrated into seen. |
|
There was a status update in the "New Topics" section about the branch API clean-up. Needs review. source: <pull.2144.v2.git.1781178567862.gitgitgadget@gmail.com> |
|
Junio C Hamano wrote on the Git mailing list (how to reply to this email): Derrick Stolee <stolee@gmail.com> writes:
> Finally, a commentary: You seem to have a habit of responding to
> review feedback only through new patch versions, but I'd rather see
> some thoughts in the discussion thread as direct replies to the review,
> especially if you think you will change direction like this. Saying
> something like "Maybe I should update the method to have two walk modes"
> in a reply would have given me an opportunity to respond and perhaps
> avoided a new version that went in this direction.
Thanks for saying this.
I haven't (yet) found it in my exchange with Kristofer, but I did
find similar irritations during review sessions with other
contributors.
I wonder if we should talk about it in the SubmittingPatches and/or
MyFirstContribution document? |
|
This patch series was integrated into seen via git@895e4b0. |
|
This patch series is no longer integrated into seen. |
|
Weijie Yuan wrote on the Git mailing list (how to reply to this email): On Thu, Jun 11, 2026 at 10:48:18AM -0700, Junio C Hamano wrote:
> I wonder if we should talk about it in the SubmittingPatches and/or
> MyFirstContribution document?
Hi, I think it might be a good idea to cover these details in
MyFirstContribution, then cross-reference them from the part of
SubmittingPatches that discusses sending a new version.
More specifically, I think these details could fit around steps 3 and 4
of "A typical life cycle of a patch series" in SubmittingPatches,
starting around line 54. That section may need some reworking of the
existing wording, rather than just an additive change.
Also, for the part about sending a new version, I wonder whether it
would be better to summarize and fold in Patrick's explanation here,
thank you Patrick:
---
From: Patrick Steinhardt <ps@pks.im>
Message-ID: <aietF4BX1Ewt3cpG@pks.im>
> By the way, how long should I wait before sending new versions of my
> patches? I have 4 outstanding at the moment.
I typically aim to send at most one version per day per patch series.
This avoids that you're "flooding" the mailing list with too many
versions of the same series, allows you to address feedback from
multiple folks in batches, and it gives you enough time to think about
the feedback without having to rush anything.
Whether I actually do end up sending a series depends on a couple of
factors:
- How big is the series? The bigger it is the more time I give folks
to perform reviews.
- How substantial were the reviews you received? Is it just a couple
of small typos? Then it probably makes sense to wait one or two more
days to get some more involved reviews. Is it something that
requires signifciant rework? Then I'd send out soon so that others
don't review a patch series that will change significantly anyway.
- How close to being merged is the series? The closer it is the less
substantial the reviews will (hopefully) get, so it makes sense to
reroll a bit faster even if you only received minor feedback.
So there isn't really a golden rule to follow here, but a lot of this
depends on gut feeling. You probably won't have that feeling yet when
starting out in a new project, but that's fine. In case we see that
behaviour doesn't quite match the norm we'll typically give a hint that
the contributor should slow down or maybe send a new iteration.
Patrick
---
Patrick's point may be beyond the scope of this thread, so I only
mention it as an aside. Maybe these could be part of the same series.
Thanks. |
|
User |
|
This patch series was integrated into seen via git@a11f9d4. |
|
Junio C Hamano wrote on the Git mailing list (how to reply to this email): Weijie Yuan <wy@wyuan.org> writes:
> On Thu, Jun 11, 2026 at 10:48:18AM -0700, Junio C Hamano wrote:
>> I wonder if we should talk about it in the SubmittingPatches and/or
>> MyFirstContribution document?
>
> Hi, I think it might be a good idea to cover these details in
> MyFirstContribution, then cross-reference them from the part of
> SubmittingPatches that discusses sending a new version.
Sorry to be nitpicky, but the above is omitting too much from your
quote. "it" in "talk about it" is totally unclear to a reader who
haven't seen the message you are replying to.
> Also, for the part about sending a new version, I wonder whether it
> would be better to summarize and fold in Patrick's explanation here,
> thank you Patrick:
Yup, that is a great example.
> From: Patrick Steinhardt <ps@pks.im>
> Message-ID: <aietF4BX1Ewt3cpG@pks.im> |
|
Weijie Yuan wrote on the Git mailing list (how to reply to this email): On Fri, Jun 12, 2026 at 07:41:48AM -0700, Junio C Hamano wrote:
> Weijie Yuan <wy@wyuan.org> writes:
>
> > On Thu, Jun 11, 2026 at 10:48:18AM -0700, Junio C Hamano wrote:
> >> I wonder if we should talk about it in the SubmittingPatches and/or
> >> MyFirstContribution document?
> >
> > Hi, I think it might be a good idea to cover these details in
> > MyFirstContribution, then cross-reference them from the part of
> > SubmittingPatches that discusses sending a new version.
>
> Sorry to be nitpicky, but the above is omitting too much from your
> quote. "it" in "talk about it" is totally unclear to a reader who
> haven't seen the message you are replying to.
Oops, so sorry! You are not nitpicky at all, this is totally my
carelessness and fault. Sorry readers!
Thank you for catching this! It shows that I still need to really
understand the previous patch I wrote, and put it into real practice:
> It is usually helpful to trim away unrelated context, such as large
> portions of the patch that are not being discussed, while _keeping
> enough quoted text_ for readers to understand *what* you are
> responding to.
Thank you! I'll immediately set a solid "pre-reply" hook in my .git
folder ;-) |
|
Weijie Yuan wrote on the Git mailing list (how to reply to this email): On Sat, Jun 13, 2026 at 12:16:34AM +0800, Weijie Yuan wrote:
> On Fri, Jun 12, 2026 at 07:41:48AM -0700, Junio C Hamano wrote:
> > Weijie Yuan <wy@wyuan.org> writes:
> >
> > > On Thu, Jun 11, 2026 at 10:48:18AM -0700, Junio C Hamano wrote:
> > >> I wonder if we should talk about it in the SubmittingPatches and/or
> > >> MyFirstContribution document?
> > >
> > > Hi, I think it might be a good idea to cover these details in
> > > MyFirstContribution, then cross-reference them from the part of
> > > SubmittingPatches that discusses sending a new version.
> >
> > Sorry to be nitpicky, but the above is omitting too much from your
> > quote. "it" in "talk about it" is totally unclear to a reader who
> > haven't seen the message you are replying to.
>
> Oops, so sorry! You are not nitpicky at all, this is totally my
> carelessness and fault. Sorry readers!
>
> Thank you for catching this! It shows that I still need to really
> understand the previous patch I wrote, and put it into real practice:
>
> > It is usually helpful to trim away unrelated context, such as large
> > portions of the patch that are not being discussed, while _keeping
> > enough quoted text_ for readers to understand *what* you are
> > responding to.
>
> Thank you! I'll immediately set a solid "pre-reply" hook in my .git
> folder ;-)
Sorry, here I mean setting a pre-reply hook to remind me how to do a
good quote when writting a reply mail :-)
Sorry for the unnecessary noise, and thank you. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This removes get_reachable_subset() and consolidates its only caller
into tips_reachable_from_bases() with a mode parameter to select
between DFS and priority queue traversal. Both functions answer the
same reachability query and use the same generation-number pruning
strategy, differing primarily in traversal order (DFS vs
generation-ordered PQ). They were introduced years apart and never
consolidated.
The unified function uses prio_queue for both modes: LIFO (stack)
when compare is NULL, min-heap when given a comparator.
Changes since v1:
Replaced the commit_stack + reverse-push pattern with a simpler
approach: push all unseen parents directly, with the first parent
pushed last so it lands on top of the LIFO stack. This preserves
first-parent DFS order without needing a temporary stack to
reverse parent order.
Deferred repo_parse_commit() to pop time for DFS mode. Parent
objects carry valid flags without a full parse, so we avoid
parsing already-SEEN parents in the inner loop. PQ mode still
parses before push so the heap can order by generation number.
Moved the generation floor check from the parent loop to pop
time, simplifying the hot path at the cost of occasionally
enqueueing commits that will later be discarded.
Added SEEN flag on base commits before enqueueing, preventing
duplicate processing if bases overlap with the traversal.
Added PQ mode to the existing test-reach tests so both DFS and
PQ paths are exercised by the test suite.
Added two new reachability tests that use all 100
commits in the grid as tips - the idea is to better detect subtle
traversal bugs.
Notes for reviewers:
The flag in remote.c changes from 1 to TMP_MARK because
tips_reachable_from_bases() uses SEEN (bit 0) internally.
TMP_MARK is already used earlier in the same function and
is cleared before the reachability block.
Test helper and test names rename from get_reachable_subset
to tips_reachable_from_bases to match the function being
exercised.
As Stolee noted in the v1 review, commit-graph is now stable and
expected for repositories where this matters, making the DFS approach
with dynamic floor raising an attractive default. The mode parameter
could be dropped in a future patch if we decide to use DFS everywhere,
but preserving each caller's existing strategy keeps this change
conservative and avoids any behavior change for add_missing_tags().
Performance was not a goal of this refactoring, but the simplified
DFS traversal turned out to be a pleasant surprise -- eliminating
merge commit revisits and deferring parse to pop time gives a
consistent speedup across repositories:
Benchmark results (median, 30 runs, pre-built binaries):
cc: Derrick Stolee stolee@gmail.com
cc: Kristofer Karlsson krka@spotify.com
cc: Weijie Yuan wy@wyuan.org