Skip to content

Commit 73a8d4a

Browse files
blackopsreplblackopsrepl
committed
fix(benchmarks): show each budget's winners in the charts
Give every chart metric separate budget columns with independently ranked solver rows. A shared longest-budget order hides wins at shorter budgets even when the numbers are visible. Charts now use the same budget-specific feasibility, time and quality order as the tables, stacking columns on phones without truncating readouts. The browser gate checks each column's ranking against its own measurements.
1 parent 2390761 commit 73a8d4a

5 files changed

Lines changed: 77 additions & 30 deletions

File tree

‎frontend/styles/index.scss‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -817,6 +817,39 @@ h3 {
817817
line-height: 1.5;
818818
}
819819

820+
.benchmark-chart__budgets {
821+
display: grid;
822+
grid-template-columns: repeat(3, minmax(0, 1fr));
823+
gap: 1.5rem;
824+
}
825+
826+
.benchmark-chart__budget h4 {
827+
margin: 0 0 0.65rem;
828+
font-size: 0.95rem;
829+
}
830+
831+
.benchmark-chart__budget .benchmark-chart__row {
832+
grid-template-columns: minmax(0, 1fr) auto;
833+
gap: 0.4rem;
834+
}
835+
836+
.benchmark-chart__budget .benchmark-chart__label {
837+
grid-column: 1;
838+
grid-row: 1;
839+
}
840+
841+
.benchmark-chart__budget .benchmark-chart__bars {
842+
grid-column: 1 / -1;
843+
grid-row: 2;
844+
grid-template-columns: 1fr;
845+
}
846+
847+
.benchmark-chart__budget .benchmark-chart__value {
848+
grid-column: 2;
849+
grid-row: 1;
850+
grid-template-columns: 1fr;
851+
}
852+
820853
.benchmark-chart__rows {
821854
margin: 0;
822855
padding: 0;
@@ -2121,6 +2154,10 @@ body.use-case-lightbox-open {
21212154
}
21222155

21232156
@media (max-width: 760px) {
2157+
.benchmark-chart__budgets {
2158+
grid-template-columns: 1fr;
2159+
}
2160+
21242161
.benchmark-chart {
21252162
padding: 1rem;
21262163
}

‎scripts/check-public-pages.mjs‎

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1015,13 +1015,15 @@ async function checkBenchmarksPage(origin) {
10151015
issue: `${problem.benchmark_name} ranked solvers ${JSON.stringify(pageRank)} but the measurements rank them ${JSON.stringify(expectedRank)}`,
10161016
});
10171017
}
1018+
const chartRank = [...problem.time_limits_seconds].sort((a, b) => a - b)
1019+
.flatMap((budget) => rankSolvers(problem, budget));
10181020
for (const chart of section.charts) {
1019-
if (chart.solvers.length > 0 && chart.solvers.join("|") !== expectedRank.join("|")) {
1021+
if (chart.solvers.length > 0 && chart.solvers.join("|") !== chartRank.join("|")) {
10201022
failures.push({
10211023
path: "/benchmarks/",
10221024
issue: `${problem.benchmark_name} chart "${chart.caption}" does not rank solvers by feasibility, time, then quality`,
10231025
found: chart.solvers,
1024-
expected: expectedRank,
1026+
expected: chartRank,
10251027
});
10261028
break;
10271029
}
@@ -1064,10 +1066,10 @@ async function checkBenchmarksPage(origin) {
10641066
}
10651067
continue;
10661068
}
1067-
if (chart.rows !== problem.solvers.length) {
1069+
if (chart.rows !== expectedBars) {
10681070
failures.push({
10691071
path: "/benchmarks/",
1070-
issue: `${problem.benchmark_name} chart "${chart.caption}" expected ${problem.solvers.length} solver rows`,
1072+
issue: `${problem.benchmark_name} chart "${chart.caption}" expected ${expectedBars} budget-specific solver rows`,
10711073
found: chart.rows,
10721074
});
10731075
}

‎src/_components/benchmark/charts.erb‎

Lines changed: 31 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
<section class="benchmark-charts" aria-label="<%= @problem.fetch('title') %> summary charts">
2-
<p class="benchmark-charts__ranking">Winner first at the <%= budgets.max %>-second budget: feasibility → time to viable → quality.</p>
2+
<p class="benchmark-charts__ranking">Each budget has its own winner-first ranking: feasibility → time to viable → quality.</p>
33
<div class="benchmark-charts__legend">
44
<% budgets.each do |budget| %>
55
<span class="benchmark-charts__legend-item">
@@ -17,12 +17,15 @@
1717
<h3>Feasible results</h3>
1818
<p>Instances solved to a hard-feasible solution, out of <%= total_instances %>.</p>
1919
</figcaption>
20-
<ol class="benchmark-chart__rows">
21-
<% solvers.each do |solver| %>
20+
<div class="benchmark-chart__budgets">
21+
<% budgets.each do |budget| %>
22+
<div class="benchmark-chart__budget" data-budget="<%= budget %>">
23+
<h4><%= budget %>-second budget</h4>
24+
<ol class="benchmark-chart__rows">
25+
<% solvers(budget).each do |solver| %>
2226
<li class="benchmark-chart__row<%= ' benchmark-chart__row--solverforge' if solverforge?(solver) %>">
2327
<span class="benchmark-chart__label"><%= solver %></span>
2428
<span class="benchmark-chart__bars">
25-
<% budgets.each do |budget| %>
2629
<% row = summary(budget, solver) %>
2730
<% percent = feasible_percent(row) %>
2831
<span
@@ -40,17 +43,17 @@
4043
<span class="benchmark-chart__bar benchmark-chart__bar--missing" data-benchmark-missing="true"></span>
4144
<% end %>
4245
</span>
43-
<% end %>
4446
</span>
4547
<span class="benchmark-chart__value">
46-
<% budgets.each do |budget| %>
4748
<% row = summary(budget, solver) %>
4849
<span><small><%= budget %>s</small> <%= row ? "#{row.fetch('feasible')}/#{row.fetch('total')}" : "—" %></span>
49-
<% end %>
5050
</span>
5151
</li>
5252
<% end %>
53-
</ol>
53+
</ol>
54+
</div>
55+
<% end %>
56+
</div>
5457
</figure>
5558

5659
<%# Then time to a viable solution. Read after feasibility and before
@@ -73,12 +76,15 @@
7376
<% end %>
7477
</figcaption>
7578
<% if drawable_times? %>
76-
<ol class="benchmark-chart__rows">
77-
<% solvers.each do |solver| %>
79+
<div class="benchmark-chart__budgets">
80+
<% budgets.each do |budget| %>
81+
<div class="benchmark-chart__budget" data-budget="<%= budget %>">
82+
<h4><%= budget %>-second budget</h4>
83+
<ol class="benchmark-chart__rows">
84+
<% solvers(budget).each do |solver| %>
7885
<li class="benchmark-chart__row<%= ' benchmark-chart__row--solverforge' if solverforge?(solver) %>">
7986
<span class="benchmark-chart__label"><%= solver %></span>
8087
<span class="benchmark-chart__bars">
81-
<% budgets.each do |budget| %>
8288
<% row = summary(budget, solver) %>
8389
<% seconds = feasible_seconds(row) %>
8490
<% fill_percent = fill(seconds, max_feasible_seconds) %>
@@ -96,17 +102,17 @@
96102
<span class="benchmark-chart__bar benchmark-chart__bar--missing" data-benchmark-missing="true"></span>
97103
<% end %>
98104
</span>
99-
<% end %>
100105
</span>
101106
<span class="benchmark-chart__value">
102-
<% budgets.each do |budget| %>
103107
<% row = summary(budget, solver) %>
104108
<span><small><%= budget %>s</small> <%= row && row['mean_feasible_seconds'] ? format('%.2f s', row['mean_feasible_seconds']) : "—" %></span>
105-
<% end %>
106109
</span>
107110
</li>
108111
<% end %>
109-
</ol>
112+
</ol>
113+
</div>
114+
<% end %>
115+
</div>
110116
<% end %>
111117
</figure>
112118

@@ -131,12 +137,15 @@
131137
<% end %>
132138
</figcaption>
133139
<% if drawable_gaps? %>
134-
<ol class="benchmark-chart__rows">
135-
<% solvers.each do |solver| %>
140+
<div class="benchmark-chart__budgets">
141+
<% budgets.each do |budget| %>
142+
<div class="benchmark-chart__budget" data-budget="<%= budget %>">
143+
<h4><%= budget %>-second budget</h4>
144+
<ol class="benchmark-chart__rows">
145+
<% solvers(budget).each do |solver| %>
136146
<li class="benchmark-chart__row<%= ' benchmark-chart__row--solverforge' if solverforge?(solver) %>">
137147
<span class="benchmark-chart__label"><%= solver %></span>
138148
<span class="benchmark-chart__bars">
139-
<% budgets.each do |budget| %>
140149
<% row = summary(budget, solver) %>
141150
<% gap = gap_percent(row) %>
142151
<% fill_percent = fill(gap, max_gap) %>
@@ -154,17 +163,17 @@
154163
<span class="benchmark-chart__bar benchmark-chart__bar--missing" data-benchmark-missing="true"></span>
155164
<% end %>
156165
</span>
157-
<% end %>
158166
</span>
159167
<span class="benchmark-chart__value">
160-
<% budgets.each do |budget| %>
161168
<% row = summary(budget, solver) %>
162169
<span><small><%= budget %>s</small> <%= row && row['gap_percent'] ? format('%.2f%%', row['gap_percent']) : "—" %></span>
163-
<% end %>
164170
</span>
165171
</li>
166172
<% end %>
167-
</ol>
173+
</ol>
174+
</div>
175+
<% end %>
176+
</div>
168177
<% end %>
169178
</figure>
170179
</div>

‎src/_components/benchmark/charts.rb‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -13,8 +13,8 @@ def budgets
1313
@budgets ||= @problem.fetch("time_limits_seconds").sort
1414
end
1515

16-
def solvers
17-
@solvers ||= @problem.fetch("solvers")
16+
def solvers(budget)
17+
@problem.fetch("solver_orders").fetch(budget.to_s)
1818
end
1919

2020
def total_instances

‎src/benchmarks.md‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -11,8 +11,7 @@ Quick and partial runs are excluded; these are not measurements of newer release
1111
**Ranking:** feasibility → time to a viable solution → solution quality.
1212
Each priority breaks ties in the one before it. Time averages only feasible results;
1313
quality averages only feasible results with a reference. **0% gap matches the reference.**
14-
Each table ranks its own budget. Charts use the longest-budget ranking, matching
15-
the first table. Problems are ranked separately.
14+
Each chart column and table ranks its own budget. Problems are ranked separately.
1615

1716
<% site.data.benchmarks.problems.each do |problem| %>
1817
<section class="benchmark-problem" id="<%= problem.fetch('benchmark_name') %>">

0 commit comments

Comments
 (0)