Skip to content

Replace Logging Piccolo benchmark with Locust - #8206

Open
Amaury Chamayou (achamayou) wants to merge 16 commits into
mainfrom
achamayou-logging-locust-benchmark
Open

Replace Logging Piccolo benchmark with Locust#8206
Amaury Chamayou (achamayou) wants to merge 16 commits into
mainfrom
achamayou-logging-locust-benchmark

Conversation

@achamayou

@achamayou Amaury Chamayou (achamayou) commented Aug 24, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Summary

  • replace the certificate-authenticated Piccolo pi_ls benchmark with pi_logging_cert_blocking_locust
  • send user-certificate-authenticated POST /app/log/blocking/private requests and wait for commit before each Locust user sends its next request
  • reuse Replace Logging JWT perf test with Locust #8201's shared Locust orchestration and one logging workload implementation for certificate and JWT authentication
  • run 640 users at a 640 users/s spawn rate across 10 Locust processes, measure for 20 seconds at a 20ms signature interval, and write across 1,000 record IDs
  • publish the result under the distinct Logging Certificate Blocking Locust Bencher label

Depends on #8201.

The old Logging result measured 100,000 pipelined requests to the non-blocking /app/log/private endpoint with Piccolo and --max-writes-ahead 1000. This replacement keeps normal user-certificate authentication, but measures one outstanding blocking-on-commit request per Locust user. The results are not directly comparable, so this does not reuse the old Logging label.

Validation

  • configured CMake with BUILD_TESTS=ON and WORKER_THREADS=2, then inspected the generated certificate and JWT CTest commands
  • parsed both benchmark CLIs and discovered the shared workload with Locust in certificate and JWT modes
  • exercised certificate selection, TLS certificate/key propagation, request endpoint/body/headers, and preserved JWT bearer handling in a focused Python check
  • compiled the changed Python files with py_compile
  • ran Black, Ruff, gersemi, and the repository ASCII check
  • ran two client-count sweeps in the Benchmark A/B workflow:
Users Throughput p50 latency p99 latency Implied mean latency Memory
320 15,079.2 tx/s 20 ms 26 ms 21.2 ms 90.7 MiB
480 22,006.6 tx/s 20 ms 24 ms 21.8 ms 102.7 MiB
560 24,984.2 tx/s 20 ms 27 ms 22.4 ms 110.9 MiB
640 25,193.8 tx/s 21 ms 37 ms 25.4 ms 117.7 MiB
720 24,312.7 tx/s 23 ms 42 ms 29.6 ms 125.3 MiB
800 24,270.6 tx/s 27 ms 46 ms 33.0 ms 132.1 MiB
900 18,929.5 tx/s 44 ms 72 ms 47.5 ms 140.2 MiB
1,000 16,704.5 tx/s 62 ms 97 ms 59.9 ms 152.4 MiB

The curve has a broad 560-640-user plateau rather than a sharp peak: 560 users already reach 99.2% of the 640-user throughput. Each user keeps one blocking request outstanding with no think time, so below saturation throughput is approximately the client count divided by the roughly 20ms commit latency. Around 560 users, the service reaches its approximately 25,000 tx/s capacity. Additional users then add queueing rather than useful work: implied mean latency rises from 22.4ms at 560 users to 33.0ms at 800, then requests increasingly wait across multiple commit intervals at 900 and 1,000 users while throughput falls.

The benchmark keeps 640 users because it reliably saturates the service with some headroom against run-to-run variation. Two separate single-count 640-user runs reproduced 24,922.5 and 25,381.6 tx/s, both with 21ms p50 and 37ms p99.

@achamayou
Replace Logging JWT perf test with Locust
e528a06 
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@achamayou
Merge branch 'main' into achamayou-logging-jwt-locust-benchmark
d510c07
@achamayou
Validate Locust spawn rate
b10b8a8 
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@achamayou
Strengthen Locust benchmark validation tests
50eeb02 
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@achamayou
Remove Locust infrastructure tests
3a1a81d 
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@achamayou
Trim redundant benchmark logging
cd566c8 
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@achamayou
Use one signing interval for JWT benchmark
e1b0e33 
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@achamayou
Merge branch 'main' into achamayou-logging-jwt-locust-benchmark
8af0ff2
@achamayou
Replace Logging Piccolo benchmark with Locust
e0d7832 
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Base automatically changed from achamayou-logging-jwt-locust-benchmark to main August 25, 2026 14:57
@achamayou
Merge main into Logging Locust benchmark
1d0d10e 
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 893d95e8-6329-4be1-a1b4-7740b8a9d97e
@github-actions

github-actions Bot commented Aug 25, 2026 •
edited
Loading

Copy link
Copy Markdown
Description

Comparing 5 available runs from this branch (#8206) against the trend of the last 30 main runs.

Each chart plots every benchmark as an axis, with values normalized so 100 is the EWMA baseline of recent main runs, using a 7-run half-life. The 5 orange branch lines run from the oldest (faintest) to the latest (darkest and thickest); the darker blue band is the main baseline +/- 1 std dev and the lighter blue band around it is +/- 2 std dev.

Axis labels show the latest branch value and its difference from the main EWMA baseline, where 0% is on the baseline. They are coloured green where the latest run improves on the baseline, red where it regresses, and grey where the difference is within one std dev of the baseline (within noise). Higher is better for throughput and rate, lower for latency and memory.

A benchmark which does not exist on main yet has no baseline of its own, so its earliest available run from this branch is used as its reference and its band is measured across this branch's runs. Its axis is normalized, scaled and coloured like any other, but the comparison is against this branch rather than against main.

Throughput (tx/s)

---
config:
  radar:
    width: 620
    height: 620
    marginTop: 90
    marginRight: 220
    marginBottom: 60
    marginLeft: 220
    axisLabelFactor: 1.12
    curveTension: 0.08
  theme: base
  themeCSS: |
    .radarCurve-0{fill:color-mix(in srgb, #62B5E5 13%, var(--color-canvas-default,var(--bgColor-default,#fff)))!important;fill-opacity:1!important;stroke:none!important;stroke-width:0!important}
    .radarCurve-1{fill:color-mix(in srgb, #62B5E5 40%, var(--color-canvas-default,var(--bgColor-default,#fff)))!important;fill-opacity:1!important;stroke:none!important;stroke-width:0!important}
    .radarCurve-2{fill:color-mix(in srgb, #62B5E5 13%, var(--color-canvas-default,var(--bgColor-default,#fff)))!important;fill-opacity:1!important;stroke:none!important;stroke-width:0!important}
    .radarCurve-3{fill:var(--color-canvas-default,var(--bgColor-default,#fff))!important;fill-opacity:1!important;stroke:none!important;stroke-width:0!important}
    .radarAxisLabel,.radarTitle{fill:var(--color-fg-default,var(--fgColor-default,#111827))!important;color:var(--color-fg-default,var(--fgColor-default,#111827))!important}
    .radarCurve-4{stroke-width:1.5px!important;stroke-opacity:0.40!important}
    .radarCurve-5{stroke-width:1.5px!important;stroke-opacity:0.50!important}
    .radarCurve-6{stroke-width:1.75px!important;stroke-opacity:1.00!important}
    .radarAxisLabel:nth-of-type(1){fill:#808A94!important}
    .radarAxisLabel:nth-of-type(2){fill:#808A94!important}
    .radarAxisLabel:nth-of-type(3){fill:#808A94!important}
    .radarAxisLabel:nth-of-type(4){fill:#808A94!important}
    .radarAxisLabel:nth-of-type(5){fill:#808A94!important}
    .radarAxisLabel:nth-of-type(6){fill:#808A94!important}
    .radarAxisLabel:nth-of-type(7){fill:#808A94!important}
    .radarAxisLabel:nth-of-type(8){fill:#2DA44E!important}
    .radarAxisLabel:nth-of-type(9){fill:#2DA44E!important}
  themeVariables:
    cScale0: "#62B5E5"
    cScale1: "#62B5E5"
    cScale2: "#62B5E5"
    cScale3: "#62B5E5"
    cScale4: "#F97316"
    cScale5: "#F97316"
    cScale6: "#F97316"
    radar:
      axisColor: "#9CA3AF"
      graticuleColor: "#E5E7EB"
      graticuleOpacity: 0
      axisStrokeWidth: 1
      curveOpacity: 0
---
radar-beta
  axis b0["Basic: 72,614 tx/s ▬ +3%"]
  axis b2["Basic Blocking Locust 100ms: 3,059 tx/s ▬ 0%"]
  axis b3["Basic Blocking Locust 20ms: 15,159 tx/s ▬ 0%"]
  axis b4["Basic Blocking Locust 2ms: 33,068 tx/s ▬ +3%"]
  axis b5["Basic JS: 4,739 tx/s ▬ 0%"]
  axis b6["Basic Multi-Threaded: 87,622 tx/s ▬ +1%"]
  axis b7["Historical Queries: 207,330 tx/s ▬ -1%"]
  axis b9["L…g C…e Blocking Locust: 25,382 tx/s ▲ 68%"]
  axis b11["L…g JWT Blocking Locust: 14,943 tx/s ▲ 2%"]
  curve stddev2_high["main EWMA + 2 std dev"]{107.51, 100.59, 100.68, 109.19, 107.14, 106.81, 111.50, 163.01, 101.60}
  curve stddev1_high["main EWMA + 1 std dev"]{103.76, 100.29, 100.34, 104.60, 103.57, 103.40, 105.75, 131.50, 100.80}
  curve stddev1_low["main EWMA - 1 std dev"]{96.24, 99.71, 99.66, 95.40, 96.43, 96.60, 94.25, 68.50, 99.20}
  curve stddev2_low["main EWMA - 2 std dev"]{92.49, 99.41, 99.32, 90.81, 92.86, 93.19, 88.50, 36.99, 98.40}
  curve branch_0["#8206 (4 runs earlier)"]{102.83, 99.72, 100.03, 103.26, 101.51, 100.48, 99.66, 100.00, 100.14}
  curve branch_2["#8206 (2 runs earlier)"]{101.50, 99.98, 100.16, 103.49, 99.34, 99.48, 102.21, 165.25, 100.78}
  curve branch_4["#8206"]{102.86, 100.15, 100.01, 103.25, 99.93, 100.91, 99.30, 168.30, 101.63}
  graticule polygon
  max 215
  ticks 0
  showLegend false
Loading

Latency (ms)

---
config:
  radar:
    width: 620
    height: 620
    marginTop: 90
    marginRight: 220
    marginBottom: 60
    marginLeft: 220
    axisLabelFactor: 1.12
    curveTension: 0.08
  theme: base
  themeCSS: |
    .radarCurve-0{fill:color-mix(in srgb, #62B5E5 13%, var(--color-canvas-default,var(--bgColor-default,#fff)))!important;fill-opacity:1!important;stroke:none!important;stroke-width:0!important}
    .radarCurve-1{fill:color-mix(in srgb, #62B5E5 40%, var(--color-canvas-default,var(--bgColor-default,#fff)))!important;fill-opacity:1!important;stroke:none!important;stroke-width:0!important}
    .radarCurve-2{fill:color-mix(in srgb, #62B5E5 13%, var(--color-canvas-default,var(--bgColor-default,#fff)))!important;fill-opacity:1!important;stroke:none!important;stroke-width:0!important}
    .radarCurve-3{fill:var(--color-canvas-default,var(--bgColor-default,#fff))!important;fill-opacity:1!important;stroke:none!important;stroke-width:0!important}
    .radarAxisLabel,.radarTitle{fill:var(--color-fg-default,var(--fgColor-default,#111827))!important;color:var(--color-fg-default,var(--fgColor-default,#111827))!important}
    .radarCurve-4{stroke-width:1.5px!important;stroke-opacity:0.40!important}
    .radarCurve-5{stroke-width:1.5px!important;stroke-opacity:0.50!important}
    .radarCurve-6{stroke-width:1.75px!important;stroke-opacity:1.00!important}
    .radarAxisLabel:nth-of-type(1){fill:#808A94!important}
    .radarAxisLabel:nth-of-type(2){fill:#808A94!important}
    .radarAxisLabel:nth-of-type(3){fill:#808A94!important}
    .radarAxisLabel:nth-of-type(4){fill:#E5484D!important}
    .radarAxisLabel:nth-of-type(5){fill:#808A94!important}
    .radarAxisLabel:nth-of-type(6){fill:#808A94!important}
    .radarAxisLabel:nth-of-type(7){fill:#E5484D!important}
    .radarAxisLabel:nth-of-type(8){fill:#808A94!important}
  themeVariables:
    cScale0: "#62B5E5"
    cScale1: "#62B5E5"
    cScale2: "#62B5E5"
    cScale3: "#62B5E5"
    cScale4: "#F97316"
    cScale5: "#F97316"
    cScale6: "#F97316"
    radar:
      axisColor: "#9CA3AF"
      graticuleColor: "#E5E7EB"
      graticuleOpacity: 0
      axisStrokeWidth: 1
      curveOpacity: 0
---
radar-beta
  axis b0["Basic Blocking Locust 100ms: 99 ms ▬ 0%"]
  axis b1["Basic Blocking Locust 20ms: 19 ms ▬ 0%"]
  axis b2["Basic Blocking Locust 2ms: 9 ms ▬ 0%"]
  axis b3["Commit Latency 16ms: 10.1 ms ▲ 118%"]
  axis b4["Commit Latency 1ms: 1.95 ms ▬ 0%"]
  axis b5["Commit Latency 256ms: 214 ms ▬ 0%"]
  axis b6["L…g Certificate Blocking Locust: 21 ms ▲ 11%"]
  axis b7["Logging JWT Blocking Locust: 20 ms ▬ 0%"]
  curve stddev2_high["main EWMA + 2 std dev"]{100.00, 100.00, 108.27, 152.75, 104.41, 102.84, 109.92, 100.00}
  curve stddev1_high["main EWMA + 1 std dev"]{100.00, 100.00, 104.13, 126.38, 102.21, 101.42, 104.96, 100.00}
  curve stddev1_low["main EWMA - 1 std dev"]{100.00, 100.00, 95.87, 73.62, 97.79, 98.58, 95.04, 100.00}
  curve stddev2_low["main EWMA - 2 std dev"]{100.00, 100.00, 91.73, 47.25, 95.59, 97.16, 90.08, 100.00}
  curve branch_0["#8206 (4 runs earlier)"]{100.00, 100.00, 89.06, 103.42, 99.04, 100.62, 100.00, 100.00}
  curve branch_2["#8206 (2 runs earlier)"]{100.00, 100.00, 100.19, 91.07, 101.90, 100.49, 110.53, 100.00}
  curve branch_4["#8206"]{100.00, 100.00, 100.19, 217.83, 100.23, 100.32, 110.53, 100.00}
  graticule polygon
  max 278
  ticks 0
  showLegend false
Loading

Memory (bytes)

---
config:
  radar:
    width: 620
    height: 620
    marginTop: 90
    marginRight: 220
    marginBottom: 60
    marginLeft: 220
    axisLabelFactor: 1.12
    curveTension: 0.08
  theme: base
  themeCSS: |
    .radarCurve-0{fill:color-mix(in srgb, #62B5E5 13%, var(--color-canvas-default,var(--bgColor-default,#fff)))!important;fill-opacity:1!important;stroke:none!important;stroke-width:0!important}
    .radarCurve-1{fill:color-mix(in srgb, #62B5E5 40%, var(--color-canvas-default,var(--bgColor-default,#fff)))!important;fill-opacity:1!important;stroke:none!important;stroke-width:0!important}
    .radarCurve-2{fill:color-mix(in srgb, #62B5E5 13%, var(--color-canvas-default,var(--bgColor-default,#fff)))!important;fill-opacity:1!important;stroke:none!important;stroke-width:0!important}
    .radarCurve-3{fill:var(--color-canvas-default,var(--bgColor-default,#fff))!important;fill-opacity:1!important;stroke:none!important;stroke-width:0!important}
    .radarAxisLabel,.radarTitle{fill:var(--color-fg-default,var(--fgColor-default,#111827))!important;color:var(--color-fg-default,var(--fgColor-default,#111827))!important}
    .radarCurve-4{stroke-width:1.5px!important;stroke-opacity:0.40!important}
    .radarCurve-5{stroke-width:1.5px!important;stroke-opacity:0.50!important}
    .radarCurve-6{stroke-width:1.75px!important;stroke-opacity:1.00!important}
    .radarAxisLabel:nth-of-type(1){fill:#E5484D!important}
    .radarAxisLabel:nth-of-type(2){fill:#808A94!important}
    .radarAxisLabel:nth-of-type(3){fill:#E5484D!important}
    .radarAxisLabel:nth-of-type(4){fill:#E5484D!important}
    .radarAxisLabel:nth-of-type(5){fill:#808A94!important}
    .radarAxisLabel:nth-of-type(6){fill:#808A94!important}
    .radarAxisLabel:nth-of-type(7){fill:#E5484D!important}
    .radarAxisLabel:nth-of-type(8){fill:#2DA44E!important}
  themeVariables:
    cScale0: "#62B5E5"
    cScale1: "#62B5E5"
    cScale2: "#62B5E5"
    cScale3: "#62B5E5"
    cScale4: "#F97316"
    cScale5: "#F97316"
    cScale6: "#F97316"
    radar:
      axisColor: "#9CA3AF"
      graticuleColor: "#E5E7EB"
      graticuleOpacity: 0
      axisStrokeWidth: 1
      curveOpacity: 0
---
radar-beta
  axis b0["Basic: 90.4 MiB ▲ 2%"]
  axis b2["Basic Blocking Locust 100ms: 90.1 MiB ▬ 0%"]
  axis b3["Basic Blocking Locust 20ms: 92.1 MiB ▲ 1%"]
  axis b4["Basic Blocking Locust 2ms: 94.9 MiB ▲ 1%"]
  axis b5["Basic JS: 70.6 MiB ▬ 0%"]
  axis b6["Basic Multi-Threaded: 88.7 MiB ▬ -1%"]
  axis b8["L…g C…e Blocking Locust: 120 MiB ▲ 31%"]
  axis b10["Logging JWT Blocking Locust: 89.8 MiB ▼ 3%"]
  curve stddev2_high["main EWMA + 2 std dev"]{104.24, 102.15, 101.64, 101.83, 103.83, 102.57, 128.29, 102.24}
  curve stddev1_high["main EWMA + 1 std dev"]{102.12, 101.07, 100.82, 100.91, 101.91, 101.29, 114.15, 101.12}
  curve stddev1_low["main EWMA - 1 std dev"]{97.88, 98.93, 99.18, 99.09, 98.09, 98.71, 85.85, 98.88}
  curve stddev2_low["main EWMA - 2 std dev"]{95.76, 97.85, 98.36, 98.17, 96.17, 97.43, 71.71, 97.76}
  curve branch_0["#8206 (4 runs earlier)"]{101.54, 99.93, 101.31, 99.24, 103.45, 98.82, 100.00, 99.99}
  curve branch_2["#8206 (2 runs earlier)"]{94.70, 100.51, 99.11, 99.89, 99.91, 97.28, 128.88, 99.20}
  curve branch_4["#8206"]{102.48, 99.53, 101.32, 101.42, 99.90, 99.49, 131.02, 97.32}
  graticule polygon
  max 152
  min 50
  ticks 0
  showLegend false
Loading

Rate (ops/s)

---
config:
  radar:
    width: 620
    height: 620
    marginTop: 90
    marginRight: 220
    marginBottom: 60
    marginLeft: 220
    axisLabelFactor: 1.12
    curveTension: 0.08
  theme: base
  themeCSS: |
    .radarCurve-0{fill:color-mix(in srgb, #62B5E5 13%, var(--color-canvas-default,var(--bgColor-default,#fff)))!important;fill-opacity:1!important;stroke:none!important;stroke-width:0!important}
    .radarCurve-1{fill:color-mix(in srgb, #62B5E5 40%, var(--color-canvas-default,var(--bgColor-default,#fff)))!important;fill-opacity:1!important;stroke:none!important;stroke-width:0!important}
    .radarCurve-2{fill:color-mix(in srgb, #62B5E5 13%, var(--color-canvas-default,var(--bgColor-default,#fff)))!important;fill-opacity:1!important;stroke:none!important;stroke-width:0!important}
    .radarCurve-3{fill:var(--color-canvas-default,var(--bgColor-default,#fff))!important;fill-opacity:1!important;stroke:none!important;stroke-width:0!important}
    .radarAxisLabel,.radarTitle{fill:var(--color-fg-default,var(--fgColor-default,#111827))!important;color:var(--color-fg-default,var(--fgColor-default,#111827))!important}
    .radarCurve-4{stroke-width:1.5px!important;stroke-opacity:0.20!important}
    .radarCurve-5{stroke-width:1.5px!important;stroke-opacity:0.30!important}
    .radarCurve-6{stroke-width:1.5px!important;stroke-opacity:0.40!important}
    .radarCurve-7{stroke-width:1.5px!important;stroke-opacity:0.50!important}
    .radarCurve-8{stroke-width:1.75px!important;stroke-opacity:1.00!important}
    .radarAxisLabel:nth-of-type(1){fill:#808A94!important}
    .radarAxisLabel:nth-of-type(2){fill:#808A94!important}
    .radarAxisLabel:nth-of-type(3){fill:#808A94!important}
    .radarAxisLabel:nth-of-type(4){fill:#808A94!important}
    .radarAxisLabel:nth-of-type(5){fill:#808A94!important}
    .radarAxisLabel:nth-of-type(6){fill:#808A94!important}
  themeVariables:
    cScale0: "#62B5E5"
    cScale1: "#62B5E5"
    cScale2: "#62B5E5"
    cScale3: "#62B5E5"
    cScale4: "#F97316"
    cScale5: "#F97316"
    cScale6: "#F97316"
    cScale7: "#F97316"
    cScale8: "#F97316"
    radar:
      axisColor: "#9CA3AF"
      graticuleColor: "#E5E7EB"
      graticuleOpacity: 0
      axisStrokeWidth: 1
      curveOpacity: 0
---
radar-beta
  axis b0["CHAMP get: 37,883,132 ops/s ▬ 0%"]
  axis b1["CHAMP put: 5,430,488 ops/s ▬ 0%"]
  axis b2["KV deserialisation: 1,636,661 ops/s ▬ +1%"]
  axis b3["KV serialisation: 1,412,230 ops/s ▬ -2%"]
  axis b4["KV s…t deserialisation: 4,191 ops/s ▬ +1%"]
  axis b5["KV snapshot serialisation: 4,593 ops/s ▬ +3%"]
  curve stddev2_high["main EWMA + 2 std dev"]{105.58, 106.40, 106.20, 107.16, 105.65, 111.89}
  curve stddev1_high["main EWMA + 1 std dev"]{102.79, 103.20, 103.10, 103.58, 102.83, 105.94}
  curve stddev1_low["main EWMA - 1 std dev"]{97.21, 96.80, 96.90, 96.42, 97.17, 94.06}
  curve stddev2_low["main EWMA - 2 std dev"]{94.42, 93.60, 93.80, 92.84, 94.35, 88.11}
  curve branch_0["#8206 (4 runs earlier)"]{101.04, 100.82, 99.09, 98.69, 100.33, 104.05}
  curve branch_1["#8206 (3 runs earlier)"]{100.46, 101.19, 98.77, 99.68, 99.81, 101.14}
  curve branch_2["#8206 (2 runs earlier)"]{100.19, 102.58, 100.22, 99.81, 99.00, 106.18}
  curve branch_3["#8206 (1 run earlier)"]{99.49, 101.53, 102.73, 97.57, 101.74, 103.85}
  curve branch_4["#8206"]{100.07, 99.94, 100.86, 98.26, 100.83, 103.25}
  graticule polygon
  max 121
  min 79
  ticks 0
  showLegend false
Loading

@achamayou
Amaury Chamayou (achamayou) marked this pull request as ready for review August 25, 2026 19:44
@achamayou
Amaury Chamayou (achamayou) requested a review from a team as a code owner August 25, 2026 19:44
Copilot AI lite review requested due to automatic review settings August 25, 2026 19:44
Copilot AI reviewed Aug 25, 2026
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates CCF’s performance benchmarking suite by replacing the legacy Piccolo-based certificate-authenticated logging benchmark with a Locust-based benchmark, aligning it with the newer shared Locust orchestration and workload approach (as introduced in the dependent PR #8201).

Changes:

  • Added a new certificate-authenticated Locust benchmark entrypoint (tests/logging_locust.py) for blocking-on-commit logging writes.
  • Updated the JWT logging benchmark entrypoint to reuse the shared logging_locustfile.py workload and explicitly select JWT authentication via CLI.
  • Extended the shared logging Locust workload to support both certificate and JWT authentication modes, and updated CMake to register the new perf test target and Bencher label.

Custom instructions used:

  • .github/copilot-instructions.md

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

File Description
tests/logging_locust.py New Locust benchmark entrypoint for user-certificate-authenticated blocking logging writes.
tests/logging_jwt_locust.py Switches JWT benchmark to the shared logging locustfile and passes explicit --authentication jwt.
tests/infra/logging_locustfile.py Generalizes the logging Locust workload to support both certificate and JWT authentication.
CMakeLists.txt Replaces the old Piccolo pi_ls perf test with a Locust-driven pi_logging_cert_blocking_locust perf test and new Bencher label.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@achamayou
Sweep Locust certificate client counts
b14e349 
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@achamayou
Tolerate slow benchmark package feeds
43a5819 
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@achamayou
Use 640 clients for logging benchmark
fe39053 
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@achamayou
Sweep Locust saturation knee
9750bc7 
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@achamayou
Restore 640-client benchmark
c25ddf1 
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot
Copilot review effort
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

bench-ab

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@achamayou