Skip to content

[Feature] Benchmarking Updates #153

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
MJC598 merged 5 commits into from
Aug 20, 2026
Merged

[Feature] Benchmarking Updates #153

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
ddd38c1
[Feature] Indexing a Simulation (#150)
MJC598 Jul 22, 2026
b423f81
[Feature] Timestep Index Operator (#151)
MJC598 Jul 27, 2026
9dfa4a3
fixing the run function in benchmarking
MJC598 Aug 19, 2026
a88f0a4
fixing benchmarking to utilize the deep copy and produce correct chec…
MJC598 Aug 19, 2026
0a5ed06
fixing release workflow name collision
MJC598 Aug 19, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .github/workflows/release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,10 @@ jobs:
cmake --preset package-${{matrix.build-variant}} && cmake --build --preset package-${{matrix.build-variant}}
- name: Run CPack
run: |
cpack --preset ${{matrix.package-type}}-${{matrix.build-variant}}
cpack --preset ${{matrix.package-type}}-${{matrix.build-variant}} \
-D CPACK_PACKAGE_NAME=respond-${{matrix.build-variant}} \
-D CPACK_RPM_PACKAGE_NAME=respond-${{matrix.build-variant}} \
-D CPACK_DEBIAN_PACKAGE_NAME=respond-${{matrix.build-variant}}

- name: Upload Linux Package to Release
uses: svenstaro/upload-release-action@v2
Expand Down
29 changes: 16 additions & 13 deletions extras/benchmark/src/benchmark_respond.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
// Created Date: 2026-04-27 //
// Author: Matthew Carroll //
// ----- //
// Last Modified: 2026-07-09 //
// Last Modified: 2026-08-18 //
// Modified By: Matthew Carroll //
// ----- //
// Copyright (c) 2026 Syndemics Lab at Boston Medical Center //
Expand Down Expand Up @@ -55,7 +55,7 @@ inline void ClobberMemory() {
#endif

struct BenchmarkConfig {
std::size_t state_size = 200;
std::size_t state_size = 64;
int steps = 365;
int warmup_iterations = 5;
int sample_iterations = 25;
Expand Down Expand Up @@ -158,7 +158,7 @@ void PrintUsage(const char *program_name) {
std::cout
<< "Usage: " << program_name << " [options]\n\n"
<< "Options:\n"
<< " --state-size <n> Number of state dimensions (default: 200)\n"
<< " --state-size <n> Number of state dimensions (default: 64)\n"
<< " --steps <n> Number of simulation timesteps per sample "
"(default: 365)\n"
<< " --warmup <n> Warm-up iterations per repetition (default: "
Expand Down Expand Up @@ -280,8 +280,6 @@ respond::Timestep CreateTestTimestep(std::size_t state_size) {
ts.CreateTransition("behavior");
ts.AddMatrixToTransition("behavior", MakeShiftMatrix(state_size, 0.985, 1));

auto temp = ts.GetTransition("behavior")->clone();

ts.CreateTransition("intervention");
ts.AddMatrixToTransition("intervention",
MakeShiftMatrix(state_size, 0.990, -1));
Expand All @@ -304,31 +302,36 @@ respond::Simulation BuildSimulation(std::size_t state_size,
size_t duration) {
respond::Simulation sim;
sim.CreateNewModel("markov");
sim.GetModels()[0]->SetHistoryCaptureInterval(history_capture_interval);
sim.GetModels()[0]->SetFinalTimestep(duration);
sim[0]->SetHistoryCaptureInterval(history_capture_interval);
sim[0]->SetFinalTimestep(static_cast<int>(duration));
auto timestep = CreateTestTimestep(state_size);

for (size_t t = 0; t < duration; ++t) {
sim.GetModels()[0]->AddTimestep(timestep);
sim[0]->AddTimestep(timestep);
}
return sim;
}

TimedRunResult TimeOneSample(respond::Simulation sim,
const Eigen::VectorXd &initial_state, int steps) {
sim.GetModels()[0]->SetState(initial_state);
sim.GetModels()[0]->CreateDefaultHistories();
sim[0]->SetState(initial_state);
sim[0]->CreateDefaultHistories();

const auto start = Clock::now();
sim.Run(steps);
const auto end = Clock::now();

const double checksum = sim.GetModels()[0]->GetState().sum();
if (sim[0]->GetTimestep() != steps) {
throw std::runtime_error("Benchmark ran an unexpected number of "
"timesteps.");
}

const double checksum = sim[0]->GetState().sum();
std::size_t recorded_points = 0;
const auto histories = sim.GetModelHistories()[0];
const auto &histories = sim.GetModelHistory(0);
const auto state_history = histories.find("state");
if (state_history != histories.end()) {
recorded_points = state_history->second.size();
recorded_points = state_history->second.GetRecordedStates().size();
}
DoNotOptimize(checksum);
ClobberMemory();
Expand Down
2 changes: 1 addition & 1 deletion tests/unit/simulation_test.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
// Created Date: 2026-02-09 //
// Author: Matthew Carroll //
// ----- //
// Last Modified: 2026-07-15 //
// Last Modified: 2026-08-19 //
// Modified By: Matthew Carroll //
// ----- //
// Copyright (c) 2026 Syndemics Lab at Boston Medical Center //
Expand Down
2 changes: 1 addition & 1 deletion tests/unit/timestep_test.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
// Created Date: 2026-07-06 //
// Author: Matthew Carroll //
// ----- //
// Last Modified: 2026-07-09 //
// Last Modified: 2026-08-19 //
// Modified By: Matthew Carroll //
// ----- //
// Copyright (c) 2026 Syndemics Lab at Boston Medical Center //
Expand Down