🐛 - Bugbot reports passing fixture tests as broken by assuming directory paths and record IDs from their names #834

Is there an existing issue for this?

• I have searched the existing issues.

Related #831 is the Python 3.14 grammar false positive and #832 concerns the explicit partial-snapshot retention contract. These two new findings instead infer fixture values from variable/path names and incorrectly claim passing tests fail.

Which actions are affected?

Pull Request

Which platforms are affected?

Linux
macOS

Description

A completed full-context Bugbot review reports two medium-severity test failures (confidence100/99) that do not occur. Both claims depend on an incorrect inference about a fixture rather than the actual helper return values: treating a database filename as a directory, and treating variable a as a record whose ID is a.

Expected: trace the fixture/helper and concrete stored values before declaring a test broken; use the exact executing runtime/CI as counterevidence. Preserve genuine tests with wrong nested paths or wrong query order. No universal suppression or gate relaxation is requested.

Reproducing the issue

Consumer PR vypdev/domus#7; exact analyzed head4505d82950fc84f2953c2079a245007979fd99bf. Review run/job https://github.com/vypdev/domus/actions/runs/41230777792/job/124624359406; check https://github.com/vypdev/domus/runs/124626029014. The check explicitly says completed, not partial, effort smart,368522ms.

Case A — vypdev/domus#7 (comment), tests/backend/test_sqlite_snapshots.py:334–335.

The finding says snapshot at tmp_path/snapshot is a sibling of the source directory tmp_path/database and cannot trigger destination_inside_source. The actual helper tests/backend/test_household_commands.py:68–70 passes the supplied path directly to SQLiteDatabase as its database FILE. Thus store.path=tmp_path/database, store.path.parent=tmp_path, and snapshot=tmp_path/snapshot IS inside that parent. The test deliberately marks that parent as an owned installation before asserting rejection. It then uses a separate destination and proves success. The alleged failing assertion passes, and the suggested relocation is unnecessary.

Case B — vypdev/domus#7 (comment), tests/backend/test_sqlite_recovery.py:107–108.

The finding says fixture IDs are a and b, so ORDER BY request_id must return (a,b). Actually variable a=request(), whose helper tests/backend/test_recovery_values.py:24–27 constructs request_id="request". Variable b explicitly changes its ID to "b". Ascending order is therefore IDs ("b","request"), exactly the asserted (b,a), and limit1 correctly returns b. The test also covers expiry and terminal-state exclusion. Changing the expectations as suggested would break the correct test.

Reproduction command on the exact unchanged source:

pytest tests/backend/test_sqlite_snapshots.py::test_owned_source_metadata_and_nested_destination_are_checked_before_copy tests/backend/test_sqlite_recovery.py -q
13 passed

The complete exact-head CI passes https://github.com/vypdev/domus/actions/runs/41230777807, including1411 backend tests and100% supported statement/branch coverage. This is executed counterevidence, not a claim based only on test names.

copilot Version

vypdev/copilot@v3 resolved to3e7ceea08e6131360e026937d1c0cc1b120472e0; codex-cli0.166.0, smart effort. Review source SHA and consumer runtime Python3.14.6 are authoritative.

Relevant Log Output

Published finding IDs:

• tests/backend/test_sqlite_snapshots.py:334:destination-is-not-nested
• tests/backend/test_sqlite_recovery.py:107:pending-order-expectation

Canonical review summary at the analyzed head: open4, fixed7, obsolete1, dismissed3; Bugbot review completed. The other two observations concern historical evidence provenance and are being clarified with exact source commits without rewriting old measurements.

Additional context and comments

Suggested regression controls: a fixture returning a database FILE rather than a same-named directory; an actually nonnested destination; a record helper whose ID differs from its variable name; a genuinely inverted query assertion. Require tracing real helper values and preserve executing-runtime/CI counterevidence. Deterministic input/schema checks and actual model false-positive observations must remain distinct.

The two tests are unchanged and should not be rewritten to match these incorrect claims. Evidence-based thread replies/dismissals will retain the original review and exact source links. No Copilot implementation, installed resources, models or acceptance/coverage gate is modified. No owner HA data, credentials or private browser state is included.