Skip to content

RHINENG-26122: remove advisories_account_data - #2345

Open
Dugowitch wants to merge 16 commits into
RedHatInsights:masterfrom
Dugowitch:rm-aad
Open

Dugowitch wants to merge 16 commits into
RedHatInsights:masterfrom
Dugowitch:rm-aad

Conversation

@Dugowitch

@Dugowitch Dugowitch commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

Secure Coding Practices Checklist GitHub Link

Secure Coding Checklist

  • Input Validation
  • Output Encoding
  • Authentication and Password Management
  • Session Management
  • Access Control
  • Cryptographic Practices
  • Error Handling and Logging
  • Data Protection
  • Communication Security
  • System Configuration
  • Database Security
  • File Management
  • Memory Management
  • General Coding Practices

Summary by Sourcery

Replace the legacy advisory account cache with workspace-scoped account_advisory aggregation throughout the application.

Enhancements:

  • Remove the legacy advisory_account_data cache and its evaluator, manager, maintenance-job, monitoring, and recovery code paths in favor of workspace-scoped account_advisory data.
  • Move advisory aggregation ownership to the aggregator service and provision its dedicated database role and permissions.
  • Simplify advisory notification and cache invalidation flows now that aggregation and notifications are handled by the aggregator.

Deployment:

  • Add database migrations to drop advisory_account_data and related functions and triggers, update account_advisory permissions, and configure the aggregator database credentials.
  • Remove obsolete advisory refresh, backfill, cleanup, and recovery jobs from the deployment configuration.

Documentation:

  • Update architecture, database, and migration documentation to describe account_advisory as the sole advisory aggregate store.

Tests:

  • Remove legacy advisory_account_data and cache consistency tests and update remaining tests to use account_advisory refreshes.

Chores:

  • Remove obsolete advisory cache metrics, admin cache-check endpoint, model types, helper functions, and configuration options.

@sourcery-ai

sourcery-ai Bot commented Sep 24, 2026 •

Copy link
Copy Markdown

Reviewer's Guide

This PR completes the migration from the legacy account-wide advisory_account_data cache to workspace-scoped account_advisory aggregates by centralizing refresh and notification ownership in the aggregator, removing evaluator and job-based legacy workflows, introducing dedicated database permissions and deployment credentials, and updating schema, tests, documentation, and monitoring.

File-Level Changes

Change Details Files
Remove the legacy account-wide advisory cache and all code paths that maintained, validated, queried, or monitored it.
  • Delete the model, table references, refresh/backfill/cleanup/recovery jobs, triggers, metrics, admin endpoint, and related utilities.
  • Stop evaluator-side advisory cache updates and instant notification handling.
  • Make manager advisory reads use workspace-scoped account aggregates or tagged queries only.
  • Update fixtures and tests to refresh and validate account_advisory without advisory_account_data.
base/models/models.go
base/database/testing.go
evaluator/evaluate.go
evaluator/evaluate_advisories.go
evaluator/notifications.go
manager/controllers/advisories.go
manager/routes/routes.go
manager/middlewares/prometheus.go
tasks/caches/*
tasks/cleaning/clean_advisory_account_data.go
main.go
dev/test_data.sql
evaluator/*_test.go
tasks/*/*_test.go
Transfer workspace-scoped advisory aggregation ownership and database access to the aggregator component.
  • Configure aggregator to use the aggregator database role and credentials.
  • Grant aggregator read access and account_advisory write/refresh-function permissions.
  • Restrict manager, evaluator, listener, and vmaas_sync account_advisory permissions to their remaining operations.
  • Create the aggregator database user and manage its password during database updates.
database_admin/migrations/170_add_aggregator_role.up.sql
database_admin/migrations/170_add_aggregator_role.down.sql
database_admin/schema/create_schema.sql
database_admin/schema/create_users.sql
database_admin/update.go
deploy/clowdapp.yaml
conf/aggregator_common.env
conf/database_admin.env
Remove the legacy database schema and deployment resources while preserving migration rollback support.
  • Drop advisory_account_data, its validation column, functions, trigger, and associated permissions in migration 169.
  • Remove legacy schema definitions and partition repair references.
  • Add down-migration definitions to recreate the removed table and functions.
  • Remove scheduled cache refresh, cleanup, backfill, and system-advisories recovery jobs and configuration.
database_admin/migrations/169_drop_advisory_account_data.up.sql
database_admin/migrations/169_drop_advisory_account_data.down.sql
database_admin/schema/create_schema.sql
database_admin/schema/repair_system_advisories_0.sql
deploy/clowdapp.yaml
Simplify event and notification contracts around aggregator-owned account notifications.
  • Remove SkipNotifications from platform events and custom-size recovery event publishing.
  • Remove per-inventory notification context and evaluator notification construction.
  • Use account-level notifications produced by the aggregator and update notification tests accordingly.
  • Remove the aggregator drift check after eliminating the legacy cache.
base/mqueue/platform_event.go
base/mqueue/platform_event_test.go
base/notification/notification.go
aggregator/events.go
aggregator/drift_check.go
aggregator/notifications_test.go
evaluator/evaluate.go
evaluator/notifications.go
Update documentation and observability to describe the workspace-scoped aggregation model.
  • Document evaluator ownership of system data and aggregator ownership of account_advisory.
  • Remove legacy table descriptions and recovery instructions.
  • Remove the legacy advisory cache dashboard panels and metrics.
AGENTS.md
docs/md/architecture.md
docs/md/database.md
docs/md/major-migration-runbook.md
dashboards/app-sre/grafana-dashboard-insights-patchman-engine-general.configmap.yaml
docs/admin/openapi.json

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@codecov-commenter

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 89.47368% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 60.25%. Comparing base (2bbe3de) to head (c3a2008).

Files with missing lines Patch % Lines
evaluator/evaluate.go 75.00% 0 Missing and 1 partial ⚠️
manager/middlewares/prometheus.go 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2345      +/-   ##
==========================================
+ Coverage   58.95%   60.25%   +1.29%     
==========================================
  Files         150      144       -6     
  Lines        9605     9077     -528     
==========================================
- Hits         5663     5469     -194     
+ Misses       3348     3044     -304     
+ Partials      594      564      -30     
Flag Coverage Δ
unittests 60.25% <89.47%> (+1.29%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Dugowitch
Dugowitch marked this pull request as ready for review September 25, 2026 12:40
@Dugowitch
Dugowitch requested a review from a team as a code owner September 25, 2026 12:40

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 1 issue

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="base/mqueue/platform_event.go" line_range="79" />
<code_context>
-	if size <= 0 {
-		size = BatchSize
-	}
+func batchSize(grouped map[int][]uuid.UUID) int {
+	// compute how many batches we will create
 	var batches = 0
 	for _, ev := range grouped {
-		batches += (len(ev) + size - 1) / size
+		batches += len(ev)/BatchSize + 1
 	}
 	return batches
</code_context>
<issue_to_address>
**nitpick (performance):** `batchSize` returns one extra batch whenever an account has an exact multiple of `BatchSize` systems; for example, 500 systems produces 2 instead of 1. `WriteEvents` still emits only the correct number of events, but it allocates an unnecessarily large `PlatformEvents` backing array for every such account, increasing memory usage during large event batches.

**Triggers:** When an account's inventory count is an exact multiple of `BatchSize`.

**Suggested fix:** Use `(len(ev) + BatchSize - 1) / BatchSize`, matching the previous calculation, and return zero for empty slices.

```suggestion
		batches += (len(ev) + BatchSize - 1) / BatchSize
```
</issue_to_address>

Sourcery assessment

Needs a human reviewer. The migration drops the production advisory_account_data table and its contents, while the down migration only recreates an empty table, so reverting cannot restore the deleted data. It also changes database roles and permissions, making an incorrect cutover potentially affect both data availability and component access.


Sourcery is free for open source - if you like our reviews please consider sharing them ✨

var batches = 0
for _, ev := range grouped {
batches += (len(ev) + size - 1) / size
batches += len(ev)/BatchSize + 1

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpick (performance): batchSize returns one extra batch whenever an account has an exact multiple of BatchSize systems; for example, 500 systems produces 2 instead of 1. WriteEvents still emits only the correct number of events, but it allocates an unnecessarily large PlatformEvents backing array for every such account, increasing memory usage during large event batches.

Triggers: When an account's inventory count is an exact multiple of BatchSize.

Suggested fix: Use (len(ev) + BatchSize - 1) / BatchSize, matching the previous calculation, and return zero for empty slices.

Suggested change
batches += len(ev)/BatchSize + 1
batches += (len(ev) + BatchSize - 1) / BatchSize

@Dugowitch

Copy link
Copy Markdown
Contributor Author

The PR is ok to review, but wait with merging, notified backport needs to be triggered first

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants