Skip to content

[ISSUE #7263& ISSUE #7264] consumption offset data and topic consumerqueue not deleted on the standby node - #7273

Open
frinda wants to merge 6 commits into
apache:developfrom
frinda:develop
Open

[ISSUE #7263& ISSUE #7264] consumption offset data and topic consumerqueue not deleted on the standby node#7273
frinda wants to merge 6 commits into
apache:developfrom
frinda:develop

Conversation

@frinda

@frinda frinda commented Aug 28, 2023 •
edited
Loading

Copy link
Copy Markdown
Contributor

Which Issue(s) This PR Fixes

Fixes #7263 and #7264

Brief Description

How Did You Test This Change?

w30003204 and others added 4 commits January 5, 2023 09:01
w30003204
[ISSUE apache#5816] fix the issue that No response is returned when n…
fcd6ac3 
…o parameter is configured for the brokerStatus command.(apache#5818)
@frinda
Merge branch 'apache:develop' into develop
d988f29
w30003204
[ISSUE apache#7263] fix delete synchronously the consumption offset d…
54fdbae 
…ata of the subscription group on the standby node, when execute command 'mqadmin deleteSubGroup -b xxx -g xxx -r true'.
w30003204
[ISSUE apache#7264] Fix to delete synchronously topic consume queue o…
5aea257 
…n the standby node, when execute command 'mqadmin deleteTopic -b xxx -t xxx to delete unused topic'.
@frinda frinda changed the title [ISSUE #7263] fix delete synchronously the consumption offset data of the subscription group on the standby node, when execute command 'mqadmin deleteSubGroup -b xxx -g xxx -r true'. [ISSUE #7263] Fix to delete synchronously the consumption offset data of the subscription group on the standby node, when execute command 'mqadmin deleteSubGroup -b xxx -g xxx -r true' and [ISSUE #7264]Fix to delete synchronously topic consume queue on the standby node, when execute command 'mqadmin deleteTopic -b xxx -t xxx to delete unused topic'. Aug 28, 2023
@frinda frinda changed the title [ISSUE #7263] Fix to delete synchronously the consumption offset data of the subscription group on the standby node, when execute command 'mqadmin deleteSubGroup -b xxx -g xxx -r true' and [ISSUE #7264]Fix to delete synchronously topic consume queue on the standby node, when execute command 'mqadmin deleteTopic -b xxx -t xxx to delete unused topic'. [ISSUE #7263] Fix to delete synchronously the consumption offset data of the subscription group on the standby node, when execute command 'mqadmin deleteSubGroup -b xxx -g xxx -r true'. [ISSUE #7264]Fix to delete synchronously topic consume queue on the standby node, when execute command 'mqadmin deleteTopic -b xxx -t xxx to delete unused topic'. Aug 28, 2023
@codecov-commenter

codecov-commenter commented Aug 28, 2023 •
edited
Loading

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 48.75%. Comparing base (1117ab8) to head (d9a18c1).
⚠️ Report is 153 commits behind head on develop.

Additional details and impacted files 
@@              Coverage Diff              @@
##             develop    #7273      +/-   ##
=============================================
- Coverage      48.85%   48.75%   -0.10%     
+ Complexity     12769    12745      -24     
=============================================
  Files           1324     1324              
  Lines          96405    96409       +4     
  Branches       12434    12434              
=============================================
- Hits           47094    47008      -86     
- Misses         43568    43642      +74     
- Partials        5743     5759      +16

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

joeCarf
joeCarf reviewed Aug 29, 2023
View reviewed changes

@joeCarf joeCarf left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the title might be too long, suggest renaming [ISSUE xx& ISSUE xxx] Fix: xxx

@frinda frinda changed the title [ISSUE #7263] Fix to delete synchronously the consumption offset data of the subscription group on the standby node, when execute command 'mqadmin deleteSubGroup -b xxx -g xxx -r true'. [ISSUE #7264]Fix to delete synchronously topic consume queue on the standby node, when execute command 'mqadmin deleteTopic -b xxx -t xxx to delete unused topic'. [ISSUE #7263& ISSUE #7264]Fix to delete synchronously the consumption offset data of the subscription group on the standby node, when execute command 'mqadmin deleteSubGroup -b xxx -g xxx -r true' and delete synchronously topic consume queue on the standby node, when execute command 'mqadmin deleteTopic -b xxx -t xxx to delete unused topic'. Aug 30, 2023
@iamgd67

iamgd67 commented Nov 8, 2023

Copy link
Copy Markdown
Contributor

lgtm

this bothers me too. I copy master's consumerqueue data to slave and restart slave to fix

@frinda

frinda commented Nov 30, 2023

Copy link
Copy Markdown
Contributor Author

seek for integration

@frinda

frinda commented Dec 4, 2023 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

I think the title might be too long, suggest renaming [ISSUE xx& ISSUE xxx] Fix: xxx

@ joeCarf It's been changed. Please submit it，Thanks

@iamgd67

iamgd67 commented Dec 4, 2023

Copy link
Copy Markdown
Contributor

I think the title might be too long, suggest renaming [ISSUE xx& ISSUE xxx] Fix: xxx

@ joeCarf It's been changed. Please submit it，Thanks

still too long, maybe this?

[ISSUE #7263& ISSUE #7264] consumption offset data and topic consumerqueue not deleted on the standby node

@frinda frinda changed the title [ISSUE #7263& ISSUE #7264]Fix to delete synchronously the consumption offset data of the subscription group on the standby node, when execute command 'mqadmin deleteSubGroup -b xxx -g xxx -r true' and delete synchronously topic consume queue on the standby node, when execute command 'mqadmin deleteTopic -b xxx -t xxx to delete unused topic'. [ISSUE #7263& ISSUE #7264] Fix to cleanup consumption offset data or topic consumerqueue on the standby node when execute command 'deleteSubGroup' or 'deleteTopic' Dec 5, 2023
@frinda

frinda commented Dec 7, 2023

Copy link
Copy Markdown
Contributor Author

I think the title might be too long, suggest renaming [ISSUE xx& ISSUE xxx] Fix: xxx

@ joeCarf It's been changed. Please submit it，Thanks

still too long, maybe this?

[ISSUE #7263& ISSUE #7264] consumption offset data and topic consumerqueue not deleted on the standby node

has changed

@frinda

frinda commented Dec 7, 2023

Copy link
Copy Markdown
Contributor Author

I think the title might be too long, suggest renaming [ISSUE xx& ISSUE xxx] Fix: xxx

@joeCarf seek for integration

@frinda
frinda requested a review from joeCarf December 7, 2023 08:38
joeCarf
joeCarf approved these changes Dec 8, 2023
View reviewed changes

@joeCarf joeCarf left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@frinda

frinda commented Dec 9, 2023

Copy link
Copy Markdown
Contributor Author

Begging to join

@frinda

frinda commented Dec 11, 2023

Copy link
Copy Markdown
Contributor Author

@mxsm @Oliverwqcwrw Help to integrate the code.

@zhiliatom

zhiliatom commented Jun 4, 2024

Copy link
Copy Markdown
Contributor

lgtm

@frinda

frinda commented Jan 6, 2025

Copy link
Copy Markdown
Contributor Author

seek for integration

@frinda frinda changed the title [ISSUE #7263& ISSUE #7264] Fix to cleanup consumption offset data or topic consumerqueue on the standby node when execute command 'deleteSubGroup' or 'deleteTopic' [ISSUE #7263& ISSUE #7264] consumption offset data and topic consumerqueue not deleted on the standby node Jan 7, 2025
@frinda

frinda commented Jan 7, 2025

Copy link
Copy Markdown
Contributor Author

@ joeCarf @ iamgd67 review again

@iamgd67

iamgd67 commented Jan 7, 2025

Copy link
Copy Markdown
Contributor

lgtm, but I am not a commiter, don't have perge permission.

@frinda

frinda commented Jan 8, 2026

Copy link
Copy Markdown
Contributor Author

seek for integration

RockteMQ-AI
RockteMQ-AI reviewed Sep 14, 2026
View reviewed changes

@RockteMQ-AI RockteMQ-AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

This PR fixes two issues in SlaveSynchronize for slave broker synchronization:

  1. Missing cleanUnusedTopic() call — When topic config changes on the slave, stale topics should be cleaned. The fix adds cleanUnusedTopic() before syncing the new config.
  2. Consumer offset table not cleared before sync — Without clearing, old offsets from a previous master could persist alongside new ones, causing incorrect offset data. The fix clears the table before putAll().

The logic is sound. However, this PR targets develop from develop (same branch), which is unusual. The head branch should ideally be a feature branch. Also, this PR is 3+ years old — the codebase may have evolved. A rebase and fresh test run would be advisable before merging.

Automated review by github-manager-bot

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

3 more reviewers
@RockteMQ-AI RockteMQ-AI RockteMQ-AI left review comments
@iamgd67 iamgd67 iamgd67 approved these changes
@joeCarf joeCarf joeCarf approved these changes
Reviewers whose approvals may not affect merge requirements

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Bug] The consumption offset data of the subscription group cannot be deleted synchronously on the standby node.

6 participants

@frinda @codecov-commenter @iamgd67 @zhiliatom @joeCarf @RockteMQ-AI