[winterhazel]

Description

During the execution of scheduled backups, if an asynchronous job expires before the backup finishes (configuration job.expire.minutes), the job's entry is removed from the database. However, the thread responsible for scheduling the backups continues attempting to retrieve information about the removed entry. Because no job is returned, accessing its status throws a NullPointerException inside this thread, and all backup schedules stop working.

The code has been fixed so that a missing/expired async job is treated as finished, so expired jobs no longer prevent backups from being scheduled.

Types of changes

• Breaking change (fix or feature that would cause existing functionality to change)
• New feature (non-breaking change which adds functionality)
• Bug fix (non-breaking change which fixes an issue)
• Enhancement (improves an existing feature and functionality)
• Cleanup (Code refactoring and cleanup, that may add test cases)
• Build/CI
• Test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Bug Severity

• BLOCKER
• Critical
• Major
• Minor
• Trivial

Screenshots (if appropriate):

How Has This Been Tested?

1. I configured job.expire.minutes to 1 minute, and kboss.timeout to 2 minutes.
2. I scheduled a backup.
3. After the backup started, I hung the process for in the agent using a debugger, so that the async job would expire.
4. Before the changes, an error was thrown in the thread that schedules the backups, and no backup was scheduled anymore.

2026-11-04 10:34:15,181 [acs-aio - 10347] WARN  [o.a.c.b.BackupManagerImpl:2552] (BackupPollTask:[ctx-9da2e362]) (logid:2186bb75) Catch throwable in backup scheduler java.lang.NullPointerException: Cannot invoke "org.apache.cloudstack.framework.jobs.impl.AsyncJobVO.getStatus()" because "asyncJob" is null
    at org.apache.cloudstack.backup.BackupManagerImpl.checkStatusOfCurrentlyExecutingBackups(BackupManagerImpl.java:2438)
    at org.apache.cloudstack.backup.BackupManagerImpl.poll(BackupManagerImpl.java:2393)
    at org.apache.cloudstack.backup.BackupManagerImpl$5.runInContext(BackupManagerImpl.java:2550)
    at org.apache.cloudstack.managed.context.ManagedContextTimerTask$1.runInContext(ManagedContextTimerTask.java:30)
    at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
    at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
    at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
    at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
    at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
    at org.apache.cloudstack.managed.context.ManagedContextTimerTask.run(ManagedContextTimerTask.java:32)
    at java.base/java.util.TimerThread.mainLoop(Timer.java:566)
    at java.base/java.util.TimerThread.run(Timer.java:516)

5. After the changes, the schedule that had an expired job was rescheduled, and the scheduler thread continued to work as expected.

2026-11-04 10:44:12,347 [acs-aio - 13033] DEBUG [o.a.c.b.BackupManagerImpl:2439] (BackupPollTask:[ctx-fc0462ec]) (logid:58761e37) The async job [97] of backup schedule [BackupSchedule {"id":5,"schedule":"43","scheduleType":0,"vmId":8}] does not exist anymore. Considering it as finished and scheduling the next backup job.

[winterhazel]

@blueorangutan package

[blueorangutan]

@winterhazel a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[codecov]

Codecov Report

❌ Patch coverage is 66.66667% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 18.32%. Comparing base (d02adcd) to head (1657ba2).

Files with missing lines	Patch %	Lines
...rg/apache/cloudstack/backup/BackupManagerImpl.java	66.66%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff            @@
##               4.22   #14587   +/-   ##
=========================================
  Coverage     18.32%   18.32%           
- Complexity    16644    16646    +2     
=========================================
  Files          5939     5939           
  Lines        537768   537772    +4     
  Branches      65929    65930    +1     
=========================================
+ Hits          98551    98561   +10     
+ Misses       428008   428001    -7     
- Partials      11209    11210    +1     

Flag	Coverage Δ
uitests	4.08% <ø> (ø)
unittests	19.41% <66.66%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[blueorangutan]

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 19908