Skip to content

Fix 2FA shutdown timer terminating a healthy logged-in session - #376

Open
chapinmark wants to merge 1 commit into
IbcAlpha:masterfrom
chapinmark:fix/2fa-orphan-shutdown-timer
Open

Fix 2FA shutdown timer terminating a healthy logged-in session#376
chapinmark wants to merge 1 commit into
IbcAlpha:masterfrom
chapinmark:fix/2fa-orphan-shutdown-timer

Conversation

@chapinmark

Copy link
Copy Markdown

Two defects in LoginManager can combine to terminate a Gateway session that has logged in successfully and is serving the API.

1. restartAfterTime() orphans the previous shutdown task

It assigns shutdownAfterTimeTask without cancelling the ScheduledFuture already held there. setLoginState(LOGGED_IN) cancels only whatever the field currently references, so any earlier task is orphaned — still scheduled, but no longer reachable by the handler meant to cancel it.

Every start arms two of these. The SecondFactorDevice selection dialog closes first (Duration since login: 1 seconds) and arms one; the real 2FA dialog closes a few seconds later and arms a second, overwriting the field. One orphan therefore survives every session, and fires SecondFactorAuthenticationExitInterval seconds after login began regardless of what happened since.

2. The re-login queued after a 2FA timeout has no LOGGED_IN guard

It runs unconditionally five seconds later. TWS/Gateway sometimes closes the 2FA dialog right on the timeout boundary and then completes the login anyway, so the session can already be LOGGED_IN when the task fires. Re-initiating the login takes it back out of LOGGED_IN, and it never returns.

Observed failure

Gateway 1045, IBC 3.23.0, SecondFactorAuthenticationExitInterval=3700, SecondFactorAuthenticationTimeout=180, ReloginAfterSecondFactorAuthenticationTimeout=yes:

15:38:37.697  device dialog closed, "IBC will exit in 3700 seconds"  (deadline 16:40:17)
16:01:24      2FA approved; 16:01:27 "Login has completed", API port 4002 open
16:07:15      IB raises "Re-login is required"; IBC clicks Re-login, new 2FA prompt
16:10:16.759  dialog closed at exactly 180s -> "Re-login ... in 5 second"
16:10:16.868  session reaches LOGGED_IN on its own, 109ms later
16:10:21      the queued re-login fires anyway (Login attempt: 4); state leaves LOGGED_IN
16:40:17.699  orphan timer fires, state != LOGGED_IN, exit 1111

The exit is 3700.002s after the orphan was armed. The Gateway process was alive and serving throughout — its own log was still being written at 16:40:17, and the socat helper forwarding 4002 logged no connection failures after 16:01:27. The container then exited 87 (1111 % 256) and its orchestrator restarted it.

The change

  • Cancel any outstanding shutdownAfterTimeTask before scheduling a replacement, so only one is ever pending and LOGGED_IN can always cancel it.
  • Guard the deferred re-login on LOGGED_IN, mirroring the guard restartAfterTime already applies inside its own scheduled task.

Either change alone prevents the failure above; both are included because each is a defect independently. No settings change, no new configuration, and no effect on installs where 2FA is approved promptly — in that case the first dialog's orphan simply finds LOGGED_IN and returns, as it does today.

Compiled clean against the full tree with ant dist (IBC_BIN pointing at Gateway 1045 jars) and verified in the resulting IBC.jar bytecode.

I appreciate this arrives close to retirement, and I'll understand entirely if you'd rather not take further changes — posting it mainly so the analysis is on record for anyone who hits the same thing.

Two independent defects in LoginManager can combine to terminate a
Gateway session that has successfully logged in and is serving the API.

1. restartAfterTime() assigns shutdownAfterTimeTask without cancelling
   the ScheduledFuture already held there. setLoginState(LOGGED_IN)
   cancels only whatever the field currently references, so any earlier
   task is orphaned: still scheduled, but no longer reachable by the
   handler meant to cancel it.

   Every start arms two of these. The SecondFactorDevice selection
   dialog closes first ("Duration since login: 1 seconds") and arms one;
   the real 2FA dialog closes a few seconds later and arms a second,
   overwriting the field. One orphan therefore survives every session.

2. The re-login scheduled after a 2FA timeout runs unconditionally five
   seconds later. TWS/Gateway sometimes closes the 2FA dialog right on
   the timeout boundary and then completes the login anyway, so the
   session can already be LOGGED_IN when that task fires. Re-initiating
   the login takes it back out of LOGGED_IN, and it never returns.

Together they end a working session. Observed with Gateway 1045 and
IBC 3.23.0, SecondFactorAuthenticationExitInterval=3700:

  15:38:37.697  device dialog closed, orphan timer armed (deadline 16:40:17)
  16:01:24      2FA approved; 16:01:27 LOGGED_IN, API port 4002 open
  16:07:15      IB forces "Re-login is required"; new 2FA prompt
  16:10:16.759  dialog closed at exactly 180s -> re-login scheduled in 5s
  16:10:16.868  session reaches LOGGED_IN on its own, 109ms later
  16:10:21      the queued re-login fires anyway, state leaves LOGGED_IN
  16:40:17.699  orphan timer fires, state != LOGGED_IN, exit 1111

The Gateway process was alive and serving the API throughout: its log
was still being written at 16:40:17, and the socat helper logged no
connection failures after 16:01:27.

Fix both: cancel any outstanding shutdown task before scheduling a new
one, and guard the deferred re-login on LOGGED_IN, mirroring the guard
restartAfterTime already applies in its own scheduled task. Either
change alone prevents the failure above; both are included because each
is a defect in its own right.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant