Skip to content

fix(iptv): download the Stalker channel list once per configuration change - #677

Merged
ProdigyV21 merged 2 commits into
ProdigyV21:mainfrom
ReichiMD:claude/nifty-fermat-bgr9md
Sep 11, 2026
Merged

fix(iptv): download the Stalker channel list once per configuration change#677
ProdigyV21 merged 2 commits into
ProdigyV21:mainfrom
ReichiMD:claude/nifty-fermat-bgr9md

Conversation

@ReichiMD

@ReichiMD ReichiMD commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

The cause

The Stalker channel list is a single response — on the portal this was measured
against, ~29 MB for 21,295 channels. Three entry points ask for it: the startup
prefetch, the live TV snapshot load and the guide backfill load. Each opened its
own portal session and pulled the whole list, so one app start cost 3 × 29 MB
and three handshakes in 17 seconds
.

Changing a source made it worse rather than better. Two entry points react to a
single configuration change, and each discarded the download the other had just
started — so one playlist toggle cost two complete lists.

What this changes

A new StalkerChannelListLoader (~140 lines, memory only) lets all three entry
points share one download:

  • concurrent callers await the download that is already in flight;
  • a caller arriving within five minutes of a usable download reuses its result;
  • the download runs in its own scope, so the startup prefetch giving up after
    its 25 s timeout does not cancel the download the live TV screen is waiting on;
  • an empty result — a failed handshake — is never remembered, so a portal cannot
    stay dark for the whole window.

Exactly two things drop the shared list, and nothing else:

The loader key, which is the configured portal set (id, URL, MAC — hashed, so
those values never leave the function). invalidateCache() deliberately no
longer drops the download: it runs on every source change, and a playlist toggled
or an EPG URL edited leaves the Stalker portals alone. Dropping it there is what
tore up the download the other entry point was already running.

A forced reload, which now means "nothing older than my request" instead of
"nothing remembered". loadSnapshot reads that timestamp before it takes the
load lock, so the second entry point reacting to the same configuration change
accepts the download that finished while it was queued — while
Settings → Refresh IPTV still reaches the portal.

The list is deliberately not persisted across app runs: its playable URLs
carry a play_token, and a list restored from disk would hand out expired ones.

M3U and Xtream sources never enter this path — loadStalkerChannels returns
immediately when no Stalker portal is enabled, and nothing else in
invalidateCache() changed.

Result, measured on a real portal

Network capture, 80 seconds, one app start plus three source toggles, portal with
21,295 channels:

before after
app start 3 × 29 MB, 3 handshakes 1 × 29 MB, 1 handshake
per source toggle 2 × 29 MB 1 × 29 MB

Live playback in the same capture stayed clean: no create_link calls, every
portal response 200, the stream started over the usual 302.

Known limit, deliberately not part of this PR

A source change that has nothing to do with the Stalker portals — toggling an
M3U playlist, say — still costs one full channel list. This PR removes the
duplicate, not the reload itself: a configuration change asks for fresh data and
the loader honours that. Whether a Stalker list should be refreshed at all in
that case is a separate call, because the play_token inside it ages, and other
portal clients reload on every start for exactly that reason. Happy to send a
follow-up if you want it.

Testing

  • Full unit-test suite green (testSideloadDebugUnitTest), 16 tests new,
    split across two levels on purpose:
    • StalkerChannelListLoaderTest (13) covers the loader itself — sharing, the
      freshness window, a caller that gives up, a changed portal set, an empty
      result, a failed download.
    • IptvRepositoryStalkerListInvalidationTest (3) covers its caller,
      because the first attempt at this fix passed every loader test and still
      pulled the list twice on the device: the duplicate came from
      invalidateCache() one level up.
  • Verified on device against a live portal, with the network capture above.
  • No string resources touched.

created by Claude (Anthropic) on behalf of @ReichiMD

…hange

Three entry points ask for the Stalker channel list — the startup prefetch,
the live TV snapshot load and the guide backfill load. Each opened its own
portal session and pulled the full list; on a portal with ~21k channels that
is 27.67 MB per entry point, measured at three downloads in 17 seconds.

They now share one download through StalkerChannelListLoader: concurrent
callers await the download that is already running, and a caller arriving
within five minutes of a usable download reuses its result. Nothing is kept
on disk — the playable URLs carry a play_token, so a list restored from disk
would hand out expired ones.

Two things decide when the shared list is dropped, and only these two:

- The loader key, which is the configured portal set (id + URL + MAC).
  invalidateCache() deliberately no longer discards the download: it runs for
  every source change, and a playlist toggled or an EPG URL edited leaves the
  portals alone. Dropping the list there tore up a download another entry
  point was already running, so a single playlist toggle still cost two full
  27.67 MB downloads.
- A forced reload, which now means "not a list from before I asked" rather
  than "not the stored list". The repository takes that timestamp before it
  takes its load lock, so the second entry point reacting to one configuration
  change accepts the download that finished while it waited, while an explicit
  "Refresh IPTV" still reaches the portal.

The download runs in its own scope: the startup prefetch gives up after 25
seconds, and its timeout must not cancel the download the live TV screen is
waiting on.

Tests cover the loader and, this time, its caller: the loader bug came back
through invalidateCache() while the loader's own tests stayed green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MbtJc9Q9QK4SJAFveyp5m3
@github-actions github-actions Bot added the area: android Changes to the Android app or Gradle build label Sep 9, 2026
@ProdigyV21

Copy link
Copy Markdown
Owner

Thanks for this, reducing repeated downloads of such a large channel list is a useful improvement. The sharing approach looks good, but two caching issues still need fixing before merging:

  1. Partial portal failures get cached as success. isReusable only checks whether the combined channel list is non-empty. If portal A succeeds but B temporarily fails, subsequent ordinary loads reuse A's list without retrying B for up to five minutes, even after B recovers. I reproduced this with the repository's loading helpers. Please cache successful portals separately, or avoid remembering the combined result when any portal failed.

  2. Removing the last portal never reaches the cleanup. loadStalkerChannels(emptyList()) would clear the shared cache, but every caller skips this function when there are no enabled portals. The removed portal's channel list and session can therefore remain in memory until the app closes. Please handle this in a configuration-change/removal path that actually runs, while preserving sharing for unrelated playlist changes.

The 13 loader tests passed locally, but an additional multi-portal recovery test exposed the first issue. Please add regression coverage for both partial failures and removing/disabling the last portal through the repository entry points. This is worth adding once those are covered.

Review follow-up on ProdigyV21#677, both points from the same comment.

Partial portal failures were cached as success. The freshness check asked
whether the merged channel list was non-empty, and with portals A and B
configured it was — A had filled it — so a load in which B failed counted as
reusable and B was not asked again for up to five minutes, including after it
recovered.

The loader is now keyed per portal (id + URL + MAC) instead of per portal set,
so every portal is downloaded, remembered and retried on its own and isReusable
asks whether *this* portal answered: a session was opened and channels came
back. Merging happens afterwards, in loadStalkerChannels, so a single portal's
failure can no longer disappear into it.

Per-portal caching rather than "remember nothing when any portal failed"
because of what the second option costs the setup this is about: with one of
two portals permanently down, dropping the combined result puts every entry
point back to re-downloading the healthy portal's full list, which is the
3 x 29 MB per start this change exists to remove. Keyed per portal, the fix
holds for the portals that work and a portal that failed still gets an
immediate retry.

The cleanup for "the last portal was removed" was unreachable.
loadStalkerChannels(emptyList()) released the shared state correctly, but every
caller returns before reaching it once no portal is enabled, so a removed
portal's channel list and its session stayed in memory until the app closed.
That branch is gone; the cleanup now runs in ensureCacheOwnership, which the
snapshot load, the cache-only warmup and the cached-snapshot read all pass
through before they can decide they have nothing to do. It releases only the
portals that disappeared from the configuration, so an unrelated playlist
change keeps the download it already has — dropping that is what used to cost a
second full 27.67 MB list on every toggle. The same step prunes cachedStalkerApis.

Regression coverage runs through the repository, not the loader alone: the
loader's own tests were green while both of these were broken one level up.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FDmLPyPaqS23awHuri7sX4
@ReichiMD

Copy link
Copy Markdown
Contributor Author

Thanks — both were real, and both are fixed on the same branch.

1. Partial portal failures

I went with the first option you offered: the loader now caches per portal instead of per
portal set. Its key is one portal (id + URL + MAC), so each portal is downloaded, remembered
and retried on its own, and isReusable now asks "did this portal answer?" (session opened
and channels returned) instead of "is the merged list non-empty?". Merging happens after that
decision, in loadStalkerChannels, so a failure can no longer disappear into it.

I preferred this over "remember nothing when any portal failed" because of what that costs the
exact setup you were testing: if one of two portals stays down, dropping the combined result
means every entry point re-downloads the healthy portal's full list again — on a 21k-channel
portal that is the 3 x 29 MB per start this PR exists to remove. Per-portal caching keeps that
fix intact for the portals that work while giving a failed portal an immediate retry. It also
means adding a portal no longer re-downloads the ones that were already there.

2. Cleanup that is never reached

You were right that loadStalkerChannels(emptyList()) is dead — I removed that branch. The
cleanup now lives in ensureCacheOwnership, which the snapshot load, the cache-only warmup and
the cached-snapshot read all pass through before they can decide they have nothing to do, so it
also runs when the last portal is gone. It calls retainOnly(currentPortalKeys): only portals
that disappeared from the configuration are released, together with the session that came with
the list, and portals that are still configured keep their in-flight or remembered download.
That is what preserves sharing for unrelated playlist changes — dropping it there is what used
to cost a second full 27.67 MB list on every toggle. The same step also prunes
cachedStalkerApis of portals that are gone.

Regression coverage, through the repository entry points

New file IptvRepositoryStalkerPortalStateTest, driving a real IptvRepository with mockk
standing in for the network only:

  • a portal that failed is asked again by the next ordinary load — portals A and B, B down:
    the first load returns A's channels only, and the next ordinary load asks B again immediately
    while A is not downloaded a second time. This is your multi-portal recovery case.
  • removing the last portal releases its channel list and session — an unrelated playlist
    change through ensureCacheOwnership keeps the download; removing the last portal releases
    it, so re-adding the portal downloads again.
  • disabling one of two portals leaves the other one's list alone — the boundary between the two.

Counter-check, run on a throwaway state: with the per-portal isReusable reverted to accepting
any answer, the first test fails; with the retainOnly call removed from ensureCacheOwnership,
the other two fail. Each test goes red exactly when its own fix is missing.

Confirmed on device as well

Both of your cases were also reproduced by hand on a real setup with two portals:

  • Second portal given a wrong MAC so it fails, then corrected: only the working portal's
    channels appear while it is broken, and after the correction the next ordinary load brings
    both back without pulling the working portal's list again.
  • Last portal disabled: the Stalker channels go, and re-enabling it loads cleanly.

A packet capture of a two-portal cold start on the same build shows one get_all_channels per
portal and one handshake per portal, no repeats (29.0 MB for a 21k-channel portal, 105 KB for a
2.2k one, no create_link). The two cases above were judged from app behaviour, not from a
capture of their own — the numeric evidence for them is the regression tests.

StalkerChannelListLoaderTest gained aPortalThatFailedIsRetriedWhileTheOthersKeepTheirLists,
and its key-related tests were rewritten for the per-portal keys.

Full suite 952 passing, up from 948. detekt unchanged at 3276 findings, rule for rule. No
strings.xml touched, and the change is still confined to the Stalker channel-list path.


created by Claude (Anthropic) on behalf of @ReichiMD

@ProdigyV21
ProdigyV21 merged commit af33d7f into ProdigyV21:main Sep 11, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: android Changes to the Android app or Gradle build

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants