Skip to content

Media directory monitoring: notifications + adaptive reconciliation, no disable on timeout - #799

Draft
AlySerry0 wants to merge 4 commits into
Syncplay:masterfrom
AlySerry0:folder-search-retry-not-disable
Draft

Media directory monitoring: notifications + adaptive reconciliation, no disable on timeout#799
AlySerry0 wants to merge 4 commits into
Syncplay:masterfrom
AlySerry0:folder-search-retry-not-disable

Conversation

@AlySerry0

@AlySerry0 AlySerry0 commented Aug 17, 2026

Copy link
Copy Markdown

Draft implementing the media-directory monitoring spec from the #796 discussion (refs #130). This revision replaces the earlier draft on this branch and is rebased onto current master (so it builds on the merged watched-subfolders EpisodeFilenameParser).

Developed with AI assistance (Claude) and reviewed/tested by me, following the updated agent docs (#763).

Approach

Filesystem notifications are used as an acceleration path over authoritative reconciliation scanning; a full recursive scan stays the completeness authority, and absence of an event is never treated as proof nothing changed.

Kept from the earlier draft: transient timeouts no longer disable folder search, the synchronous single-flight scan claim, watchdog support, and the Windows packaging.

Reworked per the spec:

  • syncplay/filemonitor.py (new) — a small, playlist-blind FileMonitor owning the watchdog observer lifecycle and the thread→reactor handoff. It knows only paths/watch characteristics: a recursive watch per media root, plus a budgeted set of supplemental non-recursive watches for Windows network roots (FOLDER_SEARCH_NETWORK_WATCH_LIMIT = 32). If watchdog is unavailable or a watch fails, it provides no events and scanning continues.
  • Event-driven cache — clear file create/delete/move events update mediaFilesCache directly and idempotently on the reactor thread (no scan to re-confirm; component/case-aware path matching so P:\TVP:\TV2). Directory events use targeted subtree reconciliation, with full reconciliation as the ambiguity/failure fallback.
  • Concurrency — single-flight full/targeted worker; events arriving during a scan collapse into one pending full reconciliation; workers only read the filesystem and return results, the reactor thread commits (no shared lock, since qt5reactor makes the GUI and reactor one thread). Downstream fileSwitchFoundFiles() is coalesced (~1s).
  • Adaptive polling — full reconciliation runs at a slow safety interval (FOLDER_SEARCH_RECONCILIATION_INTERVAL = 300) while the current/next local files resolve, and returns to the existing ~30s folderSearchDoubleCheckInterval while a needed file is unresolved (using findFilepath resolution semantics; URLs don't force urgent polling).
  • Timeout/warning — degraded operational state (cleared only after three consecutive clean full scans) is separated from a single user-visible warning per media configuration/session. folderSearchWarningThreshold becomes additional grace on a 3s base delay, so no config migration. The folderSearchUseNotifications option added in the earlier draft is removed (notifications are an internal optimisation). The warning no longer says folder searching has been disabled.
  • Watch prioritisation — supplemental network-directory watches are ranked by current-file/current/next-playlist relevance and same-series/season directories via a small getRelatedEpisodeDirectories helper reusing EpisodeFilenameParser (no os.walk, ranking hint only).
  • Dependency — pinned watchdog>=2.1.0,<4.0.0 to stay compatible with the currently declared minimum Python (deferring any deliberate 3.9 bump to a maintainer decision), added to the py2exe bundle.

The §5 direct-watch gate (validated on the reported Win 11 + Samba setup)

A non-recursive WindowsApiObserver watch placed directly on an existing deep directory received all remote (Pi-originated) create/rename/delete events — whereas the recursive root watch missed them. So the supplemental per-directory watch strategy is worthwhile: it catches exactly the remotely-added media a recursive root watch does not surface at depth.

Testing

  • Event application (§11): idempotent create/delete/move, subtree removal, TV/TV2 boundary safety, case-insensitive key reuse, move in/out of roots — unit tested (13/13).
  • Construct-under-reactor integration: start → full scan → cache populate → direct event applied → coalesced notification → clean shutdown.
  • FileMonitor live against the Samba share: supplemental non-recursive watch delivers deep remote events on the reactor thread; budget accounting correct.
  • From-source startup and a frozen py2exe build that bundles filemonitor + watchdog (incl. the Windows read_directory_changes/winapi observer); real-session behaviour smoke (join/playlist/EOF/file-switch) showed no seek/pause/playlist/protocol change.
  • Still to run with help: the watch-budget benchmark (Test B: 1/8/16/32/64 network watches — thread count, CPU, memory, reconnect behaviour) and the broader matrix on macOS/Linux.

Happy to adjust any of the constants, naming, or the Python/watchdog version decision.


Changelog line: #799 by @AlySerry0, resolving #796 and #130

New syncplay/filemonitor.py: a small, playlist-blind helper that owns the
optional watchdog observer lifecycle and reports structured filesystem
events back on the reactor thread. It knows only about paths and native
watch characteristics: recursive watches for local/media roots, plus a
budgeted set of supplemental non-recursive watches for Windows network
roots (where a single recursive watch does not reliably report remotely-
originated changes at depth). If watchdog is unavailable or a watch
cannot be established it simply provides no events.

Adds the supporting folder-search constants (reconciliation interval,
warning base delay, event coalescing window, degraded-recovery count and
the network watch budget).
A small public helper around the existing EpisodeFilenameParser that, given
target playlist filenames, returns cached directories holding the same
series/season. It reads only mediaFilesCache (no os.walk), maps matches in
a Watched subfolder to their parent, and is a ranking hint only - a false
positive cannot affect file switching or playlist behaviour.
…disable

Rewrites FileSwitchManager to use filesystem notifications as an
acceleration path over authoritative reconciliation scanning, keeping the
recursive scan as the completeness authority:

- a transient first-file/scan timeout aborts the pass but no longer sets
  folderSearchEnabled = False or requires re-affirming media directories;
- clear file create/delete/move events update mediaFilesCache directly and
  idempotently on the reactor thread, without a scan to re-confirm them;
  directory events use targeted subtree reconciliation, with full
  reconciliation as the ambiguity/failure fallback;
- single-flight scan worker; events arriving during a scan collapse into
  one pending full reconciliation; downstream fileSwitchFoundFiles() is
  coalesced;
- adaptive polling: full reconciliation drops to a slow safety interval
  while the current/next local files resolve and returns to the ~30s
  cadence while a needed file is unresolved;
- operational degraded state (cleared only after three consecutive clean
  full scans) is separated from a single user-visible warning per media
  configuration/session; the warning threshold becomes additional grace on
  a small base delay, so no config migration is needed;
- supplemental Windows network directory watches are prioritised by
  current/next playlist relevance and same-series/season directories.

The reworked warning no longer says folder searching has been disabled.
watchdog powers the optional filesystem-notification path. Pinned
watchdog>=2.1.0,<4.0.0 to stay compatible with the currently declared
minimum Python, and added to the py2exe packages so the frozen Windows
build ships it. Notifications remain optional: if the import or a native
backend is unavailable, folder search falls back to reconciliation
scanning.
@AlySerry0
AlySerry0 force-pushed the folder-search-retry-not-disable branch from 7a9c584 to 8b43a0d Compare August 19, 2026 02:13
@AlySerry0 AlySerry0 changed the title Folder search: don't disable on transient timeout; add optional watchdog notifications Media directory monitoring: notifications + adaptive reconciliation, no disable on timeout Aug 19, 2026
@Et0h

Et0h commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Cheers, I'll review this when I have the time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants