Periodically re-check local file size while playing

[mertemr]

Fixes #797

File size was only read once, when the file started playing (updateFile()). If the file kept growing on disk after that, the size sent to other users stayed stuck at the old value for the rest of the session.

This adds a periodic re-check (every 5s, same pattern as the existing askPlayer loop) that re-stats the current file and resends it only if the size changed. Respects the existing filesize/filename privacy settings.

[Et0h]

Thanks for this. I agree with periodically re-checking the size of the currently playing local file because this prevents files that grow while playing (such as an in-progress download) from being stuck at the initial size and causing incorrect warnings to persist. The main thing I would change is when the updated size is sent.

In your PR recheckFileSize() calls sendFile() whenever the observed size differs from file_['size']. sendFile() is Syncplay's normal file update path, rather than a lightweight metadata only update. A changed filesize can make the files compare as different, so when the update is received it can go through the normal "playing" / file difference notification handling. For a file which is actively downloading, the byte count could change continuously, so broadcasting every observed change could cause repeated Syncplay file change processing even though the user has not actually changed what they are watching.

So when I tested your code with a file being downloaded it ended up outputting this:

[21:18:35] Alice is playing 'BigBuckBunny.mkv' in room: 'Test'
[21:18:35] Your file differs in the following way(s): size
[21:18:36] Alice is playing 'BigBuckBunny.mkv in room: 'Test'
[21:18:40] Bob is playing 'BigBuckBunny.mkv'
[21:18:40] Your file differs in the following way(s): size
[21:18:40] Bob is playing 'BigBuckBunny.mkv'
[21:18:40] Your file differs in the following way(s): size
[21:18:40] Alice is playing 'BigBuckBunny.mkv' in room: 'Test'
[21:18:45] Bob is playing 'BigBuckBunny.mkv'
[21:18:45] Your file differs in the following way(s): size
[21:18:45] Alice is playing 'BigBuckBunny.mkv' in room: 'Test'
[21:18:49] Bob is playing 'BigBuckBunny.mkv'
[21:18:49] Your file differs in the following way(s): size
[21:18:50] Alice is playing 'BigBuckBunny.mkv' in room: 'Test'
[21:18:55] Bob is playing 'BigBuckBunny.mkv'
[21:18:55] Your file differs in the following way(s): size
[21:18:55] Alice is playing 'BigBuckBunny.mkv' in room: 'Test'
[21:19:00] Bob is playing 'BigBuckBunny.mkv' 
[21:19:00] Your file differs in the following way(s): size
[21:19:00] Alice is playing 'BigBuckBunny.mkv' in room: 'Test'
[21:19:05] Bob is playing 'BigBuckBunny.mkv'
[21:19:05] Your file differs in the following way(s): size
[21:19:05] Alice is playing 'BigBuckBunny.mkv' in room: 'Test'

I think the periodic check should mostly observe the file, and only advertise a new size when there is a good reason to send it:

• If the freshly observed filesize exactly matches another user in the same room who Syncplay identifies as playing the same file, send it immediately.
• Otherwise, only send it once that filesize has remained unchanged for 60 seconds.

The 60 second rule is only a stability fallback. It does not mean the download is permanently complete. If a download stalls for 60 seconds, gets reported, and later resumes growing, Syncplay should continue monitoring it and can report a later matching or stable size.

I would also make the filesystem check asynchronous. os.path.getsize() is normally cheap, but it can block when using network storage. client.py already imports and uses Twisted's threads, so deferToThread() is a good fit here. Returning the Deferred from the LoopingCall also means another filesize check is not started while the previous one is still outstanding.

Proposed implementation

syncplay/constants.py

Add alongside the other timing values:

FILESIZE_RECHECK_DELAY = 10  # Secs - How often to re-check the size of the currently playing local file
FILESIZE_STABLE_THRESHOLD = 60  # Secs - How long a changed filesize must remain unchanged before sending

I think 10 seconds is frequent enough for this metadata refresh without pointlessly polling the filesystem every few seconds, and the 60 second threshold avoids treating brief download stalls as meaningful completion.

SyncplayClient.__init__

Alongside _askPlayerTimer:

self._askPlayerTimer = None
self._filesizeRecheckTimer = None
self._filesizeRecheckPath = None
self._lastObservedFilesize = None
self._filesizeStableSince = None

SyncplayClient.initPlayer()

Start the filesize checker with the existing player polling:

def initPlayer(self, player):
    self._player = player
    if not self._player.alertOSDSupported:
        constants.OSD_WARNING_MESSAGE_DURATION = constants.NO_ALERT_OSD_WARNING_DURATION
    self.scheduleAskPlayer()
    self.scheduleFilesizeRecheck()
    self.__playerReady.callback(player)

Filesize recheck methods

Add near scheduleAskPlayer():

def scheduleFilesizeRecheck(self, when=constants.FILESIZE_RECHECK_DELAY):
    self._filesizeRecheckTimer = task.LoopingCall(self.recheckFilesize)
    self._filesizeRecheckTimer.start(when, now=False)

def _resetFilesizeRecheckState(self, path=None):
    self._filesizeRecheckPath = path
    self._lastObservedFilesize = None
    self._filesizeStableSince = None

def _getFilesize(self, path):
    try:
        return os.path.getsize(path)
    except OSError:
        return None

def recheckFilesize(self):
    if not self._running:
        return

    file_ = self.userlist.currentUser.file
    if not file_ or not file_.get('path'):
        self._resetFilesizeRecheckState()
        return

    path = file_['path']

    if utils.isURL(path):
        self._resetFilesizeRecheckState()
        return

    if self._config['filesizePrivacyMode'] == PRIVACY_DONTSEND_MODE:
        self._resetFilesizeRecheckState()
        return

    if path != self._filesizeRecheckPath:
        self._resetFilesizeRecheckState(path)

    return threads.deferToThread(
        self._getFilesize, path
    ).addCallback(
        self._processFilesizeRecheck, path
    )

def _processFilesizeRecheck(self, size, path):
    if not self._running:
        return

    file_ = self.userlist.currentUser.file

    # The loaded file may have changed while getsize() was running.
    if not file_ or file_.get('path') != path:
        self._resetFilesizeRecheckState()
        return

    # Do not apply a result if privacy settings changed while the stat
    # operation was in progress.
    if self._config['filesizePrivacyMode'] == PRIVACY_DONTSEND_MODE:
        self._resetFilesizeRecheckState(path)
        return

    if size is None:
        self._resetFilesizeRecheckState(path)
        return

    size = self.__executeFilesizePrivacySettings(size)

    # Use direct equality rather than sameFilesize(). sameFilesize()
    # deliberately treats 0 as compatible with any size, whereas here
    # a previously unknown size should be replaced when it becomes known.
    if size == file_['size']:
        self._resetFilesizeRecheckState(path)
        return

    if self.userlist.currentFilesizeMatchesUserInRoom(size):
        file_['size'] = size
        self._resetFilesizeRecheckState(path)
        self.sendFile()
        return

    now = time.monotonic()

    if size != self._lastObservedFilesize:
        self._lastObservedFilesize = size
        self._filesizeStableSince = now
        return

    if (
        self._filesizeStableSince is not None
        and now - self._filesizeStableSince >= constants.FILESIZE_STABLE_THRESHOLD
    ):
        file_['size'] = size
        self._resetFilesizeRecheckState(path)
        self.sendFile()

A failed getsize() here means that os.path.getsize(path) raised OSError, for example because the file disappeared, access was denied, a share became unavailable, or the filesystem reported an I/O error. That observation should not count towards the stability period, so the stability state is reset. A filesystem which is merely slow is different: the operation remains in the worker thread until it succeeds or fails, and the returned Deferred prevents overlapping LoopingCall iterations.

The stability state is keyed to the path,. The path is the thing being statted and is not affected by filename privacy. Changing from /videos/Episode1.mkv to /videos/Episode2.mkv therefore resets the state.

Peer filesize match

Add this to SyncplayUserlist, near the existing file comparison methods:

def currentFilesizeMatchesUserInRoom(self, size):
    file_ = self.currentUser.file

    if (
        not file_
        or size == 0
        or file_['name'] == PRIVACY_HIDDENFILENAME
    ):
        return False

    candidateFile = file_.copy()
    candidateFile['size'] = size

    for otherUser in self._users.values():
        if (
            otherUser.room == self.currentUser.room
            and otherUser.file
            and otherUser.file['size'] != 0
            and otherUser.file['name'] != PRIVACY_HIDDENFILENAME
            and otherUser.isFileSame(candidateFile)
        ):
            return True

    return False

This uses SyncplayUser.isFileSame() for Syncplay's existing filename, filesize and duration comparison. Before doing that comparison, it excludes values which represent unavailable information rather than evidence of a match.

In normal Syncplay file comparison, a hidden filename or filesize 0 is treated as compatible so that privacy settings do not create false file difference warnings. For this immediate filesize update check, those values should not trigger a send because they do not show that another user has the same filename and freshly observed filesize. Normal and hashed filenames and filesizes can still use the existing comparison.

Any one matching user in the same room is enough. We should not require every other user to agree because another user may simply still be downloading or have stale filesize information. We should also not pick the numerically largest filesize as authoritative, since hashed sizes cannot be ordered meaningfully and the largest observed raw size is not necessarily complete.

This peer match is a fast path for convergence. It is not proof that a download is finished. If another user happens to advertise an intermediate size which the local download later reaches, that size may be sent, but monitoring continues and a later size can still be advertised after another peer match or the stability fallback.

Privacy handling

Keep the refactoring from this PR so that the initial filesize and subsequent rechecks use the same filesize privacy transformation:

def __executePrivacySettings(self, filename, size):
    if self._config['filenamePrivacyMode'] == PRIVACY_SENDHASHED_MODE:
        filename = utils.hashFilename(filename)
    elif self._config['filenamePrivacyMode'] == PRIVACY_DONTSEND_MODE:
        filename = PRIVACY_HIDDENFILENAME

    size = self.__executeFilesizePrivacySettings(size)
    return filename, size

def __executeFilesizePrivacySettings(self, size):
    if self._config['filesizePrivacyMode'] == PRIVACY_SENDHASHED_MODE:
        size = utils.hashFilesize(size)
    elif self._config['filesizePrivacyMode'] == PRIVACY_DONTSEND_MODE:
        size = 0
    return size

The rechecker exits before doing a filesystem stat when filesizePrivacyMode == PRIVACY_DONTSEND_MODE.

Resulting behaviour

Situation	Behaviour
File is continuously growing	Observe only; do not repeatedly call sendFile()
Fresh size exactly matches a valid same file peer	Send immediately
Fresh size matches nobody	Send after it has remained unchanged for 60 seconds
All other users advertise the same matching size	Send immediately; the first valid match is sufficient
Some users match and others are still downloading	Send immediately; unanimity is not required
Download stalls for 60 seconds, then resumes	The stalled size may be sent; later growth is still monitored and can be updated again
Loaded path changes	Reset the old stability observation
getsize() raises OSError	Reset the stability observation and try again on a later loop
getsize() is slow	Run it off the reactor; no overlapping rechecks
Filesize privacy is DoNotSend	Do not periodically stat the file
Peer hides filesize (0)	Do not use that peer for the immediate match
Peer hides filename	Do not use that peer for the immediate match
Raw or hashed privacy modes	Continue to work through existing hashing and comparison helpers

[mertemr]

@Et0h you're right, I tested this against a growing file and saw the same spam.
I'll rework recheckFileSize() to only observe by default, and send either on an exact match with another user in the room or after 60s of stability using deferToThread for the stat call as you suggested.
Will push an update once it's tested. 🙏

[Et0h]

Thanks for the new commit @mertemr - can you please summarise what tests you've done and on what operating systems.

[mertemr]

@Et0h
I tested on Windows 11 with mpv, two clients, one on the local network and one remote.
I used two video files, one still downloading while playing and one already complete. The repeated "file differs in size" spam is gone. The size is only observed while the file grows and gets sent on a peer match or after 60s of no change. Switching files mid-session resets the observation.
For Linux I can't comment on those. I didin't tested yet