Skip to content

fix: stop scans from downloading iCloud and other cloud placeholders - #74

Merged
ohing504 merged 3 commits into
mainfrom
fix/skip-dataless
Oct 29, 2026
Merged

ohing504 merged 3 commits into
mainfrom
fix/skip-dataless

Conversation

@ohing504

@ohing504 ohing504 commented Oct 29, 2026 •

Copy link
Copy Markdown
Owner

What changed

  • The shared walk (fstree) no longer reads a directory flagged SF_DATALESS (macOS File Provider placeholder: iCloud Drive, Dropbox, …). It returns that directory as skipped, like a directory on another filesystem, so clean refuses an artifact that contains one.
  • fileExtents no longer opens a file that has no allocated blocks. A dataless file has 0 blocks. The sizing walk already had this guard; the clone-source walk did not.
  • Docs:
    • SPEC gets one Traversal section. It states every skip rule (symlinks, mounts, cloud placeholders) and the clean refusal. SPEC previously had none of these.
    • ARCHITECTURE drops a sizing bullet that repeated § Shared traversal. That also makes "Three invariants" list three again.
    • The Homebrew example in § Deletion Strategy now sits next to the delete methods it illustrates, and its refusal sentence links both the mount and the cloud-placeholder decisions.
    • .claude/rules/cleanup.md lists skip-dataless.md with the other traversal decisions.

Why

Reading a dataless directory makes macOS download its content and blocks the read until the download ends.

  • A clean --dry-run sat 8 minutes in one directory read under ~/Library/Mobile Documents.
  • Scans downloaded evicted iCloud items, using the disk space devclean is meant to free.

The flag comes from the lstat result the walk already has, so the check adds no system call. Behaviour on Linux is unchanged.

Verification

  • New tests:
    • A dataless child directory is not entered and is reported as skipped.
    • A dataless root is not entered.
    • Both fail when the check is removed.
  • Real machine:
    • brctl evict was run on an iCloud .epub directory. It stayed dataless through a scan of its folder and a full ~ scan. Before this change, one scan downloaded it.
    • A file evicted with brctl evict reports 0 blocks.
  • Rejected after measuring: setiopolicy_np(IOPOL_MATERIALIZE_DATALESS_FILES_OFF) for the whole process. With it set, a directory read still downloaded for 17 s, and a file open still downloaded the file.
  • go test -race -count=1 ./..., golangci-lint run ./... and GOOS=linux go build ./... all pass.

Not covered

These paths can still download a placeholder file that sits inside a directory that is already downloaded:

  • git status run by gitignore protection, for projects inside iCloud-synced Desktop or Documents.
  • The cross-volume Trash copy.
  • node_modules/.modules.yaml reads.
  • Scanners that read fixed tool directories with os.ReadDir (LLM model stores, Hugging Face cache, Xcode, code-sign clones) do not go through fstree.

Merge order

#72 and #73 still call fileExtents(path, size). Whichever lands second must be rebased onto fileExtents(path, info), keeping the 0-block guard.

Docs

  • docs/SPEC.md: new § Traversal.
  • docs/ARCHITECTURE.md: § Shared traversal, § Sizing invariants, § Deletion Strategy.
  • New decision: docs/decisions/skip-dataless.md.

🤖 Generated with Claude Code

ohing504 and others added 3 commits October 29, 2026 14:35
Reading a File Provider placeholder directory (SF_DATALESS) makes macOS
download its content and blocks until the download ends; a scan sat 8
minutes in one such read under ~/Library/Mobile Documents. The shared walk
now skips dataless directories like directories on another filesystem, and
sizing does not open dataless files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…traversal docs

- fileExtents skips files with no allocated blocks itself, covering the
  clone-source walk that had no guard; drop the call-site check and the
  exported fstree.Dataless.
- SPEC gets one Traversal section stating every skip rule and the clean
  refusal; ARCHITECTURE drops a duplicated sizing bullet and a misplaced
  example; the decision records the measured iopolicy alternative.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ohing504
ohing504 merged commit 6d1cf18 into main Oct 29, 2026
4 checks passed
@ohing504
ohing504 deleted the fix/skip-dataless branch October 29, 2026 13:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant