Skip to content

Fix a 5.8.2 regression and the SafeDisk gaps found by testing downstream libraries - #42

Merged
avgvi merged 9 commits into
masterfrom
fix/adoption-sweep-5.8.3
Nov 3, 2026
Merged

avgvi merged 9 commits into
masterfrom
fix/adoption-sweep-5.8.3

Conversation

@avgvi

@avgvi avgvi commented Nov 3, 2026

Copy link
Copy Markdown
Member

Fixes found by running diskcache-community 5.8.2 against 42 downstream libraries, for a 5.8.3 patch release. There is one commit per fix, and the Unreleased section of CHANGELOG.md has the user-facing wording.

Regression

  • Since PR Make a plain Cache.get read less and reuse a cursor per thread #38, the fast path of Cache.get took its cursor from _connect(), which always opens cache.db. Subclasses written for 5.6.3 that override _con to keep their rows in another file, as dwave-cloud-client's credentials store does, got OperationalError: no such table: Cache on every read. _cur now takes its cursor from self._con. The new test fails on master with that error.

SafeDisk

  • memoize(typed=True) keys: SafeDisk stores the classes on its allowlist, such as int and str, which every SafeDisk already read back. NoneType and bool stay refused, because the reader refuses them. defaultdict(bool), which was written but could not be read back, is now refused when written.
  • with_numpy() stores datetime64 and timedelta64 arrays. The reader already accepted them and only pickle.PickleBuffer refused them, so the writer now passes a flat byte view of the same memory. Rows for every other dtype are byte-for-byte unchanged.
  • Plain SafeDisk checks a value outside the standard-library types against the names its pickle looks up, as with_types disks already did. A dict subclass that pickles as a dict, such as DRF's ReturnDict, is stored and reads back as a dict.
  • Refusal messages: a refused function no longer points to with_types(), numpy objects point to with_numpy(), and an array of objects mentions pandas DataFrames.
  • with_types refuses scipy, torch, bitstruct and pyahocorasick classes, as it refuses numpy: crafted pickles of them crash the reader or run code.
  • The stub of with_types accepts a string or any iterable of strings for modules, as the code does.

Docs

  • The CVE page and the migration guide say up front that JSONDisk does not protect reads, and point to SafeJSONDisk.
  • The migration guide explains that Dash's DiskcacheManager takes only an upstream cache and keeps its signing secret in it, so its folder should be its own.
  • The tutorial no longer says that memoize(typed=True) needs the default Disk.

Behaviour changes to call out in the release

  • with_types with scipy, torch, bitstruct or pyahocorasick, also through DISKCACHE_SAFE_MODULES, now raises ValueError.
  • On with_types disks, an object that cannot be pickled raises TypeError instead of PicklingError.
  • The wording of refusal messages changed.
  • .pylintrc raises max-module-lines from 4200 to 4300, since core.py is now 4234 lines.

Not in this PR

The changelog lists these as known issues: a cache owned by another account opens read-only, the write check is slow on very large values, and signed entries are planned for 5.9.

Testing

  • pytest with the CI test dependencies (Django 5.2, numpy, pydantic): 840 passed, 2 skipped on Python 3.12 and 3.14, coverage 99.16% and 99.19%.
  • blue, isort, flake8, mypy (including tests/typecheck_stubs.py), pylint, doc8 and the Sphinx build are clean.
  • test_pickle_globals_reads_past_large_strings_and_buffers tests code this PR does not change. It failed once in five local full runs and passed 10 out of 10 runs on its own. It measures peak memory and looks flaky under load.

avgvi added 9 commits November 3, 2026 08:06
…t override _con to open another database work again.
…e) keys pass the write check, and refuse defaultdict(bool), which the reader refuses.
…ich raised ValueError from pickle.PickleBuffer.
…with_types disks do, and make refusal messages point to with_numpy() or say when no allowlist can help.
…with_types, as numpy is, because crafted pickles of them crash the reader or run code.
…t protect reads, and explain why Dash needs a cache folder of its own.
@read-the-docs-community

Copy link
Copy Markdown

Documentation build overview

📚 diskcache-community | 🛠️ Build #35619228 | 📁 Comparing 88e1202 against latest (30df125)

  🔍 Preview build  

4 files changed
± api.html
± fix-cve-2025-69872.html
± migrating.html
± tutorial.html

@avgvi
avgvi merged commit e09f8be into master Nov 3, 2026
21 checks passed
@avgvi
avgvi deleted the fix/adoption-sweep-5.8.3 branch November 3, 2026 16:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant