Skip to content

Preserve SortedDict invariants when update input raises - #255

Open
vitalivo wants to merge 1 commit into
grantjenks:masterfrom
vitalivo:fix/update-iterator-errors
Open

vitalivo wants to merge 1 commit into
grantjenks:masterfrom
vitalivo:fix/update-iterator-errors

Conversation

@vitalivo

Copy link
Copy Markdown

This replaces #254, which was accidentally closed and its source fork deleted. The implementation is unchanged; the original discussion and reviews remain linked there.


Calling update() on an empty SortedDict with a malformed pair or an iterator that raises can leave keys in the underlying dictionary but not in its sorted index. After catching the original exception, len(d) disagrees with iteration and _check() fails.

Materialize iterable input before taking the empty-dictionary fast path, as the nonempty path already does. The direct-dict optimization remains. Tests cover malformed pairs with and without a key function, a failing generator, and nonempty controls; three cases fail on the base revision.

Validation: all 372 tests, doctests, and stress tests pass on Python 3.12. Ruff passes for the changed library file. Repository-wide Ruff reports five existing B905 diagnostics in untouched sortedlist.py; formatting reports an existing adjacent string literal at sorteddict.py:194. No new lint or formatting issues were introduced.

@feiiiiii5

Copy link
Copy Markdown

Verified base (pr-255~1) against the branch. The fix does what it says: with a key that raises during update(), the base leaves the object failing _check(), while the branch leaves it consistent, and the new tests pass on both the empty and non-empty cases.

One semantic change is in the diff but not in the description, and it is worth a line either way: this makes SortedDict.update all-or-nothing, where the stdlib is partial.

plain dict, generator raising after two items:   d.update(gen)  -> {'a': 1, 'b': 2}
plain dict, [('valid', 1), ('invalid',)]:        d.update(...)  -> ValueError, d == {'valid': 1}
SortedDict on this branch, same inputs:                        -> ValueError, nothing applied

So for a caller who catches the exception, dict.update and SortedDict.update now disagree about what survived: the former keeps whatever it managed to consume, the latter keeps nothing. I do not think that is a mistake — refusing to apply anything is the only way to keep the index in step without a repair path, which is exactly what this PR is about — but it is a second behaviour change riding along with the index fix, and sortedcontainers is a dict-like, so the difference is observable.

Two smaller notes:

  • The initial parametrization is [{}, {'existing': 0}], and only the {} case exercises the branch that moved. The non-empty cases go straight through dict.update(self, pairs) / _list_update, so they are controls rather than coverage of the change. If the atomicity is intended, a test that contrasts the two — apply the same failing input to a plain dict and to the SortedDict and assert the documented difference — would pin the contract rather than just the outcome.
  • Hoisting pairs = dict(*args, **kwargs) above the if not self: fast path is the right shape for this, and it also means sd |= other_sorted_dict now materialises a plain dict first, which is a small allocation cost on the empty case. Not a problem, just noting it since __ior__ = update.

I checked the other two callers the reorder could touch: __init__ and fromkeys build their contents rather than going through update, so neither is affected. test_coverage_sorteddict.py covers the new branch from both the malformed-pair and failing-generator directions.

369 passed on the branch.

@feiiiiii5

Copy link
Copy Markdown

Correction: the suite count in my comment above is wrong — this branch is 372 passed, not 369. 369 was the count I measured on the #257 branch and I carried it over by mistake. Nothing else in the comment changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants