-
-
Notifications
You must be signed in to change notification settings - Fork 36
corekit: unpack a ListField's schema items from the configured field #453
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -314,5 +314,111 @@ def subclasses(cls: 'Any') -> 'Any': | |
| self.assertEqual(sorted(fell_back), []) | ||
|
|
||
|
|
||
| class ListFieldSchemaItemTests(unittest.TestCase): | ||
| """``ListField.unpack``'s schema branch must unpack from the *configured* | ||
| per-item field, not from ``self._item_type`` itself. | ||
|
|
||
| ``field = self._item_type(packet)`` builds a per-item copy through | ||
| :meth:`SchemaField.__call__ <pcapkit.corekit.fields.misc.SchemaField.__call__>`, | ||
| which applies both ``callback`` and ``length_callback`` to that copy -- the | ||
| schema branch then unpacked from ``self._item_type`` instead of from | ||
| ``field``, discarding whatever the copy carries. C.f. #433. | ||
|
|
||
| No declaration in this package passes either argument, so nothing here | ||
| parses differently today; both cases below construct a ``ListField`` | ||
| directly rather than through one of the four in-tree declarations, since | ||
| those are exactly the shape that hides the bug. | ||
|
|
||
| """ | ||
|
|
||
| def setUp(self) -> None: | ||
| purge_modules(['pcapkit']) | ||
|
|
||
| def test_a_length_callback_is_honoured(self) -> None: | ||
| """The per-item field's own resolved length must reach its schema. | ||
|
|
||
| ``field``'s ``length_callback`` resolves to ``2``; ``self._item_type``, | ||
| never having been called, is stuck at the ``-1`` its constructor left | ||
| it with. ``SchemaField.unpack`` threads its own ``self.length`` through | ||
| to ``Item.unpack``'s ``length`` argument, which lands in | ||
| ``packet['__length__']`` -- so which one was used is directly visible | ||
| to ``Item.pre_unpack`` without ``Item`` ever needing to consume it. | ||
|
|
||
| """ | ||
| from pcapkit.corekit.fields.collections import ListField | ||
| from pcapkit.corekit.fields.misc import SchemaField | ||
| from pcapkit.corekit.fields.numbers import UInt8Field | ||
| from pcapkit.protocols.schema.schema import Schema, schema_final | ||
|
|
||
| recorded = [] # type: list[int] | ||
|
|
||
| @schema_final | ||
| class Item(Schema): | ||
| """A single fixed-width byte: its own width never depends on | ||
| ``__length__``, so the list keeps making progress whichever length | ||
| got recorded.""" | ||
|
|
||
| marker: 'int' = UInt8Field() | ||
|
|
||
| @classmethod | ||
| def pre_unpack(cls, packet: 'dict[str, Any]') -> 'None': | ||
| recorded.append(packet['__length__']) | ||
|
|
||
| item_field = SchemaField(schema=Item, length=lambda pkt: 2) | ||
|
Owner
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Minor coverage gap, not blocking. This Confirmed this is a real gap and not just a style question: with the fix reverted, this test already fails on the constant case ( |
||
| list_field = ListField(length=2, item_type=item_field) | ||
|
|
||
| list_field.unpack(b'\x01\x02', {}) | ||
|
|
||
| self.assertEqual(recorded, [2, 2]) | ||
|
|
||
| def test_a_callback_is_honoured(self) -> None: | ||
| """The per-item field's ``callback`` mutation must reach ``unpack``. | ||
|
|
||
| The callback is evaluated regardless: ``field = self._item_type(packet)`` | ||
| runs it for its side effect even on the unpatched code. So what this | ||
| proves is not that the callback *fires*, but that the mutation it made | ||
| on its copy is what ``unpack`` actually used. It alternates | ||
| ``field._schema`` between two otherwise-identical schemas, one per | ||
| item; unpacking from ``self._item_type`` instead would use the schema | ||
| fixed at construction time for every item. | ||
|
|
||
| """ | ||
| from pcapkit.corekit.fields.collections import ListField | ||
| from pcapkit.corekit.fields.misc import SchemaField | ||
| from pcapkit.corekit.fields.numbers import UInt8Field | ||
| from pcapkit.protocols.schema.schema import Schema, schema_final | ||
|
|
||
| seen = [] # type: list[str] | ||
|
|
||
| @schema_final | ||
| class TypeA(Schema): | ||
| marker: 'int' = UInt8Field() | ||
|
|
||
| @classmethod | ||
| def pre_unpack(cls, packet: 'dict[str, Any]') -> 'None': | ||
| seen.append('A') | ||
|
|
||
| @schema_final | ||
| class TypeB(Schema): | ||
| marker: 'int' = UInt8Field() | ||
|
|
||
| @classmethod | ||
| def pre_unpack(cls, packet: 'dict[str, Any]') -> 'None': | ||
| seen.append('B') | ||
|
|
||
| counter = {'n': 0} | ||
|
|
||
| def alternate(field: 'Any', packet: 'dict[str, Any]') -> 'None': | ||
| field._schema = TypeB if counter['n'] % 2 else TypeA # pylint: disable=protected-access | ||
| counter['n'] += 1 | ||
|
|
||
| item_field = SchemaField(length=1, schema=TypeA, callback=alternate) | ||
| list_field = ListField(length=2, item_type=item_field) | ||
|
|
||
| list_field.unpack(b'\x01\x02', {}) | ||
|
|
||
| self.assertEqual(seen, ['A', 'B']) | ||
|
|
||
|
|
||
| if __name__ == '__main__': | ||
| unittest.main() | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Verified this independently rather than taking the PR's word for it.
Reproduced both sides of the fix. Reverted this one token locally (
field→self._item_type) and rerantests/corekit/test_fields_collections.py::ListFieldSchemaItemTests: both new tests fail with exactly the errors quoted in the PR body —Restored the fix and the whole module passes, 9/9.
Census re-derived, not trusted.
git grep -n "item_type=SchemaField" origin/main -- pcapkitfinds exactly the four sites named (hip.py:260, mh.py:535, sctp.py:702, tcp.py:393) and no fifth. NoSchemaFieldsubclass exists anywhere in the package (git grep -n "SchemaField)" ... | grep -i classis empty), and_item_typeis never read or written outside this file (git grep -n "_item_type" origin/main -- pcapkit | grep -v collections.pyis empty), soisinstance(self._item_type, SchemaField)can only be true via one of those four literal declarations — no dynamic or indirect route in. None of the four passescallback=, and thelength=lambda pkt: …each carries is on the outerListField, not the innerSchemaField; theSchemaFielditself gets either nolength=or a static int (length=4/length=8).One correction to the PR's own reasoning, not to its conclusion. The PR states that for these four sites
__call__returns a copy "whose_length_callbackisNone." That's only true for the two sites with an explicit staticlength=on the innerSchemaField(sctp.py, tcp.py). For the other two (hip.py, mh.py — nolength=on theSchemaField),SchemaField.__init__'s own default parameter value forlengthislambda _: -1, which is not anint, so_length_callbackgets set to that trivial lambda rather thanNone. Checked directly:The practical conclusion still holds — resolving that trivial callback in
__call__reproduces_length=-1,_template='1024s', identical to what__init__already left it at — so the copy is still equivalent to the original at these two sites. It's "the callback always resolves to the same constant" rather than "the callback is absent" that makes it inert here. Not a code issue, just worth tightening the description.Byte-identical captures, reproduced independently. Ran
examples/generators/make_samples.pyat this commit and again with the token reverted, and diffedsha256sumover all 24 files underexamples/captures/(18 generated + 6 committed): identical digests both ways, identical generator log lines.packside, confirmed by reading the code.ListField.packnever callsself._item_type(packet)for a schema item — it hitsisinstance(item, Schema)→item.pack(packet)directly, since every value produced by the schema branch ofunpackis already aSchemainstance.SchemaField.pack(misc.py) never references_length,_length_callback, or_templateat all. So theelif self._item_type is not None: self._item_type.pack(item, packet)branch is structurally unreachable forSchemaFielditems in practice, and even if it were reached,.packdoesn't consult the state the bug discards — confirmed both halves independently.