Design: adopt the EnumMeta/EnumSchema opt-in registration pattern for Protocol, Engine, Reassembly and TraceFlow

[JarryShaw]

Design proposal, filed to iterate on rather than as a ready change. All figures measured against 27bb315d5.

Observation: the mechanism already exists and has zero users

Every one of the four ABCs already auto-registers its subclasses via __init_subclass__. Every built-in declines it by inheriting a parallel *Base class instead:

public class (auto-registers)	descendants	non-registering base	descendants
Protocol	0	ProtocolBase	43
Engine	0	EngineBase	9
Reassembly	0	ReassemblyBase	5
TraceFlow	0	TraceFlowBase	2
EnumSchema	402	—	—

So the auto-registering half of each pair is a 0-user API, while EnumSchema — same idea, different implementation — has 402 registered descendants.

The opt-out is done by aliasing at import:

# pcapkit/foundation/engines/pcap.py:13
from pcapkit.foundation.engines.engine import EngineBase as Engine

Why this matters beyond tidiness: it has caused two real defects

#506's root cause. pcapkit/protocols/schema/schema.py:669 did isinstance(data, Protocol) where it meant ProtocolBase. Commit 680f84d4e ("revised Protocol metaclass") rewrote that import to ProtocolBase as Protocol in 20 schema/** modules but missed this one, because it is a runtime import inside pack() rather than a module-level TYPE_CHECKING one. With Protocol having 0 descendants, the branch was unreachable for three years — IPv4.make(payload=<TCP instance>) raised ProtocolUnbound despite its own signature accepting Protocol.

#513. extraction.py:390/412/434 validate issubclass(x, Engine | Reassembly | TraceFlow), so all three public register_extractor_* functions reject pcapkit's own built-ins.

Two independent failures, one cause: a public class and a base class that differ only in whether they register, where the base is what everything actually uses.

The difference from EnumSchema is two lines

# Reassembly.__init_subclass__          pcapkit/foundation/reassembly/reassembly.py:515-520
if protocol is None:
    protocol = cast('str', cls.name)          # invent a key...
Extractor.register_reassembly(protocol.lower(), cls)   # ...then register ALWAYS

# EnumSchema.__init_subclass__          pcapkit/protocols/schema/schema.py:1028-1033
if code is not None:                          # register ONLY when asked
    cls.__enum__[code] = cls

EnumSchema guards. The four ABCs fall back. That inversion is the whole reason a parallel hierarchy is needed to decline registration — and the reason cls.name gets used as a registry key, which is itself surprising: class X(Reassembly, name='zzz') registers under 'x'-style lowercased class name, silently, because name= is swallowed by **kwargs (see the docstring fix for name vs protocol).

The pattern being proposed, as EnumMeta/EnumSchema implements it

Four parts, and the four ABCs already have three of them:

1. A metaclass over abc.ABCMeta. SchemaMeta(abc.ABCMeta) at schema.py:135, EnumMeta(SchemaMeta, Generic[_ET]) at :915. All four ABCs already have one — ProtocolMeta, EngineMeta, ReassemblyMeta, TraceFlowMeta, all ABCMeta subclasses.
2. A class-level registry property on the metaclass (schema.py:933-935), an immutable proxy to __enum__. This is the part that needs the metaclass — a property in the class body would be an instance property, so MySchema.registry would not work. None of the four ABCs has this.
3. Opt-in registration in __init_subclass__ — (cls, /, code=None, *args, **kwargs), register only when code is given. All four ABCs already take an optional keyword (schema=None, name=None, protocol=None, protocol=None); only the guard differs.
4. An explicit register() classmethod (schema.py:1038) coexisting with the automatic path. All four already have their register_* equivalents.

Plus a lazy per-root registry so each namespace owns its own mapping:

if not hasattr(cls, '__enum__'):
    cls.__enum__ = collections.defaultdict(cls.__default__)

Proposed change, per class

1. Invert the fallback into a guard: if protocol is None: super().__init_subclass__(); return. Built-ins then subclass the public class directly and simply omit the keyword — no *Base needed.
2. Add a registry property to each metaclass, mirroring EnumMeta.registry, so Reassembly.registry reads like MySchema.registry.
3. Keep *Base as an alias for backwards compatibility, but it stops being a mechanism.

What this buys: #513 disappears rather than needing its own patch; the EngineBase as Engine trick stops being load-bearing, so the next sweep like 680f84d4e's cannot half-miss it; and the four hierarchies come to match the one that already works at 402 subclasses.

The cost, stated plainly

This is a breaking change for external subclassers. Anyone who today writes class MyEngine(Engine) and relies on free registration under the class name would stop being registered unless they pass the keyword. In-tree the blast radius is zero — public-class descendants are 0 across all four — so only third-party code is affected. The 1.5.0 prerelease window is the right time to take it, with a changelog entry.

Open questions, which is why this is a design issue and not a PR

1. Why do the built-ins opt out today? The aliasing is clearly deliberate but the reasoning is documented nowhere. Plausibly: registering at class-definition time would populate the registry in import order, making duplicate keys and circular imports live concerns, and the built-ins are already wired in by explicit table entries. If that reasoning is sound it should be captured before it is discarded — and it may argue for keeping the explicit tables and only fixing register_extractor_engine/reassembly/traceflow reject pcapkit's own built-in classes #513.
2. Should cls.name remain a fallback key at all? Deriving a registry key from a class name is what makes the name=-swallowed bug silent instead of loud. Requiring the keyword makes it loud.
3. Does Protocol still need to exist once ProtocolBase registers on request? Its docstring describes it as the subclass for "external customised engines with auto registration", which the guard form makes unnecessary.
4. Ordering. EnumSchema registers into a per-root defaultdict owned by the schema tree; the four ABCs register into Extractor, a different module, via a deferred import inside __init_subclass__. Whether that indirection survives the change is worth settling before coding.

[JarryShaw]

Supposedly, all Protocol subclasses should be issubclass to Protocol itself, and ProtocolBase as well. Maybe something we broke in the logic?

The auto-registration should be applied to those top level subclasses, like Link, Internet if possible and applicable, with individual protocol overridable, cause for UDP and TCP, they will have different registry rather a shared one in Transport.

Enum value or port number should be the key for the registry key and ModuleDescriptor or the actual class object as value (lazy loading? or eager loading?).

Registry ordering does not matter.

And if no registry key is passed in at class definition (old behaviour), we can skip/ignore it from the registry and let the user to manually register it. So that the change is backward compatible - we should extend this to the EnumSchema as well, classes should be able to opt-out and register on demand.

[JarryShaw]

Taking this as the design answer. Recording what it settles, plus answers to the two things left open in it.

Settled by your comment: auto-registration on top-level subclasses (Link, Internet), individually overridable, with UDP/TCP holding their own registries rather than sharing Transport's; enum value or port number as the key; ordering irrelevant; and opt-out by default — no registry key at class definition means skip, leaving the user to register manually, extended to EnumSchema too.

"Maybe something we broke in the logic?" — no, it is deliberate. The Protocol/ProtocolBase split came in with 680f84d4e ("revised Protocol metaclass"), and there are 80 *Base as <Public> alias imports whose whole purpose is declining auto-registration. Measured descendants: Protocol 0 / ProtocolBase 43, Engine 0 / EngineBase 9, Reassembly 0 / ReassemblyBase 5, TraceFlow 0 / TraceFlowBase 2. So the aliasing is the opt-out mechanism, not a bug.

And your design removes the need for it. Once a class opts in by passing a registry key, subclassing the public class no longer implies registration — so the built-ins can subclass Protocol directly and simply pass no key. That is exactly the state your first line says it should be in, reached by the design rather than by a separate fix.

"lazy loading? or eager loading?" — lazy, and the codebase already does it. Built-in registry defaults are stored as ModuleDescriptor, not classes:

__engine__ = { 'scapy': ModuleDescriptor('pcapkit.foundation.engines.scapy', 'Scapy'), ... }

and resolution is lazy with memoisation — on first use the descriptor is resolved and the mapping is rewritten in place (extraction.py:866-887):

reasm_cls_ipv4 = self.__reassembly__['ipv4']
if isinstance(reasm_cls_ipv4, ModuleDescriptor):
    reasm_cls_ipv4 = reasm_cls_ipv4.klass
    self.__reassembly__['ipv4'] = reasm_cls_ipv4  # update mapping upon import

Meanwhile register_* accepts either form and resolves a descriptor immediately. So the settled pattern is descriptors for built-ins (lazy, resolved once), class objects accepted for user registration (eager) — worth keeping as-is, since eager-loading the built-ins would import every protocol module at startup.

That leaves nothing open that I am aware of. #523 is therefore unblocked — it widens the three issubclass gates to the *Base classes as an interim alignment, and note it becomes moot rather than wrong once the aliases disappear under this design, since issubclass(x, Engine) would then be true again.

[JarryShaw]

Design answer, measured on main @ 122d32795 in a clean worktree. Every number below was produced by a __subclasses__() walk or a live id()/identity check in an interpreter that asserted its own import path first — pcapkit.__file__ = .../worktrees/agent-a9a318aa5b1dea2e8/pcapkit/__init__.py — because an editable install in site-packages will silently shadow a worktree and make all of this measure the wrong tree.

The short version: your requirements land on two different kinds of registry, and auto-registration today writes to the wrong one. Naming that split is most of the design, so it comes first.

0. There are two registries, not one

	name-keyed identity registries	code-keyed dispatch registries
answers	"which class is the one called TCP"	"given wire code X, what parses the payload"
where	pcapkit.protocols.__proto__ (protocols/__init__.py:73), Extractor.__engine__ / __reassembly__ / __traceflow__ / __output__, TraceFlow.__output__	ProtocolBase.__proto__ (protocol.py:123) and the seven classes that own their own copy
key	a string — 'TCP' upper, 'ipv4'/'scapy' lower	EtherType, TransType, LinkType, PayloadProtocolIdentifier, or a raw int port
how many	exactly six, all under pcapkit/foundation/	~30 under pcapkit/protocols/, counting the option/chunk sub-tables
value	the class	ModuleDescriptor for built-ins, class for anything registered explicitly
written by __init_subclass__ today	yes — this is the whole of the auto-registration	no — never

Your line "Enum value or port number should be the key for the registry key" describes the right-hand column. The auto-registration that exists today writes only the left-hand one: Protocol.__init_subclass__ (protocol.py:1442-1443) calls register_protocol(cls), which does protocol_registry[protocol.__name__.upper()] = protocol (foundation/registry/protocols.py:150) — a class-name key into a flat module-level dict.

There is a corollary worth saying out loud, because it is good news: your "enum value or port number should be the key" principle already holds for 100% of the protocol-dispatch tree. Every table under pcapkit/protocols/ is keyed by an enum member or a port. The six string-keyed registries are exactly the ones where no wire code exists to use instead — which third-party parsing engine, which output format, which reassembly implementation. Those should stay string-keyed; there is nothing to convert. So the keying requirement costs nothing, and what is actually at stake is only who writes into those registries and when.

So this is not "invert a guard and you have it". The guard inversion is real and cheap, but it delivers opt-out, not the keyed per-family registry; those are two separate pieces of work and I would ship them as separate PRs.

1. The 0-descendant state: deliberate, not a regression

Nothing broke. The split was designed this way, and the built-ins were pointed at the non-registering base in the same commit that created the split, for all four pairs, over about eighteen hours on 2023-10-03/04:

commit	date	subject
ce4752ad7	2023-10-03	revised Engine metaclass
1b638cc8c	2023-10-03	renamed _Engine as EngineBase (cosmetic)
deee6b134	2023-10-03	revised TraceFlow metaclass
7f42b0229	2023-10-04	revised Reassembly metaclass
680f84d4e	2023-10-04	revised Protocol metaclass — 59 files

All four carry the same three-bullet message, verbatim, with only the class name substituted:

* added a EngineMeta for better processing
* use _Engine for internally defined engines
* added Engine wrapper for external customised engines with auto registration

And in ce4752ad7 the same diff that creates EngineMeta, _Engine and Engine.__init_subclass__ also rewrites the built-ins:

--- a/pcapkit/foundation/engines/pcap.py
-from pcapkit.foundation.engines.engine import Engine
+from pcapkit.foundation.engines.engine import _Engine as Engine

There is no commit anywhere in history where a built-in subclassed a public registering class and was later switched away — which is what a regression would look like. The intent is also stated in every *Base docstring from that day and still present: "This class is for internal use only. For customisation, please use Engine instead." It shipped as 1.3.0, whose changelog opens "A redesign of the extension points, on metaclasses" (docs/source/changelog/1.3.0.rst:4).

So the 0 is deliberate. What it is not is harmless, and that is the actual case for changing it: the aliasing is a convention rather than a mechanism, so a sweep over it can miss a site and nothing fails loudly. It has happened twice — #506 (one runtime import in schema/schema.py missed by 680f84d4e's 59-file sweep, unreachable for three years) and #513 (the extraction.py gates never updated by ce4752ad7/deee6b134/7f42b0229, so the helpers rejected the library's own built-ins for three years, fixed yesterday in #523). Your design removes the convention, which is why it is worth doing even though nothing is broken right now.

Scope, while we are here: five pairs, not four. The 1.3.0 changelog also names Field and Vendor, so I measured those too — Field has 26 descendants against FieldBase's 35, and its public class carries a real __init__ and a template property with no registration at all (corekit/fields/field.py:248-291), so it is a legitimate split that should be left alone; pcapkit/vendor/default.py has no pair, only VendorMeta and Vendor. The family in scope is exactly the pairs whose public class adds nothing but registration, and that is five: the four named plus Dumper, which arrived later and almost incidentally, in 3556d392d ("revising documenations", 2023-10-08).

2. What the public classes add beyond registration: nothing at all

Read end to end, each public class is a docstring plus one __init_subclass__ whose body registers and delegates. Not one attribute, method, property or default beyond that:

public class	defined at	body beyond registration
Protocol	protocols/protocol.py:1409-1445	none — docstring, then register_protocol(cls) and super().__init_subclass__(schema, data, *args, **kwargs)
Engine	foundation/engines/engine.py:185-225	none — Extractor.register_engine(name.lower(), cls)
Reassembly	foundation/reassembly/reassembly.py:475-521	none — Extractor.register_reassembly(protocol.lower(), cls)
TraceFlow	foundation/traceflow/traceflow.py:407-452	none — Extractor.register_traceflow(protocol.lower(), cls)
Dumper	dumpkit/common.py:54-108	none — Extractor.register_dumper + TraceFlow.register_dumper

On the fifth row: Dumper/DumperBase (dumpkit/common.py:54 and :44) measures 0 and 3 descendants, with the same aliasing (dumpkit/pcap.py:16, dumpkit/null.py:18) and the same fallback. Its fallback is the worst of the five — with no fmt=, __init_subclass__ instantiates the class against a NamedTemporaryFile to read .kind (common.py:94-96). A class statement performs filesystem I/O.

Because the classes add nothing, the answer to your first line falls out of the guard form for free: once a key is what opts a class in, the built-ins can subclass the public class and simply pass no key, so issubclass(TCP, Protocol) becomes true again without a separate fix. And there is then no reason for the public classes to exist as anything but import-compatibility aliases — so keep them, deprecate them in prose, remove nothing.

3. Dead isinstance/issubclass gates: there are none left

I swept every isinstance/issubclass in pcapkit/ and tests/ whose type argument names any of Protocol, ProtocolBase, Engine, EngineBase, Reassembly, ReassemblyBase, TraceFlow, TraceFlowBase, then resolved each occurrence's binding — by importing the module and comparing identity, not by reading, because in most modules Protocol is ProtocolBase under an alias.

27 occurrences, 27 live, 0 dead. #506's site (schema/schema.py:691) and #513's three (extraction.py:395/419/443) are the only ones that were ever dead and both are fixed in this tree. register_dumper at extraction.py:368 and traceflow.py:192 is not the same shape: Dumper there is dictdumper.dumper.Dumper (measured __module__), the third-party ABC that DumperBase subclasses, so the gate is already wide.

So the case for this change is not that there is dead code left to fix. It is that the mechanism keeps producing dead code: two independent sweeps over the same aliasing trick each missed exactly one site, and neither miss could fail loudly. Three things the sweep did turn up, all worth folding in:

• Eleven RegistryError messages name the wrong class. protocol.py:451, internet.py:152, frame.py:139, link.py:132, pcapng.py:782, transport.py:95, sctp.py:616, registry/protocols.py:148 and extraction.py:396/420/444 all say "must be a Protocol/Engine/Reassembly/TraceFlow subclass" while the check accepts the *Base. Never rejects anything wrongly — it is a superset — but it tells a caller to do the one thing that is not required, which is how IPv4.from_data() cannot rebuild a parsed packet: ProtocolUnbound on a Raw payload #506 happened in the first place.
• corekit/protochain.py is one forgotten import from a NameError. Its only module-level Protocol binding is inside if TYPE_CHECKING: (:23), so all five runtime users carry their own duplicated local ProtocolBase as Protocol (:73, :112, :132, :159, :206). It works because somebody remembered five times.
• tests/protocols/schema/test_schema_unit.py:167's DummyProtocol is the only real subclass of a public class in the repo, and it exists precisely to exercise the path nothing else uses.

4. The per-family registry you describe already exists

This is the part I expected to be a restructuring and it is not. Measured, by id() and __dict__ membership:

registry owner	own __proto__?	key type	entries
Link	yes	EtherType (+ one raw int, 0x8137 for IPX)	7
Internet	yes	TransType	16
TCP	yes	int port	4
UDP	yes	int port	3
SCTP	yes	PayloadProtocolIdentifier	2
Frame	yes	LinkType	3
PCAPNG	yes	LinkType	3
Transport	no — inherits the empty root	—	0
ProtocolBase	yes (the root)	—	0

TCP.__proto__ is UDP.__proto__ is False, and Transport.__proto__ is ProtocolBase.__proto__ is True — i.e. UDP and TCP already have different registries rather than a shared one in Transport, and Transport deliberately has none. Transport.register even enforces it, and its docstring states your principle almost word for word: "This method must be called from a non-abstract class, as the protocol map should be associated directly with specific transport layer protocol type" (transport.py:83-86, enforced at :89-90).

So nothing has to move. Three things are missing, though:

(a) Ownership is not enforced anywhere else, and the failure is silent. Only Transport.register guards, and only against cls is Transport exactly. Everywhere else a class without its own __proto__ writes into its parent's. Measured: IPv4.register(65002, X) lands in Internet.__proto__ and is therefore visible from IPv6; FTP.register(65003, X) lands in ProtocolBase.__proto__, the root shared by 12 classes. Link.register and Internet.register have no guard at all (link.py:129-135, internet.py:149-155). This is what "with individual protocol overridable" needs fixed, and it is the only part of this work with in-tree behaviour change — see question 3.

(b) A single code= keyword cannot express the target, because these registries are cross-family. A layer's dispatch registry points at the layer above it: Link.__proto__ maps EtherType to IPv4/IPv6/IPX, which live under protocols.internet; Internet.__proto__ maps TransType to TCP/UDP/SCTP, under protocols.transport. So class IPv4(Internet, code=...) cannot mean "register me into Link" under plain __init_subclass__ semantics — the only registry in its MRO is Internet's. And one class needs entries in several: IPv4 belongs in Link.__proto__ (EtherType), Frame.__proto__ and PCAPNG.__proto__ (LinkType), and Internet.__proto__ (TransType.IPv4, for IP-in-IP).

The clean way out, and what I would build, is to let the key's own enum type name the target registry: EtherType → Link, TransType → Internet, LinkType → {Frame, PCAPNG}, PayloadProtocolIdentifier → SCTP. That mapping is not an invention — it already exists in the codebase as one function per enum type, and each one hard-codes exactly this targeting: register_ethertype → Link.register (registry/protocols.py:310), register_transtype → Internet.register (:352), register_linktype → Frame.register and PCAPNG.register (:193-194), register_sctp → SCTP.register (:805). The proposal is only to move that knowledge from twenty-five hand-written functions into one declarative table the keyword can consult. Then class IPv4(Internet, code=[EtherType.Internet_Protocol_version_4, LinkType.IPV4, TransType.IPv4]) is unambiguous and reads well, and the LinkType case fanning out to two registries is exactly what register_linktype already does by hand (registry/protocols.py:193-194). Raw int ports are the one genuinely ambiguous key — 20 could be TCP or UDP — so those must name their family explicitly, e.g. code={TCP: 21, UDP: 21}. There is precedent for that too: register_apptype (registry/protocols.py:576) already takes a keyword-only proto= to say which of TCP/UDP it means, and excludes SCTP because its key is a PPID rather than a port. Both forms in one keyword, then: a mapping when you need to be explicit, a flat iterable of enum members when the type says it for you.

Worth noting for scope: the same shape recurs about two dozen more times below the protocol level — TCP.__option__, SCTP.__chunk__/__parameter__/__cause__, IPv4.__option__, HIP.__parameter__, MH.__message__/__option__/__extension__, PCAPNG.__block__/__option__/__record__/__secrets__ and the rest, each with its own register_*. All are already enum-keyed and per-class. I would leave every one of them alone for now; they are the natural second customer for whatever code= ends up looking like, not part of this change.

(c) No registry property. EnumMeta.registry (schema.py:948-951) is the piece none of the five metaclasses has, and it has to live on the metaclass — a property in the class body would be an instance property, so Reassembly.registry would return the property object. This is cheap: EngineMeta and ReassemblyMeta already carry class-level properties (name, module, protocol at engine.py:38-50, reassembly.py:68-80), so it is four lines each.

One warning about copying EnumSchema here. Its per-root materialisation is if not hasattr(cls, '__enum__') (schema.py:1041), and hasattr sees inherited attributes — which is correct for schemas, where a root like hopopt.Option owns the mapping and its children register into it. Copied verbatim onto __proto__ it would reproduce bug (a) above exactly: IPv4 would be handed Internet's registry because it inherits one. Ownership has to be tested with '__proto__' in cls.__dict__, not hasattr.

5. What the registries store, and lazy vs eager

Today: built-in defaults are ModuleDescriptor (a 2-field named tuple, module and name, corekit/module.py), and every explicit register* call resolves eagerly to a class before storing (protocol.py:448-449, link.py:129-130, internet.py:149-150, transport.py:92-93). Resolution on the read path memoises: _lookup_next_layer writes the class back into the mapping on first hit (protocol.py:1308-1315), and extraction.py:866-887 does the same for the reassembly tables.

So the registries hold a union today, which answers "ModuleDescriptor or the actual class object as value" — both, and that part I would keep.

The documented intent is explicit. 1.3.0's changelog added ModuleDescriptor "so a registry entry can name a module and class that are imported lazily on first use rather than at registration" (docs/source/changelog/1.3.0.rst:9-12).

That intent is no longer met, and it is worth stating plainly: the library is already fully eager, so the laziness is vestigial. import pcapkit loads 315 of the 326 non-vendor modules in 644 ms. The only 11 it does not load are __main__, all, interface.misc and the nine toolkit.* third-party adapters. Every protocol module is already in sys.modules. And it is not avoidable by importing narrowly: import pcapkit.protocols.link.ethernet alone also loads 315 modules, because Python runs pcapkit/__init__.py first and protocols/__init__.py:73-75 builds the name-keyed registry out of globals(), which requires importing every protocol to have something to put in it.

Consequence: when Link.__proto__ resolves ModuleDescriptor('pcapkit.protocols.internet.ipv4', 'IPv4'), the module is already imported and the descriptor defers nothing but an importlib cache hit and a getattr. Verified: after import pcapkit.protocols.link.ethernet, every value in Link.__proto__ and TCP.__proto__ is still a ModuleDescriptor while pcapkit.protocols.internet.ipv4 is already in sys.modules.

What the descriptors do buy is cycle-breaking: link.py can name IPv4 without importing protocols.internet. That is a real and load-bearing property, not an optimisation — and note that per-class code= registration from 4(b) removes the need for it in the right direction, since ipv4.py already imports Internet and would only additionally need the const enum modules, which import nothing back.

Recommendation: keep ModuleDescriptor as an accepted value type, keep the built-in tables as they are, and store class objects for anything registered through code=. What that gives up: the registries stay heterogeneous, so every read path keeps its isinstance(x, ModuleDescriptor) branch. What it preserves: the option of ever making import pcapkit actually lazy, which is a worthwhile future change that converting the tables to class references would foreclose. I would not go the other way and eagerly resolve the tables at import — it would make the eagerness permanent and unfixable in exchange for deleting a two-line branch.

One thing I would change here regardless: register() resolving a ModuleDescriptor eagerly (protocol.py:448-449) defeats the purpose of passing one. It is done so the issubclass check can run. That is a genuine trade-off — validate now and import now, or store lazily and discover a bad class at first dispatch — and the current choice is defensible; it is just worth a comment saying so, since it reads like an oversight.

6. EnumSchema already opts out — verified

Yes, and this shrinks the work. EnumSchema.__init_subclass__ at schema.py:1018 reads:

if not hasattr(cls, '__enum__'):
    cls.__enum__ = collections.defaultdict(cls.__default__)

if code is not None:                      # schema.py:1044
    ...
    cls.__enum__[code] = cls

No code, no registration, no fallback to the class name — which is exactly the behaviour you are asking the other five to adopt, and it is why that hierarchy carries 402 descendants while the five public classes carry 0 between them. (Its registries are per-root rather than one global: 28 direct EnumSchema subclasses each own an __enum__, e.g. hopopt.Option with 16 entries and hip.Parameter with 48.) EnumSchema needs no change for your last requirement. Its register() classmethod at :1053 is the manual path, already there.

The one thing EnumSchema does that the five should not copy verbatim is the hasattr materialisation, for the reason in 4 above.

7. Backward compatibility: exactly what breaks

Verified by construction, in a live interpreter, three ways of writing it today:

class NoKeyword(Reassembly):                          -> registers under 'nokeyword'
class WrongKeyword(Reassembly, name='...'):           -> registers under 'wrongkeyword'
class RightKeyword(Reassembly, protocol='right_key'): -> registers under 'right_key'

The middle line is the footgun: name= is not the keyword (protocol= is), the wrong one is swallowed by **kwargs and dropped by the bare super().__init_subclass__() at reassembly.py:521, and the class-name fallback fires. No exception, no warning, no log. The keywords also diverge across the five families — Engine takes name=, Reassembly and TraceFlow take protocol=, Protocol takes schema=/data=, Dumper takes fmt=/ext= — so guessing wrong is the expected outcome, not a careless one.

Under the guard form, precisely this changes:

• Line 1 stops registering. An out-of-tree class MyEngine(Engine) that relied on the derived key is no longer in the registry, and Extractor(engine='myengine') stops finding it. This is the break, it is the one you asked for, and in-tree blast radius is zero — 0 descendants across all five public classes, so nothing the library ships is affected. The fix for a user is one keyword or one register_* call.
• Line 2 stops registering too, and that is strictly an improvement — it currently registers under a key the author did not choose and will never look up.
• Nothing else in the class-definition path changes. Every *Base import keeps working unchanged; all five public classes remain importable; every explicit register_* helper keeps accepting exactly what it accepts today; EnumSchema is untouched.
• If question 3 is answered "raise", one further thing breaks and it does reach in-tree code: IPv4.register(...), FTP.register(...) and any other call through an inherited __proto__ would start raising UnsupportedCall instead of silently writing into the parent registry. Nothing in the library calls those, but it is a public API and it is the one change here that a user could hit without having subclassed anything. I think it is worth taking — a registration that lands somewhere other than where the caller asked is worse than an exception — but it should be a deliberate choice rather than a side effect, which is why it is its own question and its own PR.

I would also fix, as freebies in the same PR: the eleven error strings from section 3, the name:/protocol: docstring mismatch on Reassembly.__init_subclass__ (reassembly.py:506) and TraceFlow.__init_subclass__ (traceflow.py:437), and Reassembly's class docstring, which currently reads "Base flow tracing class" (reassembly.py:476) — copied from TraceFlow.

One more found on the way past, unrelated to the guard but in the same machinery: the engine lookup is the only one of Extractor's four that does not memoise. extraction.py:477-480 resolves self.__engine__[self._exnam] from a ModuleDescriptor and throws the result away, while its three siblings all write the class back (:866-869, :875-878, :884-887, :941-944, :967-973, each with the comment "update mapping upon import"). Nothing explains the difference, so every Extractor.run() re-resolves the engine. One line to fix; happy to fold it in or leave it out.

Open decisions are in the next comment, numbered, one line each.

[JarryShaw]

Questions blocking implementation

Seven decisions. Each is one line to answer — "yes to all recommendations" is a complete reply. My recommendation is on every one, so silence on a question means I take the recommendation.

Everything I could settle by measuring, I settled in the design comment above and is not here.

---

1. Scope: five families or four?
There is a fifth pair the issue does not name — Dumper / DumperBase at pcapkit/dumpkit/common.py:54 and :44, measured 0 and 3 descendants, same aliasing (dumpkit/pcap.py:16), same fallback. Its fallback is the worst of the five: with no fmt=, __init_subclass__ instantiates the class against a NamedTemporaryFile to read .kind (common.py:94-96), so a class statement does filesystem I/O.
Recommendation: include it. Excluding it leaves one family behaving the old way for no reason.

2. Do the built-in __proto__ tables become per-class code= declarations, or stay as they are?
This is the big one. The literal tables in link.py:76, internet.py, tcp.py, udp.py, frame.py, pcapng.py, sctp.py are what populate the dispatch registries today. code= at class definition could replace them — class ARP(Link, code=EtherType.Address_Resolution_Protocol).
Recommendation: keep the tables; make code= additive and opt-in, for out-of-tree protocols only. Converting the tables is a large diff that changes no behaviour (the measurement in the design comment shows nothing is actually lazy today), and it moves each protocol's wire code away from the layer table where all of them are currently readable side by side. If you want the conversion, it should be its own PR after this one.

3. Writing through an inherited registry: raise, copy, or leave?
Measured: IPv4.register(code, X) writes into Internet.__proto__, shared with IPv6; FTP.register(...) writes into ProtocolBase.__proto__, shared with 12 classes. Only Transport.register guards this, and only for cls is Transport exactly. This is what "individual protocol overridable" needs to be fixed for.
Recommendation: raise UnsupportedCall, naming the owning class, exactly as Transport.register already does — loud beats silent, and it is the only option that cannot diverge. The alternative (copy-on-write into a fresh per-class dict) means later additions to the parent stop being visible to the child, which is a subtler bug than the one being fixed. Note this one is a behaviour change reachable from in-tree classes, unlike everything else here.

4. Keep cls.name as the fallback key for the name-keyed registries?
Today class MyEngine(Engine) silently registers under 'myengine', and — verified — class X(Reassembly, name='...') registers under 'x', because name= is not the keyword (protocol= is) and the wrong one is swallowed by **kwargs.
Recommendation: drop the fallback entirely (that is what your opt-out requirement asks for), and additionally reject unrecognised class keywords instead of swallowing them. The second half is not in your requirements; without it the footgun survives the change, because a misspelled keyword will read as "no key given" and silently skip registration rather than registering under the wrong one.

5. Do the public classes stay?
They add nothing but registration — verified, all five are a docstring plus one __init_subclass__. Under the guard form they become empty subclasses.
Recommendation: keep all five as plain aliases, deprecate them in the docs, remove nothing. They are a documented import path; deleting them buys nothing and breaks people. This makes your first line true — every protocol becomes issubclass to both Protocol and ProtocolBase — without a separate fix.

6. Do we unwind the *Base as <Public> alias imports in the same PR?
Once registration is opt-in, the aliases stop being load-bearing and the built-ins can import the public name directly.
Recommendation: no — leave every alias alone in this PR. It is a large mechanical diff with zero behaviour change, and mixing it with the semantic change makes the semantic change unreviewable. Worth its own follow-up, or worth never doing.

7. One PR or a sequence, and in what order?
Recommendation: three PRs — (a) invert fallback to guard in all five __init_subclass__, plus the registry property on each metaclass, plus reject-unknown-keywords; (b) the registry-ownership fix from question 3, which is the only part with in-tree behaviour change and deserves its own review; (c) optional code= support on the dispatch registries for out-of-tree protocols. Each builds green on its own and (a) alone satisfies your opt-out requirement.

---

Anything you do not answer, I will take the recommendation on and say so in the PR description. Questions 2 and 3 are the two I would most like an explicit answer to — they are the ones where I could be talked out of my own recommendation.

[JarryShaw]

Correction to one premise above, before you answer Q5

The comment of 2026-09-19T23:36 states, on the lazy-versus-eager question: "lazy, and the codebase already does it", and closes with "eager-loading the built-ins would import every protocol module at startup."

The mechanism it describes is real; the justification is not. ModuleDescriptor storage with memoised resolution does exist and works as described. But the cost it is defending against has already been paid at import time, so descriptors defer a getattr, not an import.

Measured on 122d32795, repo venv, PYTHONSAFEPATH=1, with pcapkit.__file__ asserted to be this tree:

import pcapkit: 614 ms, 315 pcapkit modules loaded, 315 of them non-vendor

A bare import pcapkit already loads essentially the whole non-vendor tree. So "eager-loading the built-ins would import every protocol module at startup" is not a live trade-off — those modules are imported either way.

That does not make descriptors pointless. Their remaining value is cycle-breaking: a registry populated at class-definition time cannot hold a class that is not yet defined. That is a real constraint and an argument for keeping the union type. It is just a different argument from the one recorded above, and it points at a different answer to "lazy or eager" — the choice is about import-cycle safety, not startup cost.

Flagging it because Q5 in the question list asks you to choose between them, and the premise as written would push that choice for the wrong reason. The recommendation in the design comment (keep the union, store classes for code= registrations, do not eagerly resolve the tables) is unaffected either way.

One smaller correction to the same comment's line references, for anyone reading it later: EnumSchema's opt-out guard is at pcapkit/protocols/schema/schema.py:1044, and the three widened gates from #523 are at pcapkit/foundation/extraction.py:395, :419 and :443. Earlier ranges quoted in this thread pointed at the docstrings and # NOTE: comments above them rather than the code.

[JarryShaw]

Q1: all, including Dumper
Q2: keep the table and go with your recommendations
Q3: I'm not entirely followed. But the expectation should be for a new class to auto-register, it should be registered to the desired parent class's registry - and there's a problem where it may not actually be added to the registry same as its parent, e.g., L2TP, iirc we added it to Internet's registry despite it being a Link subclass. So we might want to, either infer the registry based on the Enum class, or having another argument for the actual destination class('s registry).
Q4: your recommendation makes sense to me.
Q5: what public classes? If it's the layer's base classes, those are needed.
Q6: Actually my intention was to keep Base classes hidden recipe and forcing all subclasses to be inherited directly from the first child, e.g., ProtocolBase is hidden and all should inherit from Protocol itself.
Q7: separate ones so that we can review individually and the size of PRs can be reasonable.

[JarryShaw]

Thanks — Q1, Q2, Q4 and Q7 are clear and I have started on the first PR. Three things back: your L2TP hunch measured, the Q6 reconciliation, and Q5 re-asked because I asked it badly.

Q3 — your L2TP hunch, measured

You hedged with "iirc", so plainly: half right, and the half that is right matters more than the example.

Measured on main @ 691f12ab5:

L2TPv2 MRO        : L2TPv2 -> L2TP -> Link -> ProtocolBase
L2TP.__layer__    : 'Link'          L2TPv2.__layer__ : 'Link'
the ONLY registry entry anywhere:   UDP.__proto__[1701] = ModuleDescriptor('…link.l2tpv2', 'L2TPv2')
Internet.__proto__ entries for L2TP: none
TransType.L2TP (=115) is a key in Internet.__proto__:  False
EtherType members matching 'l2tp':  []
L2TP (the v1 base) reachable from any dispatch registry: False

So it is not in Internet's registry. But the phenomenon you are describing is real and stronger than you remembered: L2TPv2 is a Link-layer class registered in a Transport-layer registry. And that is correct, not a bug — L2TP runs inside UDP port 1701 (RFC 2661), and a dispatch registry is keyed by the enclosing layer, not by the registered class's own layer. Link.__proto__ would be the wrong home for it; nothing would ever look there.

That is the decisive argument between your two mechanisms. You offered "infer the registry based on the Enum class" or "another argument for the actual destination class". Inferring from the enum works for L2TPv2; inferring from the class's own parent would put it in Link.__proto__ under a key EtherType does not even define, and dispatch would break silently. So: enum type infers the destination by default (EtherType→Link, TransType→Internet, LinkType→Frame+PCAPNG, PayloadProtocolIdentifier→SCTP), and the explicit destination argument is kept for raw int ports only, where 1701 cannot say whether it means TCP or UDP. Both of your mechanisms, each where it is unambiguous. What that gives up: two spellings for one keyword, so the docstring has to explain when each applies.

One real gap did fall out, and it is probably what you were remembering. TransType.L2TP = 115 exists as a constant but is registered nowhere, so L2TP-over-IP is not dispatched at all — only the UDP encapsulation is. Separately, L2TP itself (the v1 base) is reachable from no registry. I will file that as its own issue with this evidence rather than fold it in here; say the word if you would rather it rode along with the registry PR.

Q6 — reconciling your intention with what the code does today

I read this as changing my recommendation rather than confirming it, so checking before I act. Your intention — ProtocolBase hidden, everything inheriting Protocol directly — is the opposite of today's arrangement, where the built-ins subclass *Base under an alias specifically so they are not auto-registered. My archaeology found that was deliberate (four commits, 2023-10-03/04, shipped as 1.3.0).

The guard form already answers it, and is what makes your intention reachable at all. Once registration is opted into by passing a key, subclassing the public class no longer implies registration — so the built-ins can inherit Protocol directly and simply pass no key. Nothing double-registers and nothing lands under a wrong key: with no key, __init_subclass__ does nothing, and the name-keyed registry the built-ins actually rely on is populated by the for name in __all__ walk at pcapkit/protocols/__init__.py:73-75, not by __init_subclass__ — so that path is untouched either way. The *Base classes stop being a mechanism and go back to being an implementation detail, which is exactly what you asked for.

But the ordering is load-bearing: the guard has to land before any alias is unwound. The moment a built-in inherits Protocol while today's cls.name fallback is still in place, it auto-registers under its own class name — which is the bug, not the fix. So unwinding the ~76 alias imports (24 runtime, 52 TYPE_CHECKING-only) becomes its own PR after the guard, per your Q7. My earlier "leave the aliases alone" recommendation was wrong about the destination and right about the sequencing; treating it as a later PR gets both.

Questions

8. Q5, re-asked properly — I asked about class visibility by accident and you answered that; the actual question is about registry values. Should the built-in registry defaults stay ModuleDescriptor (a module+class name pair, imported on first dispatch), or become plain class objects resolved at import time?
Context: the startup argument for descriptors does not hold — import pcapkit already loads 315 of 326 non-vendor modules in ~614 ms, and importing a single leaf module does the same, so the descriptors defer a getattr, not an import. What they genuinely buy is import-cycle safety: link.py can name IPv4 without importing protocols.internet.
Recommendation: keep ModuleDescriptor — purely for the cycle safety and to keep a future lazy import pcapkit possible. "Yes, keep them" is a complete answer.

9. Your Q5 answer reads as overturning a decision in #544 — confirm? You said the layer's base classes "are needed". #544 deliberately does not export ten abstract bases — Protocol, Link, Internet, Transport, Application, Engine, Reassembly, IP_Reassembly, TraceFlow, Field — and flagged that as the one judgement call to overturn.
Recommendation: export the five layer classes (Protocol, Link, Internet, Transport, Application), since those are what a user subclasses and what your Q6 intention makes the supported base; leave the other five for now. If you want all ten, say "all ten".

[JarryShaw]

Q6: now i remember. I deliberately made built-in protocols inherited from base so that they wont be inherited again and being registered back - as a substitute to final class incentive. We may want to revisit this decision since some of the public abstract classes, like HTTP, L2TP, IP are actually designed to be open to inheritance.

Q8: keep ModuleDescriptor, go with your recommendations.

Q9: yea now i rethink about this, we should probably export those so that they can be used for extension and/or the extracted until layer use cases. Your recommendation makes sense now.

[JarryShaw]

All nine answered — thanks. Three measured follow-ups, then where PR (a) stands.

Q6: your final-substitute reasoning is correct, and nothing is actually blocked

Measured on my branch off 691f12ab5. Subclassing the three classes you named registers nothing today:

class MyHTTP(HTTP) -> NO registry change
class MyL2TP(L2TP) -> NO registry change
class MyIP(IP)     -> NO registry change

Same for every foundation built-in (reassembly.IPv4, traceflow.TCP, engines.PCAP, dumpkit.PCAPIO). The reason is the mechanism you remembered: all of them descend from the *Base class, so there is no registering hook anywhere in their MRO. issubclass(HTTP, Protocol) is False and issubclass(HTTP, ProtocolBase) is True, for all five.

So HTTP, L2TP and IP are already open to inheritance and already safe from re-registration. The decision does not need revisiting to unblock them — nothing is blocked.

What is worth revisiting is the cost, and the opt-in guard is a strictly better version of the same idea. Today the final-ness comes from which base a class inherited. That is a property of the parent, so it cannot be varied per subclass: a class is either registering-and-closed or non-registering-and-open, never both. Keying on the keyword instead moves the decision to each subclass, and both properties then hold at once. Measured on the branch:

class Parent(Engine, name='ParentEngine')  -> registered as 'parentengine'
class Derived(Parent)                      -> registers NOTHING     <- the `final` substitute
class DerivedOptIn(Parent, name='Derived') -> registered as 'derived' <- inheritance stays open

Under the old fallback the middle line registered itself as 'derived' — a name nobody chose. So this is a consequence of PR (a) rather than separate work, and I have pinned it as a test (test_registration_is_not_inherited_by_a_subclass). For protocols the same property follows in the registry PR, since Protocol is not in PR (a).

Q9: one of your two justifications does not need the export

"Extension" does. "Extracted until layer" does not — extract(layer=...) accepts a class and immediately reduces it to layer.__layer__ (interface/core.py:154-155), and __layer__ is inherited, so a concrete protocol is already equivalent:

Transport.__layer__ = 'Transport'    TCP.__layer__ = 'Transport'
Link.__layer__      = 'Link'         Ethernet.__layer__ = 'Link'

extract(layer=TCP) and extract(layer=Transport) are the same call today. So the case for exporting rests on extension alone — which I think still carries it, but it is one reason rather than two, and it argues for the narrower list (the five layer classes) rather than all ten. I have flagged the conflict on #544 and linked your answer; the export decision is that PR's, not this one's.

Q3: L2TP, since you have now named it twice

Restating the measurement from my previous comment because it bears on Q6: the only L2TP entry in any dispatch registry is UDP.__proto__[1701] = L2TPv2. TransType.L2TP (=115) exists as a constant and is registered nowhere, so L2TP-over-IP is not dispatched at all — only the UDP encapsulation. And L2TP itself, the v1 base that L2TPv2 derives from, is reachable from no registry. I will file that as its own issue.

PR (a): what it contains

Per Q7, separate PRs. (a) is the four families whose hook already takes a registry-key keyword — Engine (name=), Reassembly (protocol=), TraceFlow (protocol=), Dumper (fmt=). Protocol is held back deliberately: it has no such keyword today, its key is the class name, and giving it one is the subject of Q3's enum-inference design — so it belongs in the registry PR rather than making (a) twice the size. Q1's "all, including Dumper" is satisfied; Dumper is in (a).

Contents: fallback inverted to a guard in all four; unrecognised class keywords now raise UnsupportedCall instead of being swallowed; Dumper's ext= without fmt= likewise; a class-level registry property on EngineMeta, ReassemblyMeta and TraceFlowMeta mirroring EnumMeta.registry (Dumper gets none — it writes two registries, so a single registry name would be ambiguous, and DumperBase has no pcapkit metaclass to hang it on). Plus the two ext.rst examples that relied on the fallback and would otherwise have silently produced unregistered classes, and the name:/protocol: docstring mismatches.

I will post the PR link here when it is up.

[JarryShaw]

PR (a) is up: #547 — the four families whose hook already takes a registry-key keyword (Engine, Reassembly, TraceFlow, Dumper). One commit, unit tier 1045 passed / 8 skipped / exit 0, pylint 25 → 20 messages with no new message ID, mypy unchanged at its 2 pre-existing errors.

Protocol is held back for the registry PR: making it opt-in means inventing a keyword, and what that keyword should mean is Q3's enum-type-inference question. Also deferred, each its own follow-up — unwinding the ~76 *Base as <Public> aliases (Q6, and the guard has to land first or a built-in inheriting Protocol auto-registers under its own class name), the per-class registry ownership fix, and TransType.L2TP being registered nowhere.

[JarryShaw]

Decision record for the Q3 mechanism, so the reason survives and not just the conclusion. The argument is already above in full; restating it in two sentences here because this is the part that gets re-litigated, and adding one consequence that was not in it.

Infer the destination registry from the key's enum type, not from the registering class's parent. The counterexample is the one you raised yourself: L2TPv2 is a Link class that lives in UDP's registry, and that is correct rather than misfiled, because a dispatch registry is keyed by the enclosing layer — L2TP runs inside UDP port 1701 (RFC 2661). Parent-inference would file it under Link.__proto__, where nothing looks, and under a key EtherType does not even define; dispatch would break silently and no test would notice. The explicit destination argument is still kept, but only for raw int ports, where 1701 cannot say whether it means TCP or UDP.

The consequence: #548 should be fixed inside the registry PR, not separately. #548 is that TransType.L2TP (=115) is registered nowhere, so an L2TP-over-IP capture falls through to Raw while the UDP encapsulation masks the gap. Under the mechanism above, the fix is a registration — L2TPv2 declaring TransType.L2TP alongside the port it already answers on, one class naming two enclosing layers. Fixing it first against today's tables would mean writing the entry by hand and then rewriting it when the mechanism lands, so it is cheaper and better-tested as the registry PR's first real customer. It is also the honest test of the design: if the mechanism cannot express "a Link class reachable from both Internet and UDP", it is the wrong mechanism.

So the ordering across the remaining work is: (a) the opt-in guard, merged or awaiting merge as #547 — (b) Protocol plus code= with enum-type inference, absorbing #548 and the per-class registry ownership fix — (c) unwinding the ~76 *Base as <Public> aliases, which cannot precede (a).

[JarryShaw]

#547 was failing the Python 3.10 jobs. Root cause found and fixed, but it surfaced a real API question that is yours rather than mine.

What broke, measured on three interpreters

3.10.21  ABCMeta.__new__(mcls, name, bases, namespace, **kwargs)     <- positional-OR-keyword
3.11.15  ABCMeta.__new__(mcls, name, bases, namespace, /, **kwargs)  <- positional-ONLY
3.14.7   ABCMeta.__new__(mcls, name, bases, namespace, /, **kwargs)

So on 3.10, four class keyword names — mcls, name, bases, namespace — collide with ABCMeta.__new__'s own parameters and raise TypeError before __init_subclass__ runs. From 3.11 they are positional-only and reach the hook.

Two of my tests used name= as their deliberately-wrong keyword, so they were asserting UnsupportedCall against a TypeError the metaclass raised first. Fixed: they use a non-colliding keyword and now run on every version instead of being skipped, with the colliding case pinned per version. One engine test lost its version guard entirely. I also corrected the PR description, which had attributed the pre-existing guard to EngineMeta being Generic — that was my claim and it was wrong; a generic metaclass takes a non-colliding keyword fine on 3.10.

The part that is a real limitation, not a test artefact

Engine's registry keyword is literally name. Verified against the branch on 3.10.21:

class MyEngine(Engine, name='my_engine') -> TypeError: ABCMeta.__new__() got multiple values for argument 'name'
class Typo(Engine, protocol='wrong')     -> UnsupportedCall: unexpected class keyword(s): protocol

This predates the change, but the change makes it load-bearing: before, the cls.name fallback registered the engine anyway, so 3.10 worked by accident. Now the keyword is the only class-definition path, so on 3.10 an engine can only be registered with Extractor.register_engine(...) — which does work on every version. protocol= and fmt= do not collide and are unaffected.

It is not fixable inside __init_subclass__, because the collision happens in the metaclass first. For now I have documented it: a Warning: in Engine.__init_subclass__, a sentence in the 1.5.0 changelog entry, and the reason on the one remaining version skip. #547 is green on 3.10 locally (220 passed, 12 skipped, exit 0 on tests/foundation + tests/dumpkit with CI's ignore globs).

Question

10. Do you want Engine's class keyword renamed so it works on Python 3.10, or is documenting the limitation enough? pyproject.toml says requires-python = ">=3.6, <4" and CI tests 3.10, so 3.10 is genuinely supported.

Three options, in increasing cost:

• (a) Document only — what feat(foundation): make subclass registration opt-in for Engine, Reassembly, TraceFlow and Dumper (#514) #547 does now. 3.10 users register engines with Extractor.register_engine. Nothing breaks on 3.11+.
• (b) Accept a non-colliding alias as well, e.g. engine=, keeping name= working on 3.11+. Backwards compatible, but two spellings for one keyword forever.
• (c) Rename to a non-colliding keyword and keep name= as a 3.11+-only alias. Cleanest end state, and it makes the class-definition path work on every supported version — but it renames a keyword that has existed since 1.3.0.

Recommendation: (a) for #547, and (c) in the registry PR if you want it fixed, because that PR is already introducing code= and is the natural place to settle keyword naming across all five families at once. Saying "(a) is fine" is a complete answer and needs nothing further from you.

One thing I would not do is drop 3.10 support to make this go away — the collision is four names wide, not a general breakage, and everything else on 3.10 is green.

[JarryShaw]

Q10: I'd fix it entirely - avoid the collision anyways.