design: is AppType's multi-bit proto necessary, given the enum is already split per transport?

[JarryShaw]

Is this a bug or a feature request? A design question, raised by the maintainer.

The question, verbatim:

one question tho, should we maintain full transport list in each AppType's enum? like for TCP.http on port 80, the current enum still carries the other transport info like UDP/TSCP/etc. is that necessary? if so, then we dont care. if not, we should then bring back the TransportProtocol retyping.

What it decides. #732's ruling was conditional — "proceed with retyping on TransportProtocol - if we no longer need it to be a Flag. But, if in any case we still prefer using a Flag, then we discard the retyping." This issue is that condition. It exists as its own issue so the reasoning is tracked rather than buried in #732, whose subject is the str-valued/port=-1 shape.

State today. AppType is split into four per-transport registries — TCP 6,147 / UDP 6,143 / SCTP 91 / DCCP 10 = 12,391 members — reached via AppType.__registries__. A member also carries a proto attribute holding a TransportProtocol Flag, which for a service IANA registers on several transports is multi-bit: TCP.http.proto names UDP too. 10,625 of 12,391 rows (85.7%) are multi-bit, so this is most of the enum, not an edge.

Two arguments that the list is redundant:

1. The split already encodes it. "Which transports bind port 80" is answerable by asking which of the four registries contain it. The multi-bit proto is then a second copy of a fact the structure already carries — and a second copy can drift from the first.
2. _dispatch already rejects multi-bit. AppType.get(80, TCP|UDP) raises, telling the caller to look one transport up at a time. So the entry point insists on a single transport while the members it returns advertise four.

The one site that decides it is pcapkit/foundation/registry/protocols.py:846, if test not in proto: — the only read known to treat proto as a set rather than a label. Load-bearing if it needs the other bits; decoration if it is only reachable once a registry has been selected.

Being measured now, before any answer — a wrong one costs a 12,391-member migration:

• Every read of proto across pcapkit/, tests/, docs/: is multi-bit ever the only source of a fact?
• Derivability: for every port present in more than one registry, does "the set of registries containing it" equal "its proto bit set"? Any disagreement means the two copies have already drifted — a defect independent of this design question.

Findings get appended here as they land. Related: #732, #775, #783.

[JarryShaw]

Measured: the multi-bit proto is fully derivable from the four registries. So no — the full transport list is not necessary.

Keyed on (service name, port), 7,054 groups, 0 disagreements, checked in both directions:

• No member's proto omits its own registry — 0 across all four.
• No single-registry group carries a multi-bit proto — 0. This is the direction the obvious test misses, and it is what closes the argument: proto never claims a transport the registries do not.
• Arithmetic cross-check falls out exactly. Popcount distribution over all 12,391 members is 1→1766, 2→10486, 3→123, 4→16; group-span distribution is 1→1766, 2→5243, 3→41, 4→4; 5243×2=10486, 41×3=123, 4×4=16. Every member's popcount equals the number of registries its (svc, port) occupies.

And the generator makes it true by construction — pcapkit/vendor/reg/apptype/apptype.py:806:

if self.TRANSPORT not in record.protos or record.port == '-1': continue

A record enters registry X iff X ∈ record.protos, so registry-set and proto-set are the same set. The multi-bit value is a second copy of a fact the split already carries.

Per-registry: TCP 6147 (5287 multi / 860 single), UDP 6143 (5276 / 867), SCTP 91 (52 / 39), DCCP 10 (10 / 0). Empty proto: 0 — no third case.

But retyping is not a cheap attribute change, and this is the part worth deciding on. proto.name is baked into each member's underlying str value, not merely its display — vendor/reg/apptype/apptype.py:251, :271, :274:

temp = '%s [%d - %s]' % (name, value, proto.name)      # __new__ -> obj._value_
return "<%s.%s: %d [%s]>" % (..., self.proto.name)     # __repr__
return '%s [%d - %s]' % (self.svc, self.port, self.proto.name)   # __str__

So TCP['http'].value is 'http [80 - tcp|udp|sctp]' today and would become 'http [80 - tcp]'. That changes the value, repr and str of 10,625 members — a public-contract change deserving breaking, not a refactor. Identity is safe either way: pickle and both copies all return the same object.

Still outstanding: pcapkit/foundation/registry/protocols.py:833 does proto = code.proto and :846 does if test not in proto: — a real consumer that receives the whole set from one member. The read audit establishing whether it needs the set is still running; that is the last input before I can say the retyping is safe rather than merely lossless.

[JarryShaw]

Answer: the multi-bit list is not necessary, and my recommendation is to retype. It is load-bearing at exactly one site, and that site is a convenience default that contradicts the rest of the library.

The one site. register_apptype(member, module) called without an explicit proto=. pcapkit/foundation/registry/protocols.py:833 does proto = code.proto, and the loop at :846 fans registration out across a hardcoded {tcp: TCP, udp: UDP} dict — not __registries__, and SCTP/DCCP are deliberately excluded. I verified it myself on 477ed00c4:

member: <TCP.http: 80 [tcp|udp|sctp]> proto = 7
TCP.__proto__[80] changed: True -> Dummy      (displaced httpv1.HTTP)
UDP.__proto__[80] changed: True -> Dummy      (displaced http.HTTP)
--- single-bit contrast: <TCP.imap: 143 [tcp]> ---
TCP.__proto__[143] changed: True
UDP.__proto__[143] changed: False

Two RegistryWarnings fired, one per registry. That is the whole of the multi-bit value's job.

Everywhere else the composite is already refused, because it gave wrong answers. Per #759, feeding a member's own .proto back into AppType.get() silently returned the wrong service in 46 of 10,625 cases — including port 888 accessbuilder→cddbp and port 999 puprouter→garcon, services that do not exist on that transport. _dispatch/get/get_all now raise ProtocolError on any composite, and tests/const/test_const_apptype_split_unit.py:709-836 AST-sweeps the library to prove no lookup site ever builds one.

So the library is internally contradictory today: hand a member to register_apptype and its composite proto is trusted to fan out; hand the same member's .proto to get() and you get a ProtocolError telling you to look one transport up at a time. Retyping removes the contradiction in favour of the half that was arrived at by fixing real bugs.

What retyping costs — this is what needs your ruling:

1. breaking, for 10,625 members. proto.name is folded into _value_ at vendor/reg/apptype/apptype.py:251, so TCP['http'].value goes from 'http [80 - tcp|udp|sctp]' to 'http [80 - tcp]'. repr and str change with it. Identity survives pickle and both copies.
2. register_apptype's no-proto= default loses the fan-out. Either require an explicit proto= from the caller — consistent with _dispatch's "one transport at a time" — or rewrite it to look the port up across __registries__, which is not what it does now. It is documented public API, so external callers may rely on the current default.
3. Four test sites and the docstrings pin the present behaviour and must change with it: test_const_apptype_split_unit.py:78-80, :630-658, :709-836, tests/foundation/registry/test_protocols.py:421-499, plus vendor/reg/apptype/apptype.py:522-526 and the same text generated into all four const files — "Members carry the whole transport protocol set IANA assigned the service."

My recommendation: retype, and make register_apptype require an explicit proto= when the fan-out is wanted. It trades a silent convenience for an explicit argument, and it aligns the registration path with the lookup path that was already corrected. Labelling needs: decision — points 1 and 2 are yours, not mine.

[JarryShaw]

I feel like now that we're already using transport specific enums, we should avoid ambiguity from the enums in all sense. so... yes, go with your recommendation.

one note tho, the Flag type might be useful when doing register_apptype but its not used in other places. so we might actually just make the function signature as register_apptype(port: int, name: str, *transport: TransportProtocol) so that we can supply a list of transport without having to maintain a Flag for it and keeping everything singular and explicit.

[JarryShaw]

and maybe allow aliasing, like register_apptype(port: int, name: str, *transport: TransportProtocol, aliases: list[str] = None), where aliases is an optional keyword argument using name as the canonical name with aliases for the port's aliases like port 80's.

[JarryShaw]

Ruling recorded, in your words:

I feel like now that we're already using transport specific enums, we should avoid ambiguity from the enums in all sense. so... yes, go with your recommendation.

one note tho, the Flag type might be useful when doing register_apptype but its not used in other places. so we might actually just make the function signature as register_apptype(port: int, name: str, *transport: TransportProtocol) so that we can supply a list of transport without having to maintain a Flag for it and keeping everything singular and explicit.

Retyping is accepted. proto becomes the registry's own single transport; the composite goes.

The varargs idea is the right shape and I agree with the reasoning — "singular and explicit" is exactly what removes the ambiguity, and it beats my own suggestion of a required proto= kwarg, because a kwarg taking a composite keeps the Flag alive for one caller's benefit. Varargs kill it properly.

But your sketch drops three things the current signature carries, so I want the caller inventory before I write it. Today it is:

def register_apptype(code: 'int | Enum_AppType', module: 'str | ModuleDescriptor[ProtocolBase] | Type[ProtocolBase]',
                     class_: 'str' = NULL, *, proto: 'TransportProtocol | str' = NULL) -> 'None':

1. code accepts an Enum_AppType member, not just an int. port: int removes that, which is a larger break than the retyping itself.
2. module accepts a ModuleDescriptor or a class object, not just a str; class_ is the second half of the string form. A single name: str cannot express either.
3. *transport would swallow class_, since class_ is currently positional — so existing three-positional-argument calls would silently bind the class name as a transport.

None of that argues against varargs; it argues that the transport part wants *transport while code/module/class_ need deciding separately. I am measuring every in-repo caller and every place TransportProtocol's Flag-ness is actually relied on (show_flag_values, in tests, | construction) before proposing the exact line — your "not used in other places" is the claim I most want to confirm or refute, since if it holds, TransportProtocol can stop being a Flag entirely and that is a much cleaner end state.

wip. I will post the concrete signature plus the migration plan here, then split the implementation into its own issues.

[JarryShaw]

my scatches are just POC not how it should actually look like - just using as a refernce/example of the idea.

[JarryShaw]

Both recorded, in your words:

and maybe allow aliasing, like register_apptype(port: int, name: str, *transport: TransportProtocol, aliases: list[str] = None), where aliases is an optional keyword argument using name as the canonical name with aliases for the port's aliases like port 80's.

my scatches are just POC not how it should actually look like - just using as a refernce/example of the idea.

Understood, and my three objections were aimed at the wrong target then — withdrawn as written. I read the sketch as a literal signature; the idea is singular-and-explicit transports plus canonical-name-with-aliases, and both survive whatever the final parameter list turns out to be. The audit still runs, because "match the sibling register_*" is what should decide the exact line, not me.

Aliasing is real, and the numbers are small: 44 ports carry more than one member — 23 TCP (49 members), 21 UDP (45), 0 SCTP, 0 DCCP. Port 80 is exactly your example:

TCP port 80:  'http'      value='http [80 - tcp|udp|sctp]'   proto=7
              'www'       value='www [80 - tcp|udp]'         proto=3
              'www-http'  value='www-http [80 - tcp|udp]'    proto=3

But a flat aliases: list[str] cannot express 9 of those 44 ports, because the members disagree about the transports. http binds SCTP; www and www-http do not. So www is not an alias of http — it is a different registration that happens to share a port. All nine:

TCP   80 [http 7, www 3, www-http 3]   113 [ident 1, auth 3]      631 [ipp 3, ipps 1]
     888 [accessbuilder 3, cddbp 1]    999 [garcon 1, puprouter 3]  2049 [shilp 3, nfs 7]
UDP   80 (same three)                  999 [applix 2, puprouter 3]  2049 [shilp 3, nfs 7]

And this is the same defect as the retyping, seen from the other side. Ports 888 and 999 are precisely the two #759 cases where a composite-proto lookup returned a service that does not exist on that transport — accessbuilder→cddbp, puprouter→garcon. The wrong answer came from picking the wrong member of an aliased pair whose proto sets disagree. Retyping to single-bit removes the ambiguity within a registry; aliasing removes it across names on one port. Doing one without the other leaves half of #759's mechanism standing.

So the alias relation needs a per-alias transport set, or aliases must be confined to the 35 ports where every member already agrees. I will fold this into the plan I post here rather than answer it off the cuff.