files=True writes doubled-dot names like "Frame 1..json"

[JarryShaw]

With files=True, every per-frame report lands on disk with two dots before the extension — Frame 1..json, Global Header..txt, Section Header 1..plist.

Cause

The name is built with a literal . while _fext already carries its own leading dot:

# pcapkit/foundation/engines/pcap.py:117
            ofile = ext._ofile(f'{ext._ofnm}/Global Header.{ext._fext}')

# pcapkit/foundation/engines/pcap.py:156
                ofile = ext._ofile(f'{ext._ofnm}/{frnum}.{ext._fext}')

# pcapkit/foundation/engines/pcapng.py:311
            ofile = ext._ofile(f'{ext._ofnm}/{name}.{ext._fext}')

Three more sites share it: pcapkit/foundation/engines/dpkt.py:152, pcapkit/foundation/engines/scapy.py:127, pcapkit/foundation/engines/pyshark.py:137.

_fext comes from Extractor.__output__, whose extensions all include the dot (pcapkit/foundation/extraction.py:169-181):

            'pcap': (ModuleDescriptor('pcapkit.dumpkit', 'PCAPIO'), '.pcap'),
            'plist': (ModuleDescriptor('dictdumper', 'PLIST'), '.plist'),
            'json': (ModuleDescriptor('dictdumper', 'JSON'), '.json'),
            'tree': (ModuleDescriptor('dictdumper', 'Tree'), '.txt'),

read at extraction.py:546 (ext = cls.__output__[fmt][1]), returned as element 3 of make_name, and assigned at extraction.py:704 (self._fext = oext):

Extractor.__output__ extensions:
   json   -> '.json'
   plist  -> '.plist'
   tree   -> '.txt'
   text   -> '.txt'
   pcap   -> '.pcap'
make_name(...) returns oext = '.json'

make_name's own docstring states the opposite at extraction.py:510 — 3. output file extension (without .) — which is presumably where the f'….{ext._fext}' idiom came from. The non-files branch at extraction.py:557 (f'{fout}{ext}') gets it right and adds no dot.

Observed symptom

$ ls -la out
-rw-r--r--  1 user group 2114 Sep 14 00:22 Frame 1..json
-rw-r--r--  1 user group 1721 Sep 14 00:22 Frame 2..json
-rw-r--r--  1 user group  423 Sep 14 00:22 Global Header..json

Both engines, every reachable format:

# PCAP engine, examples/captures/arp.pcap
json   -> ['Frame 1..json', 'Frame 2..json', 'Global Header..json']
plist  -> ['Frame 1..plist', 'Frame 2..plist', 'Global Header..plist']
tree   -> ['Frame 1..txt', 'Frame 2..txt', 'Global Header..txt']

# PCAP-NG engine, examples/captures/test.pcapng
json   -> ['Decryption Secrets 1..json', 'Frame 1..json', 'Frame 2..json', 'Frame 3..json',
           'Frame 4..json', 'Frame 5..json', 'Interface Description 1..json',
           'Interface Description 2..json', 'Interface Statistics 1..json',
           'Name Resolution 1..json', 'Section Header 1..json', 'Section Header 2..json',
           'systemd Journal Export 1..json']
tree   -> ['Decryption Secrets 1..txt', 'Frame 1..txt', ...]
plist  -> ['Decryption Secrets 1..plist', 'Frame 1..plist', ...]

format='pcap' / 'cap' cannot be checked, because it raises before any file is named (TypeError: PCAPIO.__init__() missing 1 required keyword-only argument: 'protocol') — that is a separate problem, not filed here.

Minimal reproduction

Sample captures are not tracked in git; generate them with python examples/generators/make_samples.py first.

import os

import pcapkit

for fmt in ('json', 'plist', 'tree'):
    out = 'out-' + fmt
    pcapkit.extract(fin='examples/captures/arp.pcap', fout=out,
                    format=fmt, files=True, store=False)
    print('%-6s ->' % fmt, sorted(os.listdir(out)))

pcapkit.extract(fin='examples/captures/test.pcapng', fout='out-ng',
                format='json', files=True, store=False)
print('pcapng ->', sorted(os.listdir('out-ng'))[:4], '...')

Output:

json   -> ['Frame 1..json', 'Frame 2..json', 'Global Header..json']
plist  -> ['Frame 1..plist', 'Frame 2..plist', 'Global Header..plist']
tree   -> ['Frame 1..txt', 'Frame 2..txt', 'Global Header..txt']
pcapng -> ['Decryption Secrets 1..json', 'Frame 1..json', 'Frame 2..json', 'Frame 3..json'] ...

What a fix would need to touch

• Drop the literal . from the f-string at pcapkit/foundation/engines/pcap.py:117 and :156, pcapkit/foundation/engines/pcapng.py:311, pcapkit/foundation/engines/dpkt.py:152, pcapkit/foundation/engines/scapy.py:127, pcapkit/foundation/engines/pyshark.py:137.
• Correct the docstring at pcapkit/foundation/extraction.py:510, which claims the returned extension has no dot. It is wrong either way and is what the buggy call sites were written against.
• Two test sites currently depend on the broken shape and should move with the fix:
  • tests/integration/_helpers.py:127-139 — report_stems splits at the first dot on purpose "to stay neutral about how many dots there are, rather than pinning that defect", and cites these exact lines. Once fixed it can assert the whole filename.
  • tests/foundation/engines/test_runtime_engines.py:42 — the fake extractor sets '_fext': 'json' without the dot, which is why assertEqual(sink.paths[-1], 'out/Frame 1.json') at :137, :274 and :368 passes and never caught this. That fixture contradicts the real Extractor._fext ('.json') and needs correcting, or those assertions stay vacuous after the fix.

Observed vs inferred

• Observed by running code: every filename listing above, the real ls -la, and make_name(...) returning '.json'.
• Inferred by reading code: that dpkt.py:152, scapy.py:127 and pyshark.py:137 share the defect — the expressions are identical but those engines need optional third-party packages that were not installed — and that the test_runtime_engines.py fixture is the reason the existing unit test misses it.

[JarryShaw]

Fixed by #403, verified on merged main at 89201dff6:

files: ['Frame 1.json', 'Frame 2.json', 'Frame 3.json']
any doubled dot? False

Root cause was a contract mismatch rather than a formatting slip. Extractor.__output__ spells extensions with the dot ('.json'), while make_name's own docstring promises them without — and all seven consumers compose f'{name}.{ext._fext}', supplying their own. The existing engine tests already set _fext: 'json' bare, so the dotted value was the minority spelling.

make_name now honours its documented contract and adds the dot itself in the non-files branch, so no engine module needed changing. PCAP-NG shared the idiom and is fixed by the same normalisation. TraceFlow keeps a separate __output__ and composes without a literal dot, so it still wants the dotted form and was deliberately left alone.

Closing manually: #403 said Closes #357, #358, #361, but GitHub only auto-closes the first issue in a comma list.