[davidbudzynski]

Summary

Implements the three opportunities in #107.

vswitch/nswitch, >8 pairs
  values ──► open-addressed value → last-pair-index map
  x      ──► block lookup ──► output writer (atomic parallel, string/list serial)

funique/fduplicated/uniqLen
  rows ──► 64-bit rolling hash ──► duplicate table
                                      └─ high-cardinality sample pre-sizes it

• vswitch()/nswitch() keep the value-major path for ≤8 pairs. Above that, they build the map once and scan x in cache-sized blocks. Last-duplicate-wins, NA/NaN, -0, Inf, complex, encoding, factor/Date and output recycling are unchanged.
• Duplicate tables use a 64-bit rolling row hash instead of mixing every cell plus a second mix. A bounded random row sample pre-sizes the table only for large, genuinely high-cardinality inputs; low-cardinality inputs keep the grow-on-demand table from perf(c): profile and optimize hot-path kernels #74.
• tools/benchmark-core.R is manual only: 2m/10m elements, 5/20/100 values, integer/string/factor/NA, data frames, 1/2/4 threads, optional base/data.table references.

Evidence

Rscript tools/benchmark-core.R <library> <label> 3; macOS, R 4.6.2, OpenMP disabled for these runs, medians in seconds.

Case	0.0.23	this PR	Speedup
vswitch int, 20 values, 10m	0.059	0.024	2.5x
vswitch str, 20 values, 10m	0.066	0.033	2.0x
vswitch int, 100 values, 10m	0.283	0.050	5.7x
vswitch str, 100 values, 10m	0.275	0.059	4.7x
nswitch int, 100 pairs, 10m	0.283	0.050	5.7x
nswitch str, 100 pairs, 10m	0.281	0.059	4.8x
uniqLen 10m × 2 int df	0.436	0.176	2.5x
funique 10m × 2 int df	0.450	0.180	2.5x
uniqLen 10m mixed df	0.826	0.331	2.5x
funique 10m mixed df	0.853	0.373	2.3x

2m data frames improve 1.5–1.8x. Five-value switches and scalar random-int vectors are unchanged; the dedicated scalar-hash variants stayed under the 20% stop threshold. vswitch string 100 values on 10m is 0.059 s vs data.table::chmatch 0.029 s (chmatch is a first-match lookup, not a full switch).

Verification

• R CMD check --no-manual: Status: OK.
• Full tinytest suite passes; overall coverage 96.78%.
• Differential fuzz against an independent R reference: 8,592 cases, 0 hash-path-specific mismatches (types, NA/NaN/-0/Inf, encodings, duplicates, scalar/vector defaults, outputs, block boundaries).
• OpenMP build (libomp) tested with nThread = 4 on ALTREP x/defaults; serial and parallel results match.
• Red/blue review found one critical issue: ALTREP accessors and R output pointers were being touched inside the OpenMP region, which can corrupt R's heap. All pointers are now materialised on the main thread before the parallel loop.

Pre-existing issues fixed here

These were found while fuzzing and are present on master too. They are now fixed in this PR with regression tests:

1. enc2UTF8() mixed-encoding bug. It only checked element 1 to decide whether to convert, leaving later latin1 elements unconverted and turning NA_STRING into the literal "NA". It now checks every element and preserves NA_STRING. Value-major nswitch() also compares each pair against its own converted value instead of vans[0].

2. Real row/vector hash vs REQUAL. Row and vector hashes used raw double bits while equality uses REQUAL, so -0 vs 0 (and differing NaN payloads) could be reported as distinct by funique()/uniqLen() where base R treats them equal. Double hashes now canonicalise -0/0 and NaN payloads while keeping NA distinct, across vector, matrix and data-frame paths.

Merge danger

Door: two-way — performance-only, no API, output format or data-layout changes.

Blast radius: switch functions and data-frame/matrix unique paths; low. Version bumped to 0.0.24 with NEWS notes.

Not covered here: 32-bit builds, >50 switch pairs / >8k element stress, and no PR timing assertions in CI.

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.92%. Comparing base (99413b6) to head (8db8de9).
⚠️ Report is 140 commits behind head on master.
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #108      +/-   ##
==========================================
- Coverage   98.01%   96.92%   -1.09%     
==========================================
  Files          13       15       +2     
  Lines        5083     5338     +255     
==========================================
+ Hits         4982     5174     +192     
- Misses        101      164      +63     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[davidbudzynski]

Follow-up: encoding and double-equality fixes

Picked up the two pre-existing issues from the PR body (and the related value-major nswitch() bug). Pushed as 8db8de9.

Changes

• enc2UTF8() now checks every element instead of only element 1, converts later latin1 values, and preserves NA_STRING.
• Value-major nswitch() now compares each pair against its own converted value instead of always vans[0].
• Double hashes canonicalise -0/0 and all NaN payloads while keeping NA distinct, so vector, matrix and data-frame funique()/fduplicated()/uniqLen() now match base R's REQUAL behaviour.
• Added regression tests in test-enc.R, test-switch-hash.R and test-unique.R; updated NEWS.md.

Performance check

The first version called R_IsNA()/R_IsNaN() per double and doubled the double-frame uniqLen() time (0.16s -> 0.31s on 10m x 2). Reworked it to use the inlined ISNAN() and only call R_IsNA() for actual NaNs.

Medians below: 10m rows, 15 reps, 1 thread, macOS R 4.6.2, comparing the previous PR head (accaecf) with this commit (8db8de9).

Case	previous PR head	8db8de9	change
uniqLen 10m x 2 double df	0.159s	0.165s	~4% slower
funique 10m x 2 double df	0.181s	0.179s	same
uniqLen 10m mixed int/double df	0.328s	0.256s	~22% faster
funique 10m mixed int/double df	0.368s	0.299s	~19% faster
uniqLen 10m double vector	0.378s	0.325s	~14% faster
funique 10m double vector	0.387s	0.345s	~11% faster
fduplicated 10m double vector	0.379s	0.331s	~13% faster

Scalar double paths are faster because the old code called two R API functions per element. The double-frame row hash pays about 4% for the per-element special-value check, which still leaves the PR's speedup over 0.0.23 intact (uniqLen was 0.436s there).

vswitch/nswitch with 100 int/string values and the int-frame uniqLen/funique cases measured unchanged; those paths were not touched.

Verification

• Full tinytest suite: 1675 passed, 0 failed.
• R CMD check --no-manual: Status: OK.