Store known span tags densely by tag-id in TagMap (concept draft)#11659
Closed
dougqh wants to merge 35 commits into
Closed
Store known span tags densely by tag-id in TagMap (concept draft)#11659dougqh wants to merge 35 commits into
dougqh wants to merge 35 commits into
Conversation
Experimental branch for the known-tag tagId routing idea: KnownTags maps a 64-bit
tagId ([63-48 globalSerial][47-32 fieldPos][31-0 nameHash]) to tag names via a
registered Resolver, so TagMap can route known tags by slot instead of hashing strings.
INCOMPLETE / DOES NOT COMPILE YET. These files depend on TagMap.java changes that were
lost from the working tree (uncommitted, never committed on any branch, not stashed, not
in the TracerProto prototype):
- TagMap.Entry.tagId field (read as entry.tagId / status.tagId in TagMapTagIdTest)
- TagMap.set(long tagId, String) / ledger().set(long, ...) overload (used by the test
and TagMapInsertionBenchmark via readMap.set(IDS[i], VALUES[i]))
KnownTags.java itself is self-contained. To make this branch build, the TagMap.Entry.tagId
field and the set-by-id ledger path must be reconstructed (see KnownTags' bit-layout doc
and TagMapTagIdTest for the expected API).
Based on master to keep the experiment independent of the CSS v1.3.0 stack.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Restores the load-bearing TagMap.java changes that were lost from the working tree
(session 94f3bac2): set(long tagId,...) overload family, getEntry(long), tagId-encoded
Entry factories, knownEntries slot routing in getAndSet with occupant first-writer-wins +
collidedSlots bitmask, and setKnown/setInBuckets extraction. Also recovers the matching
TagMapFuzzTest.java setById/putAllLedgerById coverage and the internal-api jmh property
config (jmhInclude/jmhWarmup/jmhIterations/jmhFork) for the insertion benchmarks.
Extracted from stash@{0} (602e6c47, "WIP on css-ring-buffer-v2") which bundled this work
alongside unrelated CSS ring-buffer changes; only the TagMap-related files were taken.
Combined with KnownTags.java + TagMapTagIdTest.java + the two insertion benchmarks already
on this branch, the integration compiles (main+test+jmh).
KNOWN WIP GAP: TagMapTest.fromMapImmutable_empty NPEs — a slot-aware op dereferences the
lazily-allocated knownEntries array on an empty/immutable map without a null guard.
TagMapTagIdTest and TagMapFuzzTest pass.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
TagMap.EMPTY (interface field, computed via the factory) could capture null when OptimizedTagMap initialized first: TagMap.<clinit> runs during OptimizedTagMap.<clinit> before its static fields are assigned, so the factory returned the not-yet-set OptimizedTagMap.EMPTY. Empty-ledger buildImmutable() and fromMapImmutable(empty) then returned null -> NPE (flaky by class-load order). Fixed with an initialization-on-demand holder (OptimizedTagMap.empty() -> EmptyHolder.EMPTY) so the empty instance initializes independently of order. Also make TagMapFuzzTest reproducible: -Ddatadog.tagmap.fuzz.seed= (random and logged when unset) + a SeedReporter TestWatcher that prints the seed and a reproduce command on failure, plus -Ddatadog.tagmap.fuzz.iterations= to run many sequences per JVM for hunting rare cases. This combination caught the EMPTY bug deterministically. TagMapEmptyInitTest guards the init order. The set(long)/getEntry(long) methods are now abstract (with LegacyTagMap impls) rather than default — explicit per implementation. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Tag-id-constructed entries resolve their name lazily from the tagId via KnownTags on first tag()/getKey(), caching into the non-volatile `tag` field — a benign race. Run tag-id entries (Object/int/boolean) plus matches() through the existing shuffled multi-threaded harness so 4 threads resolve concurrently; assert all agree on the same interned constant and that hash() equals the tagId's nameHash. Also stress a string entry's lazy hash() now that it writes into the low 32 bits of `tagId` (formerly a separate int lazyTagHash). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The tagId bit-packing was duplicated in four places (fuzz/unit/Entry tests and the insertion benchmark, which even hand-rolled the name hash). Add a single KnownTags.tagId(globalSerial, fieldPos, name) factory — the inverse of the existing globalSerial/fieldPos/nameHash extractors — that computes nameHash via the runtime's TagMap.Entry._hash so the low 32 bits always match Entry.hash(). Intended for the code generator and tests. Route all callers through it and add an encoder/decoder round-trip test. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Establishes the boundary FIRST_STORED_SERIAL=256: globalSerials [1,256) are reserved for "virtual" tags that are specially handled (redirected to span fields or processed by the tag interceptor) and NOT stored in the TagMap — hand-assigned in tracer core; [256,..) are generated convention tags that ARE stored (slotted/bucketed); 0 stays unknown/string-only. Adds isReserved()/ isStored() so setTag(long) can classify a tag by an O(1) range check (its "needsIntercept by id") before routing to the interceptor vs the slot/bucket store. Both core and the code generator agree on this boundary. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… paths Connect the tag-id fast path into the span layer: - CoreTagIds: hand-assigned tag-id constants for core tags + a KnownTags.Resolver that registers on class init (so id resolution is live before the first span). PARENT_ID is a stored tag (serial >= FIRST_STORED_SERIAL); ERROR is a reserved virtual tag (serial < FIRST_STORED_SERIAL, fieldPos sentinel so it never slots). - DDSpanContext.setTag(long, Object): O(1) range-check routing — reserved tags go to the interceptor via id dispatch, stored tags go straight to the map by id (slot/bucket), bypassing the per-tag interceptor string switch. - TagInterceptor.interceptTag(span, long, value): int-switch on globalSerial (ERROR), falling back to the string path by resolved name for other reserved ids. - Migrate the constructor's PARENT_ID set to the id; drop the now-unused import. Tests: PARENT_ID set-by-id is findable/serialized as _dd.parent_id; ERROR set-by-id sets the error flag and is not stored. Existing DDSpanContext/serialization/tracer/ interceptor suites pass with the resolver now registered tracer-wide. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Move the lazy tagId->name resolution into the base EntryChange.tag() (final) and drop Entry's override, so EntryRemoval resolves its name from a tagId too. - EntryChange.newRemoval(long) / EntryRemoval(long) carry a tag id. - TagMap.remove(long)/getAndRemove(long): OptimizedTagMap clears the slot by id (knownRemove) then falls back to the resolved-name bucket lookup; LegacyTagMap resolves the name and delegates. - Ledger.remove(long) records an id-keyed removal; fill() replays id removals via map.remove(long) (slot-aware, no name round-trip), string removals by name. Tests: unit coverage for remove/getAndRemove by id (slot clear, prior value, string-set-then-remove-by-id, ledger remove-by-id) plus a fuzz removeById action woven into the random mix (exercises slot-clear + collided-slot reclaim); ~48k seeded sequences clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Handle the per-span "common" tags (base.service / version) via the tag-id fast path. These values are fixed for the tracer's life, so build their TagMap.Entry once and share across every span (Entry is immutable + safe to share) — dropping InternalTagsAdder's per-span Entry allocation to zero (cf. PR #11555, the string-keyed precursor), and making the entries tag-id-bearing so they also land in their positional slot. - TagMap.Entry.create(long, Object)/create(long, CharSequence): tag-id keyed, null/empty-rejecting factories mirroring the String create(). - CoreTagIds.BASE_SERVICE / VERSION (stored range) + resolver entries. - InternalTagsAdder prebuilds baseServiceEntry/versionEntry in its ctor and set()s the shared entry; empty DD_SERVICE early-returns (regression test added). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… slot layout env and the product-mixin flags _dd.djm.enabled / _dd.dsm.enabled are build-time-known constant tags merged into defaultSpanTags (CoreTracer. withTracerTags). Hand-assign tag ids for them (stored range, CoreTagIds) so they occupy the shared global slot layout: defaultSpanTags slots them on build, and they merge into each span's slots via the existing slot-aware merge — sharing entries, no per-span placement, no common prototype / construction change. Runtime-configured user tags keep no id and ride in the buckets, per the rule that only agent-build-time-known tags get slots. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The common-layout + fast-merge approach doesn't need a per-map prototype, so remove the now-unused scaffolding (Prototype, createKnownEntries, TagMap.create(Prototype), OptimizedTagMap(Prototype)) — recoverable from history when template-stamping is revisited. Replace the hardcoded KNOWN_ENTRIES_CAPACITY=32 with the registered provider's slot count: KnownTags.Resolver now declares slotCount() (= max stored fieldPos + 1), captured once at registration as a dynamic constant (KnownTags.slotCount()), and OptimizedTagMap sizes knownEntries to exactly that. CoreTagIds reports 6 (its stored tags occupy fieldPos 0..5); reserved tags keep their out-of-range sentinel and never slot. Resolvers in the tests/benchmark declare their own slot counts. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Migrate the post-processors that set non-intercepted, stored common tags to id-keyed writes (the lower-friction interceptor surface — they operate on the TagMap directly, no AgentSpan/MutableSpan change): - RemoteHostnameAdder: _dd.tracer_host (its cached shared Entry is now id-bearing) - IntegrationAdder: _dd.integration - ServiceNameSourceAdder: _dd.svc_src Hand-assign their ids in CoreTagIds (stored range, fieldPos 6..8; SLOT_COUNT 9) + resolver entries. These tags now occupy the shared slot layout; since they're id'd, any string set of the same tag also slots via keyOf (unification). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… slots Migrate the two remaining stored-tag post-processors to id-keyed access: - PeerServiceCalculator: reads peer.service via getEntry(id); writes peer.service and _dd.peer.service.remapped_from via set(long,...) (was Map put which bypassed the interceptor anyway — recalculation, no behavior change). - HttpEndpointPostProcessor: reads http.method/http.route/http.url via getEntry(id). Hand-assign their ids in CoreTagIds (stored range, fieldPos 9..13; SLOT_COUNT 14) + resolver entries. peer.service / http.method / http.url are intercepted-but- stored: the string set-path still runs the interceptor side-effect then slots via keyOf, so these id reads find the same entry. http.route is not intercepted. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ored tags Codify the "id but no fast slot" tier: a tag can carry a stable id (so keyOf/ nameOf unify it with its string form) while deliberately not owning a positional slot, so it lives in the hash buckets and doesn't widen knownEntries[] for every span. This is how narrow/low-priority tags get ids without slot bloat. - KnownTags.NO_SLOT (0xFFFF): canonical out-of-slot-range fieldPos sentinel. The existing routing already buckets any fieldPos >= slotCount() (setKnown/knownGet/ knownRemove), so no engine change is needed — only a named encoding. - KnownTags.tagId(serial, name): overload stamping NO_SLOT. - KnownTags.isUnslotted(tagId): stored serial + NO_SLOT. - CoreTagIds: drop the local RESERVED_FIELD_POS constant; ERROR now uses the tagId(serial, name) overload. No tag reassignments. - TagMapTagIdTest: unslotted-tier coverage (set/get/remove by id + string, NO_SLOT survives on the stored entry, unification both directions). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Wire the tag-id fast-path up through the span layer and migrate the first real decorator, so we can measure the id+slot path on a real span. - AgentSpan: new default setTag(long tagId, Object) that resolves the id to its name and delegates to setTag(String, Object) — zero blast radius for the dozen other AgentSpan implementors (OTSpan/OtelSpan/Spark/etc). DDSpan overrides it to take the fast-path via the already-present DDSpanContext.setTag(long, Object). - Relocate the hand-assigned tag-id registry from dd-trace-core CoreTagIds to internal-api as KnownTagIds, so both core AND instrumentation (decorators, which only see internal-api) reference one registry — the single source of truth the eventual codegen will replace. Updated all core/test references. - KnownTagIds: slot peer.hostname/peer.ipv4/peer.ipv6 (non-intercepted, common on client/producer spans), SLOT_COUNT 17. - BaseDecorator.onPeerConnection: set peer.hostname/ipv4/ipv6 by id. peer.port left on the string path (int overload; deferred). Updated the inherited Spock onPeerConnection expectations (minimal Groovy edit, per decision; full groovy->java migration of the decorator test hierarchy deferred). - PeerConnectionBenchmark (jmh): measures onPeerConnection on a real DDSpan for the string-vs-id A/B. tag: ai generated tag: no release note Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Mirror the String-keyed primitive setters on the tag-id surface so numeric/boolean
tags id-key without boxing:
- AgentSpan: default setTag(long, {CharSequence,boolean,int,long,float,double})
resolving name -> string path (zero blast radius); DDSpan overrides each to
context.setTag(tagId, value).
- DDSpanContext: typed setTag(long, ...) routing like setTag(long, Object) —
reserved -> interceptor (boxes only on that rare path), else store by id (no box).
- KnownTagIds: slot peer.port (SLOT_COUNT 18).
- BaseDecorator.setPeerPort(int/String) now id-keys peer.port; updated the
inherited Spock PEER_PORT expectations across the decorator test hierarchy.
All peer.* tags are non-intercepted (case b), so this preserves behavior.
tag: ai generated
tag: no release note
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…n the id Close the latent regression in the id set-path: setTag(long) previously only routed reserved serials to the interceptor, so id-setting an intercepted-but- stored tag (http.method/url, peer.service) would silently skip its side-effect. - KnownTags: encode an INTERCEPTED flag in the tagId sign bit (bit 63), so the check is a single `tagId < 0` (isIntercepted) — fast, matching "most sets are by id". globalSerial now masks to 15 bits. Helper KnownTags.intercepted(id). - KnownTagIds: flag the intercepted ids (ERROR, HTTP_METHOD, HTTP_URL, PEER_SERVICE); leave non-intercepted ids (peer.*, base.service, http.route, …) clear so they keep the fast store path. - DDSpanContext.setTag(long, …): 3-case routing — (a) reserved + (c) intercepted- stored -> interceptor (then store if not handled); (b) non-intercepted -> store by id directly. - TagInterceptor.interceptTag(long): dispatched on serial; specialized cases for hot tags (ERROR), default resolves the name and runs the proven string interception, so behavior matches the string set-path exactly. - Tests: reflective consistency guard (every id's INTERCEPTED bit must agree with needsIntercept(name)) + behavioral proof that id-setting peer.service runs the interceptor side-effect and stores. tag: ai generated tag: no release note Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The id interceptor dispatch previously resolved nameOf -> string switch for every intercepted tag except ERROR. Give the hot tags dedicated serial-keyed arms so the id path is fully string-free (no name resolution, no string switch): - HTTP_METHOD_SERIAL / HTTP_URL_SERIAL: the serial already distinguishes the two, so the url-as-resource rule is invoked with the known name constant directly. - PEER_SERVICE_SERIAL: mirrors the Tags.PEER_SERVICE string arm (sets the peer.service source, then interceptServiceName). Other intercepted ids still fall back to the name path, so behavior is unchanged. Test: urlAsResourceNameRuleViaTagId drives the http.method/url arms end-to-end and asserts the same resource name as the string path. tag: ai generated tag: no release note Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The two highest-volume server-span tags now set by id. Both are intercepted-but- stored (case c): setTag(long) routes them through the (now specialized) id interceptor arm, which runs the url-as-resource rule and stores them in their slots — identical behavior to the string path, verified by urlAsResourceNameRuleViaTagId. Scope note: http.status is not a setTag (it's span.setHttpStatusCode, already a dedicated fast field); span.kind uses a cached TagMap.Entry + setSpanKindOrdinal fast field and is deferred (needs id-aware Entry-path interception). Updated the HttpServerDecoratorTest Spock HTTP_METHOD/HTTP_URL expectations. tag: ai generated tag: no release note Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Petclinic span/tag capture shows the macro levers are component + span.kind (every span) and db.type/instance/user/operation/pool.name (58% of spans, JDBC) — none yet slotted. Register them (plus language) as slotted ids so the existing string/cached-Entry decorator sets are upgraded into positional slots via keyOf on store — no decorator or test changes needed. span.kind flagged INTERCEPTED (the consistency guard enforces it). SLOT_COUNT 18 -> 26. This is the registration-only step (slot storage benefit); decorator-level id set(long) migration can follow if the macro signal warrants. tag: ai generated tag: no release note Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ft spec) Language-agnostic declaration spec for the AttributeValueTable / codegen work: - structural span types compose via `extends` (multiple parents allowed); the `base` root holds common tags (incl. error, process-constants) implicitly in every span. peer.* lives in a `client` abstract layer (open question, noted). - product/enrichment mixins (profiling, dsm, appsec, ci_visibility) compose on the side via `applies: all | [types]`, gated by `enabled_by`. - tag fields carry logical type + OTel `aliases`; tracer-impl hints (slot, intercepted, source) are marked separately for the cross-language split. Reconciled from the TracerProto OTel-convention hierarchy + the tags PetClinic actually emits. Draft input for the design doc; not wired into the build. tag: no release note Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Storage (typed arrays: byte[] types / long[] prims / Object[] objs by slot, no per-tag Entry), write/read paths, the no-Entry serialize cursor (the real alloc win), API-compat plan, and how product mixins interact with the fixed layout (unslotted vs composed-at-registration vs per-span-type). Measurement: standalone JMH first (vs OptimizedTagMap, -prof gc), then integrate + petclinic A/B. tag: no release note Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…lver-driven - AttributeValueTable is an interface; array/segment-backed impl ships first, a codegen POJO-per-span-type impl can replace it later (same opaque contract). - set(long,..)->boolean: false on no-slot OR type-mismatch -> caller buckets (Entry). - reads route through get(long)->EntryReader (flyweight, EntryReadingHelper pattern; coercion via TagValueConversions; materialize via existing EntryReader.entry()) — no separate typed getters, no bespoke visitor; reuses the Iterable<EntryReader> serialize path unchanged. - drop the separate Layout abstraction — consult KnownTags.Resolver directly, extended with typeOf (type-reject) + tagIdAt (iteration). static per-slot type => no per-span type array. - product mixins = lazily-allocated segments (fieldPos = [segment][offset]). tag: no release note Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Honest accounting: write path + allocation/GC improve; read/serialize carries some intrinsic extra CPU per tag for a generic layout-driven store (flyweight + array read + name resolve + coercion) that only POJOs (generated fields) fully recover. Net likely neutral-to-positive pre-POJO (cheap frequent writes, single serialize pass, lower GC). Measurement plan upgraded to a three-way JMH incl. a hand-written POJO to confirm the codegen endgame before building the generator. tag: no release note Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace the positional-by-fieldPos + segments/bitmask scheme with a dense association list of only the tags present: long[] ids + Object[] values. - Mixins need no special machinery — a product tag is just another (id,value) pair; the list holds only what's set. Dropped the segment/[segment][offset] scheme entirely. - id is stored => iteration names via nameOf(ids[i]); no fieldPos reverse lookup. Resolver needs only typeOf added (type-reject + reader type()). - Maps directly onto the existing EntryReadingHelper flyweight + TagValueConversions. - Trade-offs: O(n) scan (fine for small spans) and boxing of fresh per-span primitives (status_code/port). Prebuilt primitive entries are NOT a loss — Entry caches its box, so storing objectValue() reuses the shared box (0 per-span alloc). Parallel long[] prims is a deferred hatch if primitive-heavy spans show it. tag: no release note Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A parallel long[] prims adds a whole extra per-span array + per-entry type tracking, costing more than the few small boxes it saves -> rejected. Single Object[] values, box the few fresh primitives (prebuilt-primitive entries reuse Entry's cached box, so 0 per-span alloc there). Cleaned stale open-questions/perf references to prims/segments/positional. tag: no release note Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ghout) Fix lingering positional/segment/forEachKnown references in the intro, impl list, and API-compat strategy to match the dense (id,value) array design. tag: no release note Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…timizedTagMap Phase 1 replaces OptimizedTagMap's Entry[] knownEntries with dense long[] ids + Object[] values in place — no new type/interface/codegen; it also removes the positional collision machinery (collidedSlots, occupancy, bucket-eviction). AttributeValueTable (interface + codegen POJO) is demoted to phase 2, extracted from the working dense impl when warranted. tag: no release note Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ilers hook) Validates phase 1. db.client-like tag set, build + build-iterate, gc profiler: build: current 14.99M ops/s 752 B/op | dense 21.32M(+42%) 248 B(-67%) | pojo 38.5M 64 B buildIter: current 8.37M ops/s 720 B/op | dense 10.19M(+22%) 224 B(-69%) | pojo 25.9M ~0* Dense beats Entry[] on BOTH throughput and allocation (no read-path regression); POJO is the ~2-3x / near-zero-alloc endgame. (* pojo buildIter ~0 = escape-analysis scalar replacement; escaping it allocs ~64 B/op.) Also: -PjmhProfilers hook in internal-api build.gradle.kts. tag: no release note Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace OptimizedTagMap's positional Entry[] knownEntries + collidedSlots collision machinery with a dense store: long[] knownIds + Object[] knownValues + int knownCount. Known tags (globalSerial != 0) scan-by-id to overwrite or append (grow from 8); unknown tags stay in the hash buckets. Deletes all collision logic (first-writer-wins, collidedSlots, bucket-eviction); fieldPos/slotCount no longer size storage. - Setting a known tag allocates NO Entry. getEntry/getAndSet/getAndRemove materialize on demand (explicit calls); knownRemove is O(1) swap-remove. - Iteration/forEach/serialize yield a REUSED flyweight EntryReader (EntryReadingHelper) for dense entries -> zero per-tag Entry on the serialize path; chained with bucket Entry-s. The one retention site (entrySet -> new HashMap<>(tagMap)) materializes Map.Entry via .mapEntry() in a dedicated EntriesIterator; no other consumer retains the reader. - Tests: TagMap* incl fuzz (50k iters) + DDSpanSerializationTest + dd-trace-core span/interceptor/tagprocessor suites all green. TagMapTagIdTest assertSame-> value-equality (getEntry now materializes fresh per call). Validated by AttrStoreBenchmark: dense beats the old Entry[] on throughput AND allocation (~+22-42% / -67-69%). tag: ai generated tag: no release note Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Stop allocating a transient Entry when a value lands in the flat (dense) store: - putKnownValue(long,Object): dense write core (scan-by-globalSerial -> overwrite or append/grow), builds no Entry, stores the value reference. - typed set(long,...) route to putKnownValue for known ids (strings/objects by reference = zero alloc; primitives boxed once, no Entry); globalSerial==0 -> bucket. set(String,...) resolve keyOf first: known -> putKnownValue, else bucket Entry (name preserved; no-resolver -> keyOf 0 -> bucket, unchanged). set(EntryReader) stores the reader's value via putKnownValue for known tags. setKnown(Entry) refactored to materialize prior only for getAndSet's return. - typed getters (getString/getInt/getBoolean/...) read knownValues directly via TagValueConversions, no Entry materialized; miss -> bucket path unchanged. getEntry still materializes (contract). Behavior note: getBoolean of a known NON-numeric object now coerces via TagValueConversions.toBoolean (-> false) vs the old ANY-Entry (value != null -> true). No production caller reads such a tag as boolean; boolean tags are stored as Boolean (fast path, identical). Acceptable. All TagMap*/fuzz/serialization/dd-trace-core suites green (force re-run). Benchmark pending. tag: ai generated tag: no release note Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The bucket array (Object[1<<4], for unknown/globalSerial==0 tags) was eagerly allocated + zeroed in every map's constructor, wasted on the common all-known span that never buckets anything. Now `buckets` starts null and is allocated only on the first unknown-tag insertion (setInBuckets / putAll); every read/scan/size/merge/ iterate path treats null as empty. EMPTY and clear() carry null buckets. A span whose tags are all known (the dense path) now allocates zero bucket array. TagMap* incl fuzz (empty/all-known/all-unknown/mixed/putAll/iterate) + serialization + dd-trace-core suites green (force re-run). tag: ai generated tag: no release note Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ration - OptimizedTagMap: positional dense store (knownValues[fieldPos], no linear scan, no per-tag Entry); id-keyed value reads getObject(long)/getString(long); cache resolved tagId on set(EntryReader) so shared cached entries skip keyOf. - TagSet: generic open-addressed string set; keyOf resolved through it. - Migrate hot decorators to setTag(long): span.kind/language/component, db.*, http.route — to skip the name->id keyOf on those set-sites. - Benchmarks: TagSet (SetBenchmark/KeyOfBenchmark), AttrStore (+ -prof gc), TagMapInsertion (id vs name insert/read). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
dougqh
added
comp: core
Contributor
🟢 Java Benchmark SLOs — All performance SLOs passed
PR vs. master results
Commit: Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion. |
…afety on merge
- TagMapInsertion{,Baseline}Benchmark -> TagMapAccess{,Baseline}Benchmark (it
covers reads now, not just insertion).
- TagMapFuzzTest.testMerge: after putAll, assert the SOURCE is unchanged, and
stays unchanged after the dest is independently mutated (guards against the
dest sharing a mutable BucketGroup chain with the source).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Contributor
Author
|
Supplanted by #11814. This was the original concept draft on the pre-kill-Legacy |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Concept
Known span tags (tag names known at agent build time) get a generated
longtag-id and are stored densely and positionally insideOptimizedTagMap—knownValues[fieldPos], no linear scan, no per-tagEntry. Name→id resolves through a generic open-addressed table (TagSet/KnownTags.keyOf); instrumentation migrates fromsetTag(String, …)tosetTag(long, …). Unknown / runtime-configured tags continue to live in the hash buckets, unchanged.Pieces here:
OptimizedTagMap— positional dense store; id-keyed value reads (getObject(long)/getString(long)); shared cached decorator entries cache their resolved id so they don't re-keyOfper span.TagSet— generic open-addressed string set (withSetBenchmark/KeyOfBenchmark).setTag(long):span.kind/language/component,db.*,http.route.AttrStoreBenchmark(+-prof gc),TagMapInsertionBenchmark.Measured — the win is allocation
TagMap$Entry, the changing List<Span> for List<DDSpan> #1 tracer allocator (~1% of process allocation under petclinic, by JFRallocation-by-class).±0.001 B/op).insertById≈ 3×insertByString;TagSetkeyOf≈ 2× a stringswitchand on par withHashSet.Status / honest caveats
WIP. The payoff is allocation / GC-pressure, not steady-state CPU. On a roomy heap, CPU/req is currently a small transitional regression: any still-string-keyed
setTagpays akeyOf(name→id) resolution that master doesn't, until instrumentation finishes migrating tosetTag(long). The CPU/GC payoff shows up under heap pressure (less allocation → fewer/shorter GCs).Follow-ups: broaden the id-migration (more integrations + id constants for
http.status_code,user_agent, client/server address,db.operation, …); typed-codegen path toward zero-per-span allocation; constrained-heap macro measurement.🤖 Generated with Claude Code