Convert ingestion aggregation source column types before indexing

[Vamsi-klu]

Why

Ingestion-level aggregations (SUM(col1) AS col2) are a core rollup-on-write pattern. When col1 is not in the schema (common — only the aggregated metric is stored), DataTypeTransformer never converted it. Aggregators then saw raw strings/objects, threw (ClassCastException / parse errors), and left the mutable segment on the partial-row corruption path described in #16316.

Impact

• Correct, reliable rollup-on-write for JSON/Avro string numerics without stuffing raw columns into the table schema.
• Fewer mid-row exceptions during realtime consumption for aggregation tables.
• Complements (does not replace) fail-soft indexing in MutableSegmentImpl.index() can leave segment in corrupted state when encountering exception #16316 — prefer failing/converting before mutation when possible.
• Uses existing SourceFieldConfig / transformer machinery — no new schema field type.

How

• Auto-derive aggregation source columns that are absent from the schema and register them for type conversion in RecordTransformerUtils (alongside optional explicit SourceFieldConfig).
• Keep ValueAggregatorUtils.toDouble as a safety net; document that pipeline conversion is preferred so bad values fail before MutableSegmentImpl mutates.
• Tests for string sources not in schema through the transform + mutable aggregation path.

Test plan

• MutableSegmentImplIngestionAggregationTest — string/non-schema sources for SUM/MIN/MAX-style configs.
• RecordTransformerTest — aggregation source auto type conversion order/behavior.
• ./mvnw -pl pinot-segment-local -am -Dtest=MutableSegmentImplIngestionAggregationTest,RecordTransformerTest -Dsurefire.failIfNoSpecifiedTests=false test

Related

fixes: #16317
related: #16316

Reviewers

Suggested: Jackie-Jiang (issue design guidance)

---

Was generative AI tooling used to co-author this PR?

• Yes — Grok Build (xAI)

Generated-by: Grok Build (xAI)

[codecov-commenter]

Codecov Report

❌ Patch coverage is 62.90323% with 23 lines in your changes missing coverage. Please review.
✅ Project coverage is 65.47%. Comparing base (2bcbdfe) to head (e154347).

Files with missing lines	Patch %	Lines
...ocal/recordtransformer/RecordTransformerUtils.java	62.90%	15 Missing and 8 partials ⚠️

Additional details and impacted files

@@            Coverage Diff            @@
##             master   #19087   +/-   ##
=========================================
  Coverage     65.47%   65.47%           
  Complexity     1421     1421           
=========================================
  Files          3426     3426           
  Lines        217315   217366   +51     
  Branches      34509    34522   +13     
=========================================
+ Hits         142283   142327   +44     
- Misses        63513    63518    +5     
- Partials      11519    11521    +2     

Flag	Coverage Δ
custom-integration1	100.00% <ø> (ø)
integration	100.00% <ø> (ø)
integration1	100.00% <ø> (ø)
integration2	0.00% <ø> (ø)
java-25	65.47% <62.90%> (+<0.01%)	⬆️
temurin	65.47% <62.90%> (+<0.01%)	⬆️
unittests	65.47% <62.90%> (+<0.01%)	⬆️
unittests1	56.82% <14.51%> (-0.01%)	⬇️
unittests2	37.88% <62.90%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[Vamsi-klu]

Ready for review — all required CI checks are green on this PR.

Issue: #16317

Could the following folks take a look when convenient?
@Jackie-Jiang

Formal GitHub "Request review" is unavailable from a fork contributor account on apache/pinot (API returns 404/403), so this is the review ping.

Thank you!