Allow all raw index config in star-tree index

[Jackie-Jiang]

Add the following configs into StarTreeAggregationConfig as counterpart of ForwardIndexConfig:

• deriveNumDocsPerChunk
• indexVersion
• targetMaxChunkSize
• targetDocsPerChunk

During table config validation:

• Add a check to allow only one of functionColumnPairs or aggregationConfigs to be configured
• Fix a bug where function duplicate check is mistakenly performed across multiple star-tree indices

Backward Incompatible

• Apply more strict table config validation during table creation or update

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 80.58252% with 20 lines in your changes are missing coverage. Please review.

Project coverage is 62.18%. Comparing base (59551e4) to head (134c8bd).
Report is 496 commits behind head on master.

Files	Patch %	Lines
...ot/segment/spi/index/startree/AggregationSpec.java	67.85%	8 Missing and 1 partial ⚠️
...he/pinot/segment/local/utils/TableConfigUtils.java	76.47%	1 Missing and 3 partials ⚠️
...ot/spi/config/table/StarTreeAggregationConfig.java	71.42%	3 Missing and 1 partial ⚠️
...cal/startree/v2/builder/BaseSingleTreeBuilder.java	62.50%	3 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #13225      +/-   ##
============================================
+ Coverage     61.75%   62.18%   +0.43%     
+ Complexity      207      198       -9     
============================================
  Files          2436     2533      +97     
  Lines        133233   138872    +5639     
  Branches      20636    21504     +868     
============================================
+ Hits          82274    86355    +4081     
- Misses        44911    46065    +1154     
- Partials       6048     6452     +404     

Flag	Coverage Δ
custom-integration1	<0.01% <0.00%> (-0.01%)	⬇️
integration	<0.01% <0.00%> (-0.01%)	⬇️
integration1	<0.01% <0.00%> (-0.01%)	⬇️
integration2	0.00% <0.00%> (ø)
java-11	62.13% <80.58%> (+0.42%)	⬆️
java-21	62.07% <80.58%> (+0.44%)	⬆️
skip-bytebuffers-false	62.17% <80.58%> (+0.42%)	⬆️
skip-bytebuffers-true	62.03% <80.58%> (+34.31%)	⬆️
temurin	62.18% <80.58%> (+0.43%)	⬆️
unittests	62.17% <80.58%> (+0.43%)	⬆️
unittests1	46.69% <61.16%> (-0.20%)	⬇️
unittests2	27.84% <19.41%> (+0.10%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[satwik-pachigolla]

Add a check to not allow only one of functionColumnPairs and aggregationConfigs to be configured

Can we add the 'breaking-change' and / or 'incompatible' labels to the PR since this will break in cases where both configs were being used at the same time.

[Jackie-Jiang]

@satwik-pachigolla It won't break existing tables since it is checked during table config validation. Pinot won't allow new table config/table config update with both configs

[satwik-pachigolla]

@Jackie-Jiang It isn't a breaking change but I would still label it incompatible / backwards-incompat. If someone pulls in this commit without realising this change in behavior, it will break any unrelated config change automation or be confusing for users who are not as knowledgeable about Pinot. This will force Pinot maintainers to either revert this or rush an update to all table configs.

[jadami10]

for future viewers, clarification on

Add a check to not allow only one of functionColumnPairs and aggregationConfigs to be configured

it should be: Add a check to not only allow only one of functionColumnPairs and aggregationConfigs to be configured

[Jackie-Jiang]

Fixed, thanks for pointing it out @jadami10