Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Multi-stage] Optimize group key generation #12394

Merged
merged 1 commit into from
Feb 13, 2024

Conversation

Jackie-Jiang
Copy link
Contributor

  • Use primitive type for single key column generation
  • For 2 key columns, combine 2 dictionary ids into a long and use long as the key to generate the ids
  • For multiple key columns, combine dictionary ids into a list and use it as the key to generate the ids
  • Support BigDecimal column as key

The algorithm is the same as NoDictionaryMultiColumnGroupKeyGenerator

@codecov-commenter
Copy link

codecov-commenter commented Feb 10, 2024

Codecov Report

Attention: 174 lines in your changes are missing coverage. Please review.

Comparison is base (43dadbf) 61.73% compared to head (fceadd0) 61.71%.

Files Patch % Lines
...upby/NoDictionaryMultiColumnGroupKeyGenerator.java 44.68% 45 Missing and 7 partials ⚠️
.../operator/groupby/OneFloatKeyGroupIdGenerator.java 0.00% 31 Missing ⚠️
...e/operator/groupby/OneLongKeyGroupIdGenerator.java 0.00% 31 Missing ⚠️
...operator/groupby/OneDoubleKeyGroupIdGenerator.java 59.37% 8 Missing and 5 partials ⚠️
...me/operator/groupby/MultiKeysGroupIdGenerator.java 70.00% 9 Missing and 3 partials ⚠️
.../query/aggregation/groupby/utils/ValueToIdMap.java 0.00% 8 Missing ⚠️
...time/operator/groupby/TwoKeysGroupIdGenerator.java 80.95% 3 Missing and 5 partials ⚠️
.../query/aggregation/groupby/utils/FloatToIdMap.java 57.14% 3 Missing ⚠️
...e/query/aggregation/groupby/utils/LongToIdMap.java 57.14% 3 Missing ⚠️
...time/operator/groupby/GroupIdGeneratorFactory.java 70.00% 2 Missing and 1 partial ⚠️
... and 5 more
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #12394      +/-   ##
============================================
- Coverage     61.73%   61.71%   -0.03%     
  Complexity      207      207              
============================================
  Files          2428     2436       +8     
  Lines        132828   133069     +241     
  Branches      20545    20607      +62     
============================================
+ Hits          82007    82118     +111     
- Misses        44811    44921     +110     
- Partials       6010     6030      +20     
Flag Coverage Δ
custom-integration1 <0.01% <0.00%> (-0.01%) ⬇️
integration <0.01% <0.00%> (-0.01%) ⬇️
integration1 <0.01% <0.00%> (-0.01%) ⬇️
integration2 0.00% <0.00%> (ø)
java-11 61.67% <57.66%> (+<0.01%) ⬆️
java-21 61.55% <57.66%> (-0.07%) ⬇️
skip-bytebuffers-false 61.69% <57.66%> (-0.02%) ⬇️
skip-bytebuffers-true 61.52% <57.66%> (-0.06%) ⬇️
temurin 61.71% <57.66%> (-0.03%) ⬇️
unittests 61.70% <57.66%> (-0.03%) ⬇️
unittests1 46.87% <57.66%> (-0.05%) ⬇️
unittests2 27.70% <0.00%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Jackie-Jiang Jackie-Jiang merged commit dc427fb into apache:master Feb 13, 2024
19 checks passed
@Jackie-Jiang Jackie-Jiang deleted the optimize_group_key_gen branch February 13, 2024 07:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bugfix enhancement multi-stage Related to the multi-stage query engine performance
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants