Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve logging in range index handler for index updates #13381

Merged

Conversation

yashmayya
Copy link
Collaborator

  • Currently, if a range index is added to a column and segment reload is triggered for all segments, not all segments will necessarily have a range index created for that column. Segments where the column is a sorted column will skip creation of the range index.
  • This isn't very clearly documented and since there's no associated logging as well, users could be quite confused about why certain segments had the range index created while others didn't (and might assume that something went wrong with the segment reload).
  • Also, if there range index updates needed for multiple columns (additions or removals), only the first one is logged.
  • This PR fixes both the above issues by adding some clear logging.

Comment on lines +85 to +87
if (columnMetadata == null) {
continue;
}
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not quite clear when we'd hit this case but I've retained the existing check.

@codecov-commenter
Copy link

codecov-commenter commented Jun 13, 2024

Codecov Report

Attention: Patch coverage is 77.77778% with 2 lines in your changes missing coverage. Please review.

Project coverage is 61.98%. Comparing base (59551e4) to head (7c24ad5).
Report is 617 commits behind head on master.

Files Patch % Lines
.../index/loader/invertedindex/RangeIndexHandler.java 77.77% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #13381      +/-   ##
============================================
+ Coverage     61.75%   61.98%   +0.23%     
+ Complexity      207      198       -9     
============================================
  Files          2436     2550     +114     
  Lines        133233   140260    +7027     
  Branches      20636    21802    +1166     
============================================
+ Hits          82274    86947    +4673     
- Misses        44911    46716    +1805     
- Partials       6048     6597     +549     
Flag Coverage Δ
custom-integration1 <0.01% <0.00%> (-0.01%) ⬇️
integration <0.01% <0.00%> (-0.01%) ⬇️
integration1 <0.01% <0.00%> (-0.01%) ⬇️
integration2 0.00% <0.00%> (ø)
java-11 61.95% <77.77%> (+0.24%) ⬆️
java-21 61.87% <77.77%> (+0.24%) ⬆️
skip-bytebuffers-false 61.97% <77.77%> (+0.23%) ⬆️
skip-bytebuffers-true 61.84% <77.77%> (+34.11%) ⬆️
temurin 61.98% <77.77%> (+0.23%) ⬆️
unittests 61.98% <77.77%> (+0.23%) ⬆️
unittests1 46.55% <0.00%> (-0.34%) ⬇️
unittests2 27.66% <77.77%> (-0.07%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Jackie-Jiang Jackie-Jiang merged commit 0ca1ab6 into apache:master Jun 17, 2024
19 of 20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants