Improve exception logging when we fail to index / transform message #12594
Conversation
tibrewalpratik17
force-pushed
the
update_error_log
branch
from
a9693db
to
a392791
Compare
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #12594 +/- ##
============================================
- Coverage 61.75% 61.74% -0.02%
- Complexity 207 211 +4
============================================
Files 2436 2451 +15
Lines 133233 133725 +492
Branches 20636 20704 +68
============================================
+ Hits 82274 82562 +288
- Misses 44911 45055 +144
- Partials 6048 6108 +60
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
There was a problem hiding this comment.
Thanks @tibrewalpratik17 for the PR, left a comment for the clarification.
| @@ -572,6 +572,7 @@ private boolean processStreamEvents(MessageBatch messagesAndOffsets, long idlePi | |||
| // Decode message | |||
| StreamDataDecoderResult decodedRow = _streamDataDecoder.decode(messagesAndOffsets.getStreamMessage(index)); | |||
| msgMetadata = messagesAndOffsets.getStreamMessage(index).getMetadata(); | |||
| StreamPartitionMsgOffset messageOffset = messagesAndOffsets.getNextStreamPartitionMsgOffsetAtIndex(index); | |||
There was a problem hiding this comment.
It seems getNextStreamPartitionMsgOffsetAtIndex returns the offset of the next message for the given index. Should not we use getMessageOffsetAtIndex(int index) for getting the current message's offset that we wanted to use in the error messages?
Adds offset value in the log when a record is failed to index / transformed. This will help in faster debugging of problematic events.
There was a bug where the log was not showing the value of the decodedRow but was giving the address as below:
Updated to show the decoded row value in the log.