Uploaded image for project: 'Samza'
  1. Samza
  2. SAMZA-2593

Update task callback to store only necessary fields instead of the message envelope

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 1.6
    • None
    • None

    Description

       TaskCallbackImpl currently stores the entire message envelope. In case of jobs that have 1000s of pending callbacks, this adds up to significant memory pressure. The only use for message envelope is to trace the SSP information and the offset associated with it so that we commit the right offset during our commit cycle.

      Suggested solution: Modify TaskCallbackImpl to only store SSP and Offset information.

       

      Attachments

        Activity

          People

            bharathkk Bharath Kumarasubramanian
            bharathkk Bharath Kumarasubramanian
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 0.5h
                0.5h