Update getValidDocIdsMetadataFromServer to make call in batches to servers and other bug fixes

[tibrewalpratik17]

label:
bugfix
upsert

This patch addresses multiple things:

1. We saw issue with upsert-compaction not working for few servers of a table but working for others. This was primarily because we send all segment names from controller to server in payload and even if one is missing (which can be due to concurrent deletion) it results in NOT_FOUND error. We are moving the failure behaviour of API in this scenario to a warn log instead and continue proceeding to return valid doc ids for whatever segments are available.
2. We send high number of segments in the payload from controller to server. Sending requests in batches so that request-body is nominal in size and if there is issue with one segment, then entire host data is not missed but only batch's data.
3. For CompletionServiceHelper code, if we enabled multiRequestsPerServer flag and the uri is same for the requests to a server, then only last response will be returned back to the caller. We add a count value to the uri key in this scenario to honour all responses from a single host.

cc @klsince @snleee

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 33.33333% with 20 lines in your changes missing coverage. Please review.

Project coverage is 62.04%. Comparing base (59551e4) to head (4cfb8ce).
Report is 562 commits behind head on master.

Files	Patch %	Lines
...t/controller/util/ServerSegmentMetadataReader.java	0.00%	13 Missing ⚠️
...psertcompaction/UpsertCompactionTaskGenerator.java	0.00%	4 Missing ⚠️
...oller/api/resources/PinotTableRestletResource.java	0.00%	1 Missing ⚠️
...pinot/controller/util/CompletionServiceHelper.java	90.90%	0 Missing and 1 partial ⚠️
...che/pinot/server/api/resources/TablesResource.java	0.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #13314      +/-   ##
============================================
+ Coverage     61.75%   62.04%   +0.29%     
+ Complexity      207      198       -9     
============================================
  Files          2436     2544     +108     
  Lines        133233   139682    +6449     
  Branches      20636    21606     +970     
============================================
+ Hits          82274    86662    +4388     
- Misses        44911    46526    +1615     
- Partials       6048     6494     +446     

Flag	Coverage Δ
custom-integration1	<0.01% <0.00%> (-0.01%)	⬇️
integration	<0.01% <0.00%> (-0.01%)	⬇️
integration1	<0.01% <0.00%> (-0.01%)	⬇️
integration2	0.00% <0.00%> (ø)
java-11	62.00% <33.33%> (+0.29%)	⬆️
java-21	61.93% <33.33%> (+0.30%)	⬆️
skip-bytebuffers-false	62.02% <33.33%> (+0.27%)	⬆️
skip-bytebuffers-true	61.90% <33.33%> (+34.17%)	⬆️
temurin	62.04% <33.33%> (+0.29%)	⬆️
unittests	62.03% <33.33%> (+0.29%)	⬆️
unittests1	46.58% <ø> (-0.31%)	⬇️
unittests2	27.74% <33.33%> (+0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[klsince]

LGTM, just some comments on variable naming