Fix few metric rules which were affected by the database prefix handling #13290
Conversation
|
Can you list the broken metrics and how they are fixed in this PR? |
@soumitra-st Updated the description with metric details. |
|
How shall we test it out? One possibility is to get the list of available metrics on broker/controller/server using |
|
Looks good, deferring to @suddendust to double check. |
Yeah it's a pain to test these changes. Right now just making sure no random string appear in |
|
Please do add testing done and screenshots of validated rules as they appear in jconsole / prometheus explore |
| @@ -73,6 +66,14 @@ rules: | |||
| table: "$1$3" | |||
| tableType: "$4" | |||
| partition: "$5" | |||
| - pattern: "\"org\\.apache\\.pinot\\.common\\.metrics\"<type=\"ServerMetrics\", name=\"pinot\\.server\\.endToEndRealtimeIngestionDelayMs\\.(([^.]+)\\.)?([^.]*)_(OFFLINE|REALTIME)\\.(\\w+)\"><>(\\w+)" | |||
There was a problem hiding this comment.
realtimeIngestionDelayMs is a different metric than this. Can you validate if that one is being successfully exported?
There was a problem hiding this comment.
Yeah. realtimeIngestionDelayMs is handled by moving the metric rule earlier present on line 16 to line 175.
Will add screenshots on validation as well
| @@ -112,6 +118,12 @@ rules: | |||
| - pattern: "\"org\\.apache\\.pinot\\.common\\.metrics\"<type=\"BrokerMetrics\", name=\"pinot\\.broker\\.routingTableUpdateTime\"><>(\\w+)" | |||
| name: "pinot_broker_routingTableUpdateTime_$1" | |||
| cache: true | |||
| - pattern: "\"org\\.apache\\.pinot\\.common\\.metrics\"<type=\"BrokerMetrics\", name=\"pinot\\.broker\\.adaptiveServerSelectorType\"><>(\\w+)" | |||
| name: "pinot_broker_adaptiveServerSelectorType_$1" | |||
There was a problem hiding this comment.
Can we please add lables here?
There was a problem hiding this comment.
Its does not have any extracted info to add as a label. More of a static config info metric
| name: "pinot_broker_adaptiveServerSelectorType_$1" | ||
| cache: true | ||
| - pattern: "\"org\\.apache\\.pinot\\.common\\.metrics\"<type=\"BrokerMetrics\", name=\"pinot\\.broker\\.adaptiveServerSelectorType\\.(\\w+)\"><>(\\w+)" | ||
| name: "pinot_broker_adaptiveServerSelectorType_$1_$2" |
| cache: true | ||
| labels: | ||
| database: "$2" | ||
| table: "$1$3" |
There was a problem hiding this comment.
Do we want to delimit $1 from $3?
There was a problem hiding this comment.
$1 matches the whole databaseName. part hence no need to delimit explicitly.
| cache: true | ||
| labels: | ||
| database: "$2" | ||
| table: "$1$3" |
Controller metrics validations:
Broker metrics validations
Server metrics validation
|
Description
This PR fixes few jmx metric rules that were broken due to the database prefix handling on metrics.
name=\"pinot\\.server\\.(([^.]+)\\.)?([^.]*)_(OFFLINE|REALTIME)\\.(\\w+)\"pinot.broker.requestSize.<tableName>pinot.broker.adaptiveServerSelectorTypepinot.broker.adaptiveServerSelectorType.<type>pinot.controller.dataDir.[exists, fileOpLatencyMs]pinot.server.endToEndRealtimeIngestionDelayMs.<tableName>labels
bugfix