Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Add subgraph name and version as label to prometheus metrics #5278

Open
1 of 2 tasks
mmd-afegbua opened this issue Mar 13, 2024 · 0 comments
Open
1 of 2 tasks
Labels
enhancement New feature or request

Comments

@mmd-afegbua
Copy link

mmd-afegbua commented Mar 13, 2024

Description

Currently, I check for the status of my subgraph nodes with the following prometheus query:
(ethereum_chain_head_number{} - ignoring(deployment, job, network, shard) deployment_head{} > 100) and ignoring(deployment, job, network, shard)(deriv(deployment_head{}[10m]) * 60 <= 0)

In simple terms, the query checks for the differences between the ethereum_chain_head_number metric and the deployment_head metric. If it is more than 100 blocks AND there hasn't been any significant changes in the deployment_head value over a 10 minutes window, the node is flagged as out of sync. The two metrics do not have the same labels, hence the ignoring of labels as seen in the query.

This query works fine for nodes hosting one or more subgraphs of different networks. However, when a subgraph of the same network but different subgraphName is deployed to a node, the only way to differentiate them is through the deployment label, which is a hash number that changes for any update and therefore, complex to manage. This introduces an issue as deployment_head starts coming off as a duplicate metric.

sample:

Assuming the two metrics below are for two subgraphs (sepolia-1 and sepolia-2), I can't tell them apart, and as far as I don't filter by deployment, it is a duplicate metric when evaluating it.

deployment_head{deployment=<HASH IPFS deployment number 1>, instance=<node_url>, job="mymetrics", network="sepolia", shard="primary"}

deployment_head{deployment=<HASH IPFS deployment number 2>, instance=<node_url>, job="mymetrics", network="sepolia", shard="primary"}

Proposal:

Add subgraphName and subgraphVersion to the BlockStreamMetrics struct. subgraphName being top priority.

deployment_head{deployment=<HASH IPFS deployment number 1>, instance=<node_url>, job="mymetrics", network="sepolia", shard="primary", subgraph="sepolia-1", version="v0.1.0"}

deployment_head{deployment=<HASH IPFS deployment number 2>, instance=<node_url>, job="mymetrics", network="sepolia", shard="primary", subgraph="sepolia-2", version="v0.1.0"}

Are you aware of any blockers that must be resolved before implementing this feature? If so, which? Link to any relevant GitHub issues.

No response

Some information to help us out

  • Tick this box if you plan on implementing this feature yourself.
  • I have searched the issue tracker to make sure this issue is not a duplicate.
@mmd-afegbua mmd-afegbua added the enhancement New feature or request label Mar 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant