refactor: Extract "HttpStats" utility class

[zharinov]

Changes

• Wrap HTTP statistics into the helper class
• Add median and max data

Context

• Ref: Measure package cache hit percent #27035

Documentation (please check one with an [x])

• I have updated the documentation, or
• No documentation update is required

How I've tested my work (please select one)

I have verified these changes via:

• Code inspection only, or
• Newly added/modified unit tests, or
• No unit tests but ran on a real repository, or
• Both unit tests + ran on a real repository

[viceice]

otherwise LGTM

[rarkins]

Is the output structure identical? Adding @nabeelsaabna because he parses some of these stats in our backend

[rarkins]

@zharinov do you have a before/after example?

[zharinov]

No, it's not identical

[zharinov]

And previous ones are different too, BTW

[zharinov]

DEBUG: Package cache statistics (repository=renovate-testing/26315-extract-rust-versions-for-releases)
       "get": {"count": 2, "avgMs": 2, "medianMs": 2, "maxMs": 2, "totalMs": 3},
       "set": {"count": 2, "avgMs": 3, "medianMs": 4, "maxMs": 4, "totalMs": 6}
DEBUG: HTTP stats (repository=renovate-testing/26315-extract-rust-versions-for-releases)
       "urlCounts": {
         "https://api.github.com/graphql (POST, 200)": 1,
         "https://api.github.com/repos/renovate-testing/26315-extract-rust-versions-for-releases/branches/main/protection (GET, 404)": 1,
         "https://api.github.com/repos/renovate-testing/26315-extract-rust-versions-for-releases/contents/renovate.json (GET, 304)": 1,
         "https://api.github.com/repos/renovate-testing/26315-extract-rust-versions-for-releases/issues/3 (GET, 304)": 1,
         "https://api.github.com/repos/renovate-testing/26315-extract-rust-versions-for-releases/pulls (GET, 304)": 1,
         "https://crates.io/api/v1/crates/log (GET, 200)": 1,
         "https://raw.githubusercontent.com/rust-lang/crates.io-index/master/3/l/log (GET, 200)": 1
       },
       "statsByHost": {
         "api.github.com": {
           "count": 5,
           "reqAvgMs": 278,
           "reqMedianMs": 269,
           "reqMaxMs": 373,
           "queueAvgMs": 0,
           "queueMedianMs": 0,
           "queueMaxMs": 1
         },
         "crates.io": {
           "count": 1,
           "reqAvgMs": 657,
           "reqMedianMs": 657,
           "reqMaxMs": 657,
           "queueAvgMs": 1,
           "queueMedianMs": 1,
           "queueMaxMs": 1
         },
         "raw.githubusercontent.com": {
           "count": 1,
           "reqAvgMs": 301,
           "reqMedianMs": 301,
           "reqMaxMs": 301,
           "queueAvgMs": 0,
           "queueMedianMs": 0,
           "queueMaxMs": 0
         }
       },
       "totalRequests": 7
DEBUG: Lookup statistics (repository=renovate-testing/26315-extract-rust-versions-for-releases)
       "crate": {"count": 1, "avgMs": 1010, "medianMs": 1010, "maxMs": 1010, "totalMs": 1010}

[rarkins]

Looks ok, I will wait for @nabeelsaabna to confirm we can use it or adapt to it

[nabeelsaabna]

I see mainly name changes that we can update easily 👍

urls -> urlCounts
hostStats -> statsByHost

old log example

{
	"level": 20,
	"urls": {
		"https://api.github.com/graphql (POST,200)": 1
	},
	"hostStats": {
		"api.github.com": {
			"requestCount": 1,
			"requestAvgMs": 93,
			"queueAvgMs": 0
		}
	},
	"totalRequests": 3,
	"msg": "http statistics"
}

[nabeelsaabna]

Side note: for using this message in graphs/gathering metrics and trends it would be more useful to extract the (POST,200) from the URL key
e.g.

{
	"urls": {
		"https://api.github.com/graphql": {
			"POST": {
				"200": 1
				"404": 3
			}
		},
		"https://crates.io/api/v1/crates/log": {
			"GET": {
				"200": 1
			}
		}
	}
}

[zharinov]

Please decide what changes to make, I'll do it later today. Now I have to go and pass Spanish exam, see you.

[nabeelsaabna]

Approving naming/structure changes

[nabeelsaabna]

Please decide what changes to make, I'll do it later today. Now I have to go and pass Spanish exam, see you.

no changes need in this PR

[zharinov]

Almost done adjusting the structure:

"urls": {
         "https://api.github.com/graphql": {"POST": {"200": 1}},
         "https://api.github.com/repos/renovate-testing/26315-extract-rust-versions-for-releases/branches/main/protection": {
           "GET": {"404": 1}
         },
         "https://api.github.com/repos/renovate-testing/26315-extract-rust-versions-for-releases/contents/renovate.json": {
           "GET": {"304": 1}
         },
         "https://api.github.com/repos/renovate-testing/26315-extract-rust-versions-for-releases/issues/3": {
           "GET": {"304": 1}
         },
         "https://api.github.com/repos/renovate-testing/26315-extract-rust-versions-for-releases/pulls": {
           "GET": {"304": 1}
         }
       }

Would you like to have it like this? Or never mind?

[zharinov]

The final variant:

DEBUG: HTTP statistics (repository=renovate-testing/26315-extract-rust-versions-for-releases)
       "urls": {
         "https://api.github.com/graphql": {"POST": {"200": 1}},
         "https://api.github.com/repos/renovate-testing/26315-extract-rust-versions-for-releases/branches/main/protection": {
           "GET": {"404": 1}
         },
         "https://api.github.com/repos/renovate-testing/26315-extract-rust-versions-for-releases/contents/renovate.json": {
           "GET": {"304": 1}
         },
         "https://api.github.com/repos/renovate-testing/26315-extract-rust-versions-for-releases/issues/3": {
           "GET": {"304": 1}
         },
         "https://api.github.com/repos/renovate-testing/26315-extract-rust-versions-for-releases/pulls": {
           "GET": {"304": 1}
         }
       },
       "hosts": {
         "api.github.com": {
           "count": 5,
           "reqAvgMs": 259,
           "reqMedianMs": 248,
           "reqMaxMs": 308,
           "queueAvgMs": 0,
           "queueMedianMs": 0,
           "queueMaxMs": 0
         }
       },
       "requests": 5

• Message is HTTP statistics
• Field names are urls, hosts, requests

[rarkins]

I think the get/post improvement is useful!

[viceice]

needs prettier fix

[renovate-release]

🎉 This PR is included in version 37.249.1 🎉

The release is available on:

• GitHub release
• npm package (@latest dist-tag)
• 37.249.1

Your semantic-release bot 📦🚀