Client side access logging

[howardjohn]

Title: Client side access logging

Description:
It would be nice to be able to output standard Envoy access logs from the client. This would allow fine grained analysis of request patterns, rather than aggregations. This allows monitoring things like traffic trends (ex: a spike of latency every 30s from cronjob on the same host, decaying latency due to thermal throttling, etc).

[optional Relevant Links:]

Any extra documentation required to understand the issue.

[mum4k]

Thank you for starting this conversation @howardjohn, for completeness, could you provide a reference to what is meant by standard Envoy access logs, i.e. a link t the relevant portion of the Envoy documentation or maybe even an example?

[howardjohn]

Sure. Envoy provides a lot of configuration here https://www.envoyproxy.io/docs/envoy/latest/configuration/observability/access_log/usage . In terms of my priority I would say P0 is log to a file with default format in json or text format. P1 would be custom format. P2 is all the custom loggers

…

On Sat, Dec 4, 2021, 12:16 PM Jakub Sobon ***@***.***> wrote: Thank you for starting this conversation @howardjohn <https://github.com/howardjohn>, for completeness, could you provide a reference to what is meant by standard Envoy access logs, i.e. a link t the relevant portion of the Envoy documentation or maybe even an example? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#780 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAEYGXM5NMC4YSBCQIP54RLUPJZIRANCNFSM5JL3RXLA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[howardjohn]

Example of how this is useful: fortio/fortio#498. A similar feature in Fortio (or really my not-yet-merged fork of fortio) was able to find suboptimal traffic patterns that existing metric aggregations miss.

One concern is in Envoy the access log config is in listener rather than cluster. Although I did notice that there is an AccessLogManager input to cluster so maybe it would work.

[mum4k]

Thank you @howardjohn, adding the ability to enable the logging shouldn't be too hard and would be useful when analyzing performance patterns. There might be an impact on Nighthawk's ability to generate load (it might slow it down), but that would only happen when this is enabled and we can cross that bridge once we get there.

Please let me know if you would be interested in contributing this feature, in which case we can get together and design it. Alternatively we can mark this request as help needed.