feat(streaming): adapt backfill rate limit according to barrier latency

[kwannoel]

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

wip.

• Show grafana monitoring results.
• Add more scenarios: Join with Agg after, UDF high latency.
• Include more description on the alternative solutions.
• Add frontend support for setting adaptive rate limit.

We have a few test scenarios:

Amplification is medium x100

Backpressure can handle this case well. Adaptive Rate Limit does not have any discernible effect.

Amplification is extreme, e.g. x500

In this case, adaptive rate limit will not be able to keep barrier latency low.
Since barrier latency is a lagging metric, it means that rate limit will only be adjusted much later, and will take a while for it to take effect. In such a scenario, barrier latency will remain high.

Checklist

• I have written necessary rustdoc comments
• I have added necessary unit tests and integration tests
• I have added test labels as necessary. See details.
• I have added fuzzing tests or opened an issue to track them. (Optional, recommended for new SQL features Sqlsmith: Sql feature generation #7934).
• My PR contains breaking changes. (If it deprecates some features, please create a tracking issue to remove them in the future).
• All checks passed in ./risedev check (or alias, ./risedev c)
• My PR changes performance-critical code. (Please run macro/micro-benchmarks and show the results.)

• My PR contains critical fixes that are necessary to be merged into the latest release. (Please check out the details)

Documentation

• My PR needs documentation updates. (Please use the Release note section below to summarize the impact on users)

Release note

If this PR includes changes that directly affect users or other significant modifications relevant to the community, kindly draft a release note to provide a concise summary of these changes. Please prioritize highlighting the impact these changes will have on users.

[kwannoel]

The default adaptive rate limit strategy doesn't work well when I test it with the high amplification case.

1. The rate limit increases too fast (since it scales rate limit exponentially 2x).
2. We cannot simply get "current barrier latency". The previous barrier may not have been collected yet.
3. Even if we have threshold = 0, and we do linear increments, this linear increment of rate limit can also be too large (1000).

Then we end up having too high a rate limit due to 1 and 2. At that point, when adaptive rate limit kicks in, it's too slow as well, since its barrier latency is high (as rate limit is too high).
After we return to normalcy, threshold becomes 0, and the rate limit fluctuates between 1000, and the initial rate limit (1). At 1000, the rate limit is too high as well in amplification cases. Leading to high barrier latency still.