[Merged by Bors] - Add Exponential Moving Average into diagnostics #4992

CAD97 · 2022-06-11T23:14:29Z

Objective

Add Time-Adjusted Rolling EMA-based smoothing to diagnostics.
Closes Provide smoothed diagnostics readings #4983; see that issue for more more information.

Terms

EMA - Exponential Moving Average
SMA - Simple Moving Average

Solution

We use a fairly standard approximation of a true EMA where $EMA_{\text{frame}} = EMA_{\text{previous}} + \alpha \left( x_{\text{frame}} - EMA_{\text{previous}} \right)$ where $\alpha = \Delta t / \tau$ and $\tau$ is an arbitrary smoothness factor. (See Provide smoothed diagnostics readings #4983 for more discussion of the math.)
The smoothness factor is here defaulted to $2 / 21$; this was chosen fairly arbitrarily as supposedly related to the existing 20-bucket SMA.
The smoothness factor can be set on a per-diagnostic basis via Diagnostic::with_smoothing_factor.

Changelog

Added

Diagnostic::smoothed - provides an exponentially smoothed view of a recorded diagnostic, to e.g. reduce jitter in frametime readings.

Changed

LogDiagnosticsPlugin now records the smoothed value rather than the raw value.
- For diagnostics recorded less often than every 0.1 seconds, this change to defaults will have no visible effect.
- For discrete diagnostics where this smoothing is not desirable, set a smoothing factor of 0 to disable smoothing.
- The average of the recent history is still shown when available.

CAD97 · 2022-06-13T00:57:44Z

I should note that due to exactly how diagnostics were calculated/tracked, the translation is for numerically (almost¹) equivalent now

before	after
`diagnostics[FRAME_TIME].value()`	`diagnostics[FRAME_TIME].value()`
`diagnostics[FRAME_TIME].average()`	`diagnostics[FRAME_TIME].average()`
(none)	`diagnostics[FRAME_TIME].smoothed()`
`1 / diagnostics[FRAME_TIME].value()`	`diagnostics[FPS].value()`
`diagnostics[FPS].value()` (is also)	`diagnostics[FPS].average()`
`1 / diagnostics[FRAME_TIME].average()`	`diagnostics[FPS].average()`
`diagnostics[FPS].average()`	(none)²
(none)	`diagnostics[FPS].smoothed()`

floats (aliens.png) ↩
If for some reason you do want the old FPS average³, this is achievable by averaging 20 frames worth of data of the new averaged FPS or FRAME_TIME. This is most simply done by recording a diagnostic with a 20-length history of 1 / diagnostics[FRAME_TIME].average() and taking its .average(); this is the old way that the FPS diagnostic was calculated. ↩
Perhaps for consistency of performance metrics over time? Though the engine update is likely to outweigh the impact of changing how the FPS diagnostic is recorded. ↩

CAD97 · 2022-06-13T01:24:51Z

I went ahead and added per-diagnostic customization of the smoothing factor so it can be more easily tuned.

I will reiterate that the default factor of $2 / 21$ here is completely arbitrary, and it should ideally be tuned for each application. $0.1$ is a rather common starting point, so we're in good company, but the exact smoothing feel desired differs per developer, and the default should probably be chosen by someone more familiar with bevy.

Discrete diagnostic measurements like asset or entity count probably will want to use a smoothing factor of $0$ to disable smoothing, especially as this PR currently prints the smoothed diagnostic in LogDiagnosticPlugin. This will avoid smoothing these counts and presenting a fractional number of entities 🙃 I haven't done so yet to let the stakeholders here decide appropriate tuning for the smoothing factor(s).

alice-i-cecile

Yes please. This is well implemented. I like that you changed the examples to use this, and I appreciate that you can control the smoothing on a per-diagnostic basis.

hymm

I'm in favor of having this built in and using this in the examples.

crates/bevy_diagnostic/src/diagnostic.rs

CAD97 · 2022-10-21T00:54:08Z

Rebased for the upstream changes. Note that #3871 already did the change from logging FPS as $1/SMA_\text{frametime}$ to just $1/\text{frametime}$, so this PR no longer contains that change.

alice-i-cecile · 2022-10-24T13:28:16Z

bors r+

# Objective - Add Time-Adjusted Rolling EMA-based smoothing to diagnostics. - Closes #4983; see that issue for more more information. ## Terms - EMA - [Exponential Moving Average](https://en.wikipedia.org/wiki/Moving_average#Exponential_moving_average) - SMA - [Simple Moving Average](https://en.wikipedia.org/wiki/Moving_average#Simple_moving_average) ## Solution - We use a fairly standard approximation of a true EMA where $EMA_{\text{frame}} = EMA_{\text{previous}} + \alpha \left( x_{\text{frame}} - EMA_{\text{previous}} \right)$ where $\alpha = \Delta t / \tau$ and $\tau$ is an arbitrary smoothness factor. (See #4983 for more discussion of the math.) - The smoothness factor is here defaulted to $2 / 21$; this was chosen fairly arbitrarily as supposedly related to the existing 20-bucket SMA. - The smoothness factor can be set on a per-diagnostic basis via `Diagnostic::with_smoothing_factor`. --- ## Changelog ### Added - `Diagnostic::smoothed` - provides an exponentially smoothed view of a recorded diagnostic, to e.g. reduce jitter in frametime readings. ### Changed - `LogDiagnosticsPlugin` now records the smoothed value rather than the raw value. - For diagnostics recorded less often than every 0.1 seconds, this change to defaults will have no visible effect. - For discrete diagnostics where this smoothing is not desirable, set a smoothing factor of 0 to disable smoothing. - The average of the recent history is still shown when available.

bors · 2022-10-24T14:11:30Z

Pull request successfully merged into main.

Build succeeded:

# Objective - Add Time-Adjusted Rolling EMA-based smoothing to diagnostics. - Closes bevyengine#4983; see that issue for more more information. ## Terms - EMA - [Exponential Moving Average](https://en.wikipedia.org/wiki/Moving_average#Exponential_moving_average) - SMA - [Simple Moving Average](https://en.wikipedia.org/wiki/Moving_average#Simple_moving_average) ## Solution - We use a fairly standard approximation of a true EMA where $EMA_{\text{frame}} = EMA_{\text{previous}} + \alpha \left( x_{\text{frame}} - EMA_{\text{previous}} \right)$ where $\alpha = \Delta t / \tau$ and $\tau$ is an arbitrary smoothness factor. (See bevyengine#4983 for more discussion of the math.) - The smoothness factor is here defaulted to $2 / 21$; this was chosen fairly arbitrarily as supposedly related to the existing 20-bucket SMA. - The smoothness factor can be set on a per-diagnostic basis via `Diagnostic::with_smoothing_factor`. --- ## Changelog ### Added - `Diagnostic::smoothed` - provides an exponentially smoothed view of a recorded diagnostic, to e.g. reduce jitter in frametime readings. ### Changed - `LogDiagnosticsPlugin` now records the smoothed value rather than the raw value. - For diagnostics recorded less often than every 0.1 seconds, this change to defaults will have no visible effect. - For discrete diagnostics where this smoothing is not desirable, set a smoothing factor of 0 to disable smoothing. - The average of the recent history is still shown when available.

alice-i-cecile added C-Enhancement A new feature A-Diagnostics Logging, crash handling, error reporting and performance analysis C-Breaking-Change A breaking change to Bevy's public API that needs to be noted in a migration guide labels Jun 12, 2022

hymm self-requested a review June 21, 2022 19:50

alice-i-cecile approved these changes Oct 10, 2022

View reviewed changes

hymm suggested changes Oct 10, 2022

View reviewed changes

crates/bevy_diagnostic/src/diagnostic.rs Show resolved Hide resolved

Add Exponential Moving Average into diagnostics

024d77b

CAD97 force-pushed the ema branch from 2e95eb4 to 024d77b Compare October 21, 2022 00:51

alice-i-cecile requested a review from hymm October 21, 2022 00:53

hymm approved these changes Oct 21, 2022

View reviewed changes

alice-i-cecile added the S-Ready-For-Final-Review This PR has been approved by the community. It's ready for a maintainer to consider merging it label Oct 21, 2022

bors bot changed the title ~~Add Exponential Moving Average into diagnostics~~ [Merged by Bors] - Add Exponential Moving Average into diagnostics Oct 24, 2022

bors bot closed this Oct 24, 2022

CAD97 deleted the ema branch October 24, 2022 14:58

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Merged by Bors] - Add Exponential Moving Average into diagnostics #4992

[Merged by Bors] - Add Exponential Moving Average into diagnostics #4992

CAD97 commented Jun 11, 2022 •

edited

CAD97 commented Jun 13, 2022 •

edited

CAD97 commented Jun 13, 2022

alice-i-cecile left a comment

hymm left a comment

CAD97 commented Oct 21, 2022

alice-i-cecile commented Oct 24, 2022

bors bot commented Oct 24, 2022

[Merged by Bors] - Add Exponential Moving Average into diagnostics #4992

[Merged by Bors] - Add Exponential Moving Average into diagnostics #4992

Conversation

CAD97 commented Jun 11, 2022 • edited

Objective

Terms

Solution

Changelog

Added

Changed

CAD97 commented Jun 13, 2022 • edited

Footnotes

CAD97 commented Jun 13, 2022

alice-i-cecile left a comment

Choose a reason for hiding this comment

hymm left a comment

Choose a reason for hiding this comment

CAD97 commented Oct 21, 2022

alice-i-cecile commented Oct 24, 2022

bors bot commented Oct 24, 2022

CAD97 commented Jun 11, 2022 •

edited

CAD97 commented Jun 13, 2022 •

edited