Table virtualizer doesn't render enough rows after a height increase, until a scroll

[rajsite]

🧹 Tech Debt

It looks like on a fresh load of a component the virtualizer is miscounting the space and creating more rows than necessary. This isn't user facing (extra rows outside the viewport are not visible) but may be an unnecessary performance cost. We may find it is a minor cost, I think the page size may be used or something, should verify we never i.e. instance every row in the view unnecessarily for a large data set.

Found during #1000 where a workaround was added for a test to nudge the scroll so the correct, smaller amount of rows ins rendered in the viewport:

nimble/packages/nimble-components/src/table/tests/table.spec.ts

Line 460 in 5902e83

element.viewport.scrollTop = 2;

Additional discussion: https://github.com/ni/nimble/pull/1000/files#r1089556479

[msmithNI]

This looks like it could be a user-visible bug.
To reproduce, open the table story in Storybook, use the large data set. Change table height to 150px in DevTools. Then change table height to something bigger like 400px (copy/paste in the new height so it happens in a single frame, instead of per character as you type). The # of rows will be wrong (too small for the new size) until you scroll.

It does look to be related to the dependencies update somehow. Those same steps don't reproduce the issue in this Storybook from my virtualization PR (before the dependency updates went in).

[msmithNI]

FYI @rajsite the dependencies PR did result in a new TanStack Virtual version getting pulled in since we don't pin that version currently (3.0.0-beta.34 => 3.0.0-beta.41).

I was able to trace this back to a TanStack Virtual change. The issue is reproducible without our code so I made a test case and filed them a bug - TanStack/virtual#492

[rajsite]

Interesting. it looks like the 3.0 prereleases have been going on for almost exactly a year https://github.com/TanStack/virtual/releases/tag/v3.0.0-alpha.0

Do we treat it like regular releases if they are in perpetual prerelease (and keep the semver range as is and handle breaks) or pin it and do controlled updates between prerelease versions? We don't have renovate on and our manual bump cadence is infrequent so not sure I feel strongly either way.

Do we pin at a lower version that doesn't include the regression until the issue is fixed?

[msmithNI]

I'm not sure if they have a timeframe for v3 being non-beta. Their bug template said "v3 on the horizon with beta builds" but as you said that's been true for a pretty long time now.

I'll probably give them the weekend to see if there's any movement on the bug I filed / see what they say. We can pin if needed after a few days.