[DataGrid] Dynamic virtualization range #12353

romgrk · 2024-03-06T19:50:14Z

Implement both improvements proposed in the linked issue. Read the comments below for more implementation details.

Before: https://deploy-preview-12349--material-ui-x.netlify.app/x/react-data-grid/#pro-plan
After: https://deploy-preview-12353--material-ui-x.netlify.app/x/react-data-grid/#pro-plan

Before/After:

Kooha-2024-03-10-20-48-32.webm

mui-bot · 2024-03-06T19:54:57Z

Deploy preview: https://deploy-preview-12353--material-ui-x.netlify.app/

Updated pages:

Generated by 🚫 dangerJS against a5db4a4

romgrk · 2024-03-07T14:51:23Z

packages/x-data-grid/src/hooks/features/virtualization/useGridVirtualScroller.tsx

+      const dy = scrollPosition.current.top - previousScroll.current.top;
+      const dx = scrollPosition.current.left - previousScroll.current.left;
+      scrollCache.buffer = {
+        top: dimensions.rowHeight * 10,
+        left: 50 * 3,
+      };


I tried predicting* the optimal buffer size but failed, so I just put a static 10 rows or 3 columns buffer.

*I had code in place to collect scroll points as event, which I hoped we could run some math on to figure out where the next scroll event would be. I tried linear regression, polynomial regression, and even a small (<1kb) ML model (simple perceptron architecture) with synaptic.js, but the problem is the data. One would expect scroll events to mostly fit to a bell curve. Which they do as a whole. But the intervals and delta values are just too irregular from one point to the next to be able to predict the next point in a sequence. I have gathered some data (me scrolling the grid) and plotted it here, the last graph is an example of "generally like a bell curve but unpredictable from one point to the next". Maybe with wheel events added in the mix we could get a better error rate (best I could get was 50% error on average), but that will take some more research.

I can't access the link attached. Could you update the permissions?

romgrk · 2024-03-09T01:23:21Z

packages/x-data-grid/src/models/props/DataGridProps.ts

+   * Column region to render before/after the viewport
+   * @default 100
   */
-  columnBuffer: number;
+  columnBufferPx: number;
  /**
-   * Number of extra rows to be rendered before/after the visible slice.
-   * @default 3
+   * Row region to render before/after the viewport
+   * @default 100
   */
-  rowBuffer: number;
+  rowBufferPx: number;


I don't love the new name with a Px suffix, but leaving the name as is didn't feel like a good option because it would silently make the grid less performant for the users using this prop.

rowBufferRegion could be another option, JSDoc can explain the unit of the region (px)

romgrk · 2024-03-09T01:43:19Z

packages/x-data-grid/src/hooks/features/virtualization/useGridVirtualScroller.tsx

+enum ScrollDirection {
+  NONE,
+  UP,
+  DOWN,
+  LEFT,
+  RIGHT,
+}
+
+const EMPTY_SCROLL_CACHE = {
+  direction: ScrollDirection.NONE,
+  lastTimestamp: -1,
+  lastPosition: { top: 0, left: 0 },
+  buffer: { top: 0, left: 0 },
+};
+type ScrollCache = typeof EMPTY_SCROLL_CACHE;


The general idea for the logic is that scroll events trigger right after a new frame has been rendered, so too late for us to prevent white areas that way. If we want to prevent white areas, we need to diplay enough rows/columns so that there is enough of them to cover the area from the current viewport to where the viewport will be at the next scroll event.

As mentioned above predicting by how much the user will have scrolled at the next scroll event is a problem is itself, but by having a substantially large buffer in one direction we try to ensure users see as few white areas as possible.

MBilalShafi

Nice improvement 🚀
Looks smoother

packages/x-data-grid/src/hooks/features/virtualization/useGridVirtualScroller.tsx

packages/x-data-grid/src/DataGrid/DataGrid.tsx

MBilalShafi · 2024-03-09T05:32:38Z

packages/x-data-grid/src/models/props/DataGridProps.ts

+   * Column region to render before/after the viewport
+   * @default 100
   */
-  columnBuffer: number;
+  columnBufferPx: number;
  /**
-   * Number of extra rows to be rendered before/after the visible slice.
-   * @default 3
+   * Row region to render before/after the viewport
+   * @default 100
   */
-  rowBuffer: number;
+  rowBufferPx: number;


rowBufferRegion could be another option, JSDoc can explain the unit of the region (px)

docs/data/data-grid/demo/FullFeaturedDemo.js

docs/data/migration/migration-data-grid-v6/migration-data-grid-v6.md

github-actions · 2024-03-10T06:53:46Z

This pull request has conflicts, please resolve those before we can evaluate the pull request.

packages/x-data-grid/src/hooks/features/virtualization/useGridVirtualScroller.tsx

docs/data/data-grid/virtualization/virtualization.md

docs/data/migration/migration-data-grid-v6/migration-data-grid-v6.md

packages/x-data-grid/src/hooks/features/virtualization/useGridVirtualScroller.tsx

…-v6.md Co-authored-by: Andrew Cherniavskii <andrew.cherniavskii@gmail.com> Signed-off-by: Rom Grk <romgrk@users.noreply.github.com>

…nto feat-virtualization-range

packages/x-data-grid/src/hooks/features/virtualization/useGridVirtualScroller.tsx

github-actions · 2024-03-17T00:30:10Z

This pull request has conflicts, please resolve those before we can evaluate the pull request.

github-actions · 2024-03-19T14:22:54Z

This pull request has conflicts, please resolve those before we can evaluate the pull request.

romgrk added 2 commits March 6, 2024 14:03

feat: pixel buffers instead of item buffers

9043276

feat: scroll position prediction

4dadf5c

romgrk added the component: data grid This is the name of the generic UI component, not the React module! label Mar 6, 2024

romgrk added 3 commits March 6, 2024 19:48

feat: predict with linear regression

8742d37

feat: more simple buffer

82e24c2

lint

036fc26

romgrk commented Mar 7, 2024

View reviewed changes

romgrk added 3 commits March 7, 2024 10:28

Merge branch 'next' into feat-virtualization-range

3453534

ci: run

54564fb

feat: finish buffer changes

3d73f8f

romgrk marked this pull request as ready for review March 9, 2024 01:21

romgrk commented Mar 9, 2024

View reviewed changes

romgrk added 2 commits March 8, 2024 20:27

doc: update guide

272ae19

lint

f49842e

romgrk commented Mar 9, 2024

View reviewed changes

lint

7a5cb7b

romgrk requested review from cherniavskii and MBilalShafi March 9, 2024 01:45

romgrk mentioned this pull request Mar 9, 2024

[DataGrid] Performance: Improve virtualization overscan logic #11344

Closed

romgrk added 2 commits March 8, 2024 20:51

lint

3d69375

lint

1a8f08d

MBilalShafi approved these changes Mar 9, 2024

View reviewed changes

github-actions bot added the PR: out-of-date The pull request has merge conflicts and can't be merged label Mar 10, 2024

Merge branch 'next' into feat-virtualization-range

7171c7c

github-actions bot removed the PR: out-of-date The pull request has merge conflicts and can't be merged label Mar 10, 2024

feat: render context rework

239c512

romgrk commented Mar 11, 2024

View reviewed changes

packages/x-data-grid/src/hooks/features/virtualization/useGridVirtualScroller.tsx Show resolved Hide resolved

lint

241b730

cherniavskii reviewed Mar 13, 2024

View reviewed changes

romgrk and others added 4 commits March 13, 2024 11:36

Update docs/data/migration/migration-data-grid-v6/migration-data-grid…

df5f06b

…-v6.md Co-authored-by: Andrew Cherniavskii <andrew.cherniavskii@gmail.com> Signed-off-by: Rom Grk <romgrk@users.noreply.github.com>

Merge branch 'next' into feat-virtualization-range

156d9fe

Merge branch 'feat-virtualization-range' of github.com:romgrk/mui-x i…

27c77d7

…nto feat-virtualization-range

lint

ca76f30

github-actions bot removed the PR: out-of-date The pull request has merge conflicts and can't be merged label Mar 14, 2024

romgrk added 4 commits March 14, 2024 03:10

refactor: column width as constant

ab1052c

lint

f489228

lint

1b5ef93

lint

b492aac

cherniavskii approved these changes Mar 14, 2024

View reviewed changes

cherniavskii added performance v7.x labels Mar 14, 2024

romgrk added 3 commits March 14, 2024 17:53

Merge branch 'next' into feat-virtualization-range

ee8c402

test: disable virtualization in jsdom

9ac297b

lint

d43b2a8

romgrk commented Mar 15, 2024

View reviewed changes

packages/x-data-grid/src/hooks/features/virtualization/useGridVirtualScroller.tsx Show resolved Hide resolved

github-actions bot added the PR: out-of-date The pull request has merge conflicts and can't be merged label Mar 17, 2024

romgrk added 2 commits March 19, 2024 07:05

fix: async autosizing demo

97defb7

Merge branch 'next' into feat-virtualization-range

6704b08

github-actions bot removed the PR: out-of-date The pull request has merge conflicts and can't be merged label Mar 19, 2024

lint

4d45e22

github-actions bot added the PR: out-of-date The pull request has merge conflicts and can't be merged label Mar 19, 2024

Merge branch 'next' into feat-virtualization-range

93d9bbd

github-actions bot removed the PR: out-of-date The pull request has merge conflicts and can't be merged label Mar 20, 2024

lint

a5db4a4

romgrk merged commit 1418ac2 into mui:next Mar 20, 2024
17 checks passed

romgrk deleted the feat-virtualization-range branch March 20, 2024 06:05

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[DataGrid] Dynamic virtualization range #12353

[DataGrid] Dynamic virtualization range #12353

romgrk commented Mar 6, 2024 •

edited by oliviertassinari

mui-bot commented Mar 6, 2024 •

edited

romgrk Mar 7, 2024

MBilalShafi Mar 9, 2024

romgrk Mar 9, 2024

romgrk Mar 9, 2024 •

edited

MBilalShafi Mar 9, 2024

romgrk Mar 9, 2024

MBilalShafi left a comment

MBilalShafi Mar 9, 2024

github-actions bot commented Mar 10, 2024

github-actions bot commented Mar 17, 2024

github-actions bot commented Mar 19, 2024

[DataGrid] Dynamic virtualization range #12353

[DataGrid] Dynamic virtualization range #12353

Conversation

romgrk commented Mar 6, 2024 • edited by oliviertassinari

mui-bot commented Mar 6, 2024 • edited

romgrk Mar 7, 2024

Choose a reason for hiding this comment

MBilalShafi Mar 9, 2024

Choose a reason for hiding this comment

romgrk Mar 9, 2024

Choose a reason for hiding this comment

romgrk Mar 9, 2024 • edited

Choose a reason for hiding this comment

MBilalShafi Mar 9, 2024

Choose a reason for hiding this comment

romgrk Mar 9, 2024

Choose a reason for hiding this comment

MBilalShafi left a comment

Choose a reason for hiding this comment

MBilalShafi Mar 9, 2024

Choose a reason for hiding this comment

github-actions bot commented Mar 10, 2024

github-actions bot commented Mar 17, 2024

github-actions bot commented Mar 19, 2024

romgrk commented Mar 6, 2024 •

edited by oliviertassinari

mui-bot commented Mar 6, 2024 •

edited

romgrk Mar 9, 2024 •

edited