[DataGrid] Server-side pagination for an unknown number of items

[thomasdowell]

To enable server-side pagination for an unknown number of items in the TablePagination component, the value of the count prop must be -1. When the value of the DataGrid component's rowCount prop is set to -1, the server-side pagination functionality is not complete.

Current Behavior 😯

"Total rows: -1" is rendered and the onPageChange function is not triggered.

Expected Behavior 🤔

"Total rows: -1" should not render and the onPageChange function should trigger until there are no more pages to get from the server.

Steps to Reproduce 🕹

Steps:

1. Go to this CodeSandbox which demos this material-ui example
2. Navigate to demo.js
3. Change rowCount={100} to rowCount={-1} on line 55

Context 🔦

Server-side pagination

Benchmark

• https://www.ag-grid.com/javascript-grid/server-side-model-pagination/#server-pagination

[oliviertassinari]

@thomasdowell Thanks for the feature request. The grid doesn't support the unknown last row case yet. This request is close to #404 but for when pagination=true.

[ahurlburt]

Are there any known workarounds to this? I'm using DynamoDB and item count is either estimated (to get quickly) or expensive to get accurately. It would be nice if the data grid just allowed the next page to load and disable showing the total count in this case.

I see the infinite scroll for client side was marked important and closed, @oliviertassinari I see you marked that issue as important. Seeing dynamo's (and nosql in general) growth in popularity I would assume many people are going to start running into this issue (unless there is a valid workaround I'm not aware of)?

[oliviertassinari]

I'm using DynamoDB and item count is either estimated (to get quickly) or expensive to get accurately.

@ahurlburt I had the same experience with PostgreSQL after 1m rows on a table.

Regarding the possible solution of this problem, I think that there are two different paths that we could follow (complementary):

• Option 1. We make it work with the props only. Meaning, we can ask developers to set rowCount={-1} when the don't know how many rows they have, then use the number of rows in the rows prop (if server-side mode) and the size of the page to know where the displayed rows are, relative to the full data set.
• Option 2. We add the notion of a data fetcher, something that makes it easy to manipulate the state. I saw this implemented in AG Grid https://www.ag-grid.com/javascript-grid/server-side-model-pagination/#server-pagination. We could also make it work, maybe with an optional hook that extends the existing API (either props or the imperative API directly for performance). If we do this, I think that we should be cautious and double check that react-query or raw React can't already solve the problem nicely.

This duality of choices is similar to what @DanailH is working on for #1247. He started by building the missing API for option 1 (not the same issue as we talk here)

[Bersaelor]

I'm also working on at least two use cursor-based pagination cases where I don't know the total amount of rows, until I hit the last page. (One is dynamodb)

This is the main reason why getting DataGridPro hasn't been worth it for me.

[jspasiuk]

This issue is affecting me as well!

Right now can't use the data-grid because of this limitation about the row count. I had to make a custom list with infinite scroll.

+1

[ibrahimhajjaj]

Same here folks, the total items is unknown till reaching that last page

[leotuna]

Workaround 👇

<DataGrid
  {...props}
  sx={{
    '.MuiTablePagination-displayedRows': {
      display: 'none', // 👈 to hide huge pagination number
    },
  }}
  paginationMode="server"
  rowCount={Number.MAX_VALUE} // 👈 to maximize pages
/>

[adamrneary]

Even for teams not using Relay, the Relay pagination spec is a reliable and consistent spec for cursor-based pagination: https://relay.dev/docs/guided-tour/list-data/pagination

We don't use Relay, but our pagination interface matches this, and it would be great if <DataGrid /> was able to consume it directly. Very specifically:

<DataGrid
  {...props}
  paginationMode="server"
  pageSize={pageSize}
  onPageSizeChange={(newPageSize) => setPageSize(newPageSize)}
  rowsPerPageOptions={[10, 25, 100]}
  hasPreviewPage={hasPreviousPage}
  onPreviousPage={fetchPreviousPage}
  hasNextPage={hasNextPage}
  onNextPage={fetchNextPage}
/>

This world speaks to many of the commenters above, where you don't have a total row count and you might not even know what page you're on, but you're getting n rows back and forward/backward cursors that match the above.

UI wise, this means we aren't showing MuiTablePagination-displayedRows (in sync with @leotuna 's workaround right above). Just forward and backward chevrons and a page size select.

Just a thought — hope it helps!

[adamrneary]

In the spirit of temporary workarounds, this hook manages to encapsulate the logic we need to apply in the meantime — in case others find it useful!

export function useGridPagination() {
  const [page, setPage] = useState(0);
  const [pageSize, setPageSize] = useState(10);

  const {
    paginationState,
    fetchPreviousPage,
    fetchNextPage,
    resetPaginationState,
  } = usePagination(pageSize);

  useEffect(() => {
    if ((paginationState.first ?? paginationState.last) !== pageSize) {
      resetPaginationState();
    }
  }, [paginationState, pageSize, resetPaginationState]);

  const handlePageChange = useCallback(
    (newPage: number, pageInfo?: GQPageInfo | null) => {
      if (!pageInfo) return;
      setPage(newPage);
      if (newPage <= page) {
        fetchPreviousPage(pageInfo);
      } else {
        fetchNextPage(pageInfo);
      }
    },
    [fetchNextPage, fetchPreviousPage, page]
  );

  const gridPaginationProps = useCallback(
    (pageInfo?: GQPageInfo | null) => ({
      pagination: true,
      paginationMode: 'server' as 'server',
      pageSize,
      onPageSizeChange: (newPageSize: number) => setPageSize(newPageSize),
      rowsPerPageOptions: [10, 25, 100],
      page,
      rowCount: pageInfo?.hasNextPage
        ? Number.MAX_VALUE
        : pageSize * (pageInfo?.hasPreviousPage ? 2 : 1),

      onPageChange: (newPage: number) => handlePageChange(newPage, pageInfo),
    }),
    [handlePageChange, page, pageSize]
  );

  return {
    paginationState,
    gridPaginationProps,
  };
}

Then the grid just needs the following:

<DataGrid
  rows={rows}
  columns={columns}
  {...gridPaginationProps(pageInfo)}
  sx={{
    '.MuiTablePagination-displayedRows': {
      display: 'none', // 👈 to hide huge pagination number
     },
  }}
/>

[clarkmcc]

It's even more hacky, but I just did

<DataGrid
  rowCount={9999999999}
  sx={{
    ".MuiTablePagination-displayedRows": {
      display: "none",
    },
  }}
/>

[VladyslavBondarenko]

Workaround 👇

<DataGrid
  {...props}
  sx={{
    '.MuiTablePagination-displayedRows': {
      display: 'none', // 👈 to hide huge pagination number
    },
  }}
  paginationMode="server"
  rowCount={Number.MAX_VALUE} // 👈 to maximize pages
/>

Would be perfect to have a custom render for the rowCount. I'd like to have the word many there (1-10 of many) unless it's the last page (so the number of rows is known).

Edit:
Found https://mui.com/x/react-data-grid/localization/#translation-keys,
solved with

localeText={{
  MuiTablePagination: {
    labelDisplayedRows: ({ from, to, count }) =>
      `${from} - ${to} of ${
        count === Number.MAX_VALUE ? "many" : count
      }`
  }
}}

With the Number.MAX_VALUE, the "next" button remains enabled on the last page, need to handle this case.

[github-actions]

⚠️ This issue has been closed. If you have a similar problem but not exactly the same, please open a new issue.
Now, if you have additional information related to this issue or things that could help future readers, feel free to leave a comment.

@thomasdowell: How did we do? Your experience with our support team matters to us. If you have a moment, please share your thoughts in this short Support Satisfaction survey.