# Lazy Loading

Canonical page: https://infinite-table.com/docs/learn/working-with-data/lazy-loading

With `InfiniteTable` you can lazily load data on demand - loading data is triggered by the user scrolling to a certain visible row range. So when the user stopped scrolling (after [`scrollStopDelay`](https://infinite-table.com/docs/reference/infinite-table-props.md#scrollStopDelay) ms passed), the `DataSource` is loading the records that are in the viewport. Also, the table will render as if all the remote data is loaded into viewport - so the scroll height is correspondingly set.

We call this `"lazy loading"`, and it needs to be enabled by specifying the [DataSource.lazyLoad](https://infinite-table.com/docs/reference/datasource-props/index.md#lazyLoad) prop.

**Example: Lazy loading ungrouped and unpivoted data**

```ts
import '@infinite-table/infinite-react/index.css';
import {
  InfiniteTable,
  DataSource,
  DataSourceData,
  InfiniteTablePropColumns,
} from '@infinite-table/infinite-react';
import * as React from 'react';
import { useMemo } from 'react';

type Developer = {
  id: number;
  firstName: string;
  lastName: string;
  country: string;
  city: string;
  currency: string;
  preferredLanguage: string;
  stack: string;
  canDesign: 'yes' | 'no';
  hobby: string;
  salary: number;
  age: number;
};

const columns: InfiniteTablePropColumns<Developer> = {
  id: { field: 'id', header: 'ID', defaultWidth: 100 },
  salary: { field: 'salary', header: 'Salary' },
  age: { field: 'age', header: 'Age' },
  firstName: { field: 'firstName', header: 'First Name' },
  preferredLanguage: {
    field: 'preferredLanguage',
    header: 'Preferred Language',
  },
  lastName: { field: 'lastName', header: 'Last Name' },
  country: { field: 'country', header: 'Country' },
  city: { field: 'city', header: 'City' },
  currency: { field: 'currency', header: 'Currency' },
  stack: { field: 'stack', header: 'Stack' },
  canDesign: { field: 'canDesign', header: 'Can Design' },
  hobby: { field: 'hobby', header: 'Hobby' },
};

export default function App() {
  const lazyLoad = useMemo(() => ({ batchSize: 40 }), []);
  return (
    <DataSource<Developer>
      data={dataSource}
      primaryKey="id"
      lazyLoad={lazyLoad}
    >
      <InfiniteTable<Developer>
        debugId="simple-lazy-load-example"
        columns={columns}
        columnDefaultWidth={130}
      />
    </DataSource>
  );
}

const dataSource: DataSourceData<Developer> = ({
  pivotBy,
  aggregationReducers,
  groupBy,

  lazyLoadStartIndex,
  lazyLoadBatchSize,
  groupKeys = [],
  sortInfo,
}) => {
  if (sortInfo && !Array.isArray(sortInfo)) {
    sortInfo = [sortInfo];
  }
  const startLimit: string[] = [];
  if (lazyLoadBatchSize && lazyLoadBatchSize > 0) {
    const start = lazyLoadStartIndex || 0;
    startLimit.push(`start=${start}`);
    startLimit.push(`limit=${lazyLoadBatchSize}`);
  }
  const args = [
    ...startLimit,
    pivotBy
      ? 'pivotBy=' + JSON.stringify(pivotBy.map((p) => ({ field: p.field })))
      : null,
    `groupKeys=${JSON.stringify(groupKeys)}`,
    groupBy
      ? 'groupBy=' + JSON.stringify(groupBy.map((p) => ({ field: p.field })))
      : null,
    sortInfo
      ? 'sortInfo=' +
        JSON.stringify(
          sortInfo.map((s) => ({
            field: s.field,
            dir: s.dir,
          })),
        )
      : null,
    aggregationReducers
      ? 'reducers=' +
        JSON.stringify(
          Object.keys(aggregationReducers).map((key) => ({
            field: aggregationReducers[key].field,
            id: key,
            name: aggregationReducers[key].reducer,
          })),
        )
      : null,
  ]
    .filter(Boolean)
    .join('&');
  return fetch(
    process.env.NEXT_PUBLIC_BASE_URL + `/developers10k-sql?` + args,
  ).then((r) => r.json());
};
```

The [DataSource.lazyLoad](https://infinite-table.com/docs/reference/datasource-props/index.md#lazyLoad) prop can be either a boolean or an object with a `batchSize: number` property. If `batchSize` is not specified, it will load all records from the current row group (makes sense for grouped and/or pivoted data). For ungrouped and unpivoted data, make sure you set `batchSize` to a conveninent number.

Simply specifying `lazyLoad=true` makes more sense for grouped (or/and pivoted) data, where you want to load all records from the current level at once. If you want configure it this way, new data will only be requested when a group row is expanded.

For lazy loading to work, the [`data`](https://infinite-table.com/docs/reference/datasource-props/index.md#data) function in the `<DataSource/>` component must return a Promise that resolves to an an object with `data` and `totalCount` properties.

```tsx
{
  data: [ ... ],
  totalCount: 10000
}
```

The [DataSource.data](https://infinite-table.com/docs/reference/datasource-props/index.md#data) function will be called with an object with the following properties:

- `sortInfo` - details about current sorting state
- `pivotBy` - an array that describes the current pivot state
- `aggregationReducers` - an object with the aggregation to apply to the data
- `groupBy` - array that specifies the current grouping information
- `groupKeys` - an array of the current group keys (if grouping is enabled). This uniquely identifies the current group.

- `lazyLoadStartIndex` - the index (in the total remote datasource) of the first record to be loaded
- `lazyLoadBatchSize` - the number of records to be loaded in this batch

Find out about server-side grouping

Find out about server-side pivoting

## How lazy loading fetches data

When lazy loading is enabled, and the [`sortInfo`](https://infinite-table.com/docs/reference/datasource-props/index.md#sortInfo)  changes (eg: user clicks on a column header), the DataGrid will discard current data and call the [`data`](https://infinite-table.com/docs/reference/datasource-props/index.md#data) function prop again, to fetch the new data. The same happens when the [`filterValue`](https://infinite-table.com/docs/reference/datasource-props/index.md#filterValue) or [`groupBy`](https://infinite-table.com/docs/reference/datasource-props/index.md#groupBy) changes. This is done automatically by the component, and you don't need to do anything.

**Example: Lazy loading grouped data**

This demo lazily loads grouped data as the user scrolls down. Expand some groups to see the lazy loading in action.

When the user stops scrolling, after [`scrollStopDelay`](https://infinite-table.com/docs/reference/infinite-table-props.md#scrollStopDelay) milliseconds, the DataGrid will fetch the next batch of data from the server.

```ts
import '@infinite-table/infinite-react/index.css';
import {
  InfiniteTable,
  DataSource,
  DataSourceData,
  InfiniteTablePropColumns,
  DataSourceProps,
} from '@infinite-table/infinite-react';
import * as React from 'react';
import { useMemo } from 'react';

type Developer = {
  id: number;
  firstName: string;
  lastName: string;
  country: string;
  city: string;
  currency: string;
  preferredLanguage: string;
  stack: string;
  canDesign: 'yes' | 'no';
  hobby: string;
  salary: number;
  age: number;
};

const columns: InfiniteTablePropColumns<Developer> = {
  country: { field: 'country', header: 'Country' },
  id: { field: 'id', header: 'ID', defaultWidth: 100 },
  salary: {
    field: 'salary',
    header: 'Salary',
    renderValue: ({ value, rowInfo }) => {
      if (rowInfo.isGroupRow) {
        return (
          <>
            Avg: <b>{value}</b>
          </>
        );
      }
      return value;
    },
  },
  age: { field: 'age', header: 'Age' },
  firstName: { field: 'firstName', header: 'First Name' },
  preferredLanguage: {
    field: 'preferredLanguage',
    header: 'Preferred Language',
  },
  lastName: { field: 'lastName', header: 'Last Name' },

  city: { field: 'city', header: 'City' },
  currency: { field: 'currency', header: 'Currency' },
  stack: { field: 'stack', header: 'Stack' },
  canDesign: { field: 'canDesign', header: 'Can Design' },
  hobby: { field: 'hobby', header: 'Hobby' },
};

const groupBy: DataSourceProps<Developer>['groupBy'] = [
  {
    field: 'country',
  },
  {
    field: 'stack',
  },
];

const aggregationReducers: DataSourceProps<Developer>['aggregationReducers'] = {
  salary: {
    field: 'salary',
    reducer: 'avg',
  },
};

export default function App() {
  const lazyLoad = useMemo(() => ({ batchSize: 40 }), []);
  return (
    <DataSource<Developer>
      data={dataSource}
      primaryKey="id"
      groupBy={groupBy}
      lazyLoad={lazyLoad}
      aggregationReducers={aggregationReducers}
    >
      <InfiniteTable<Developer>
        debugId="grouped-lazy-load-example"
        columns={columns}
        columnDefaultWidth={130}
        groupColumn={{
          id: 'group-col',
          defaultSortable: false,
        }}
        groupRenderStrategy="single-column"
      />
    </DataSource>
  );
}

const dataSource: DataSourceData<Developer> = ({
  pivotBy,
  aggregationReducers,
  groupBy,

  lazyLoadStartIndex,
  lazyLoadBatchSize,
  groupRowsState,
  groupKeys = [],
  sortInfo,
}) => {
  if (sortInfo && !Array.isArray(sortInfo)) {
    sortInfo = [sortInfo];
  }
  const startLimit: string[] = [];
  if (lazyLoadBatchSize && lazyLoadBatchSize > 0) {
    const start = lazyLoadStartIndex || 0;
    startLimit.push(`start=${start}`);
    startLimit.push(`limit=${lazyLoadBatchSize}`);
  }
  const args = [
    ...startLimit,
    pivotBy
      ? 'pivotBy=' + JSON.stringify(pivotBy.map((p) => ({ field: p.field })))
      : null,
    `groupKeys=${JSON.stringify(groupKeys)}`,
    `prefetchGroupKeys=${JSON.stringify(groupRowsState?.expandedRows || [])}`,
    groupBy
      ? 'groupBy=' + JSON.stringify(groupBy.map((p) => ({ field: p.field })))
      : null,
    sortInfo
      ? 'sortInfo=' +
        JSON.stringify(
          sortInfo.map((s) => ({
            field: s.field,
            dir: s.dir,
          })),
        )
      : null,
    aggregationReducers
      ? 'reducers=' +
        JSON.stringify(
          Object.keys(aggregationReducers).map((key) => ({
            field: aggregationReducers[key].field,
            id: key,
            name: aggregationReducers[key].reducer,
          })),
        )
      : null,
  ]
    .filter(Boolean)
    .join('&');
  return fetch(
    process.env.NEXT_PUBLIC_BASE_URL + `/developers10k-sql?` + args,
  ).then((r) => r.json());
};
```

Batching also happens for groups - when a group is expanded, the DataGrid will fetch the first batch of data in the expanded group and then fetch additional batches as the user scrolls down. When scrolling goes beyound the group, the DataGrid is smart enough to request a batch of data from sibling groups.

Lazy loading when grouping is enabled needs data for non-leaf rows to be in another format (as opposed to the format used for non-grouped data or for the non-grouped scenario). See example above for details.

For more docs on this, read [Server side grouping with lazy loading](https://infinite-table.com/docs/learn/grouping-and-pivoting/grouping-rows.md#server-side-grouping-with-lazy-loading).