Infinite Scroll Implementation in Vanilla-Grid

This document explains how infinite scroll is implemented in Vanilla-Grid, including prefetch thresholds, load chaining, known-total resolution, scrollbar interactions, and integration with virtualization.

1. Feature Scope

Infinite scroll provides:

Relevant options:

2. Internal State Model

Core state fields:

3. Initialization and Reset Behavior

Constructor sets defaults and callback references.

setRows(rows) always sets _loadedRowCount = rows.length first, regardless of infiniteScroll — getLoadedRowCount() (and callers built on it, like <vn-grid-toolbar-status>'s "Loaded X/Y" summary) must report a correct count even for a plain, non-paginated load, where the DataManager returns every matching row in one shot.

It then resets the remaining infinite-scroll runtime state when infiniteScroll is enabled:

  1. _hasMoreData = true
  2. _isLoadingMore = false
  3. Cancel _loadMoreShimmerTimer; set _loadMoreSkeletonVisible = false
  4. If _totalRowCount exists, _hasMoreData = _loadedRowCount < _totalRowCount

When infiniteScroll is disabled, _hasMoreData is simply forced to false — a single-shot load has nothing left to fetch for the current query.

This ensures a fresh baseline whenever host code replaces data.

setRows(rows) also unconditionally resets the viewport scroll position to the top:

Rationale: when a server-side sort (or any other operation) fully replaces the dataset via setRows, the browser may clamp the stale scrollTop to the new (smaller) scroll range instead of resetting to zero. Without an explicit reset, renderVisibleRows starts from the clamped position, showing rows from the middle of the new dataset rather than row 0. This is especially visible with tall viewports and large datasets loaded over multiple infinite-scroll pages.

The tracking variables (lastScrollTop, lastProgrammaticScrollTop, lastProgrammaticScrollAt) are zeroed in tandem so the browser's asynchronous scroll event fired after the write is treated as a zero-delta no-op and does not trigger a redundant re-render or confuse the momentum guard.

Note: appendRows (used by the infinite-scroll load-more path) does not reset scroll position, since appending more rows must preserve the user's current scroll position.

_resetInfiniteScrollState() (invoked by reloadDataManager() before a filter/sort/reload re-query) clears the same bookkeeping — _loadedRowCount = 0, empty rows/displayRows, _totalRowCount = null, backoff counters — and sets _hasMoreData = !!infiniteScroll. It is not forced true: a non-infinite-scroll (static/in-memory) grid never has more to scroll-load, so seeding it false during the reset→requery window keeps hasMoreRows() from briefly returning true and flashing a (scroll to load more) status hint (in <vn-grid-toolbar-status>) while the query is in flight. Infinite-scroll grids still start the window true, exactly as a fresh setRows() does.

4. Prefetch Trigger in Render Loop

Prefetch is initiated from renderVisibleRows(...) after virtualization updates.

Guard conditions required:

Threshold logic:

So prefetch starts when roughly the last 30% of the currently loaded page is reached.

5. Core Load Pipeline (_loadMoreIfNeeded)

_loadMoreIfNeeded(force = false) does the actual fetch orchestration.

5.1 Entry guards

Returns immediately if any of these is true:

Additionally, when not forced:

5.2 Fetch + append flow

  1. Call _setLoadingMore(true) — sets _isLoadingMore and adds the vn-grid-loading-more container class (§10)
  2. If loadMoreShimmerDelay >= 0, schedule _showLoadMoreBanner() + _showLoadMoreSkeletons() after the delay
  3. Await onLoadMore(_loadedRowCount, pageSize)
  4. On resolve:
    • cancel shimmer timer; call _hideLoadMoreSkeletons()
    • if returned rows are non-empty:
      • set _suppressPrefetch = true
      • call appendRows(newRows)
      • increment _loadedRowCount += newRows.length
      • clear _suppressPrefetch
  5. Update _hasMoreData:
    • if _totalRowCount exists: compare against _loadedRowCount
    • else if returned rows < pageSize: assume end reached (false)
  6. If result is empty/non-array: _hasMoreData = false
  7. Invoke the optional infiniteScroll.onLoadMoreSuccess(newRows, { loadedRowCount, hasMoreRows }) callback (fires even when newRows is empty). appendRows() itself dispatches no DOM event, so on <vn-grid> this is auto-wired to also dispatch vn-grid-load-more-succeeded (VanillaGridEvents.LOAD_MORE_SUCCEEDED) — the only signal that reflects an incremental, scroll-triggered append (LOADED only fires for the initial load / reload path).

Errors cancel the shimmer timer, hide skeleton rows, are logged, and do not crash the grid; retry is possible on future triggers (subject to the failure backoff, §5.4). A successful load resets the backoff (_loadMoreFailureCount = 0, _loadMoreRetryAt = 0). Whatever the outcome, .finally() calls _setLoadingMore(false), which clears the class and the banner along with the flag.

infiniteScroll.shimmerDelay gates both built-in affordances — the banner and the tail skeleton rows — so a fast backend paints neither, and a negative value ("never show") means no built-in affordance at all. The vn-grid-loading-more class is never delayed.

5.3 End-of-track page chaining

In finally, if _isHoldingScrollbarAtBottom() is true and the failure backoff gate is open:

  1. Set scrollTop to maximum
  2. Update scrollbar thumb
  3. Re-enter _loadMoreIfNeeded(true)

This allows chained page loading while the user keeps dragging at the end. The backoff check prevents the chain from turning into a request storm when the thumb is held at the bottom against a failing backend.

5.4 Failure backoff (_loadMoreFailureCount / _loadMoreRetryAt)

Every rejected onLoadMore fetch:

  1. Increments _loadMoreFailureCount.
  2. Computes an exponential backoff: min(30000, 1000 × 2^(failureCount − 1)) — 1 s, 2 s, 4 s, … capped at 30 s.
  3. Sets _loadMoreRetryAt = Date.now() + backoff. Non-forced calls return immediately while the gate is closed; user-forced retries (force === true, e.g. dropping the scrollbar thumb at the bottom) bypass the gate but still count failures.
  4. Invokes the optional infiniteScroll.onLoadMoreError(error, { failureCount, retryDelayMs }) callback. On <vn-grid> this is auto-wired to dispatch the vn-grid-load-more-failed CustomEvent (VanillaGridEvents.LOAD_MORE_FAILED, detail { error, failureCount, retryDelayMs }) so hosts can surface an error row or toast.

The grid never gives up: _hasMoreData stays true and retries continue at the capped interval indefinitely. The backoff state resets on a successful load, on _resetInfiniteScrollState() (i.e. reloadDataManager()), and on every fresh full data load.

Both continuation handlers (.then / .catch / .finally) bail out early when the grid has been destroyed (_disposed).

6. Bottom Detection Helpers

_isNearBottomOfLoadedData()

Computes near-bottom using both:

Uses the greater of both and a threshold of rowHeight * 3.

_isHoldingScrollbarAtBottom()

Returns true when any condition indicates active bottom hold:

This keeps behavior stable even when track metrics change during drag.

7. Total-Row-Count State (Push Model)

The grid does not pull or poll for the total row count. setTotalRowCount(count) is a plain state setter: whichever DataManager is currently attached calls it (indirectly, via _fireTotalRowCountChanged() → the manager's onTotalRowCountChanged field, wired automatically by VanillaGridElement.setDataManager()) whenever it determines the total for the current query — typically at the end of its own fetchRows(). This replaced a pull model where the grid itself gated a single resolve attempt behind a grid-owned "attempted once" flag plus a generation counter — a design that broke whenever two independent reload paths fired for the same user action (e.g. clearColumnFiltersAndSorting(), which triggers both a DataManager onSortChanged reload and its own reloadDataManager()): the shared flag could be "spent" by whichever reload's in-flight count request got aborted first, silently discarding the real total from the reload that actually won.

setTotalRowCount(count):

  1. Normalizes count (Number.isFinite → Math.trunc, else null).
  2. No-ops (no event fired) if the normalized value is unchanged from _totalRowCount — including the unset→unset case.
  3. Updates _totalRowCount and, when finite, recomputes _hasMoreData = _loadedRowCount < _totalRowCount.
  4. Invokes onTotalRowCountChanged(count) if configured.
  5. Refreshes the scroll indicator (_updateScrollIndicator()).

Concurrent firing (load start, not after render) is now a DataManager concern, not the grid's. ODataDataManager.fetchRows(), for example, kicks off its own count request concurrently with the row fetch — see docs/vanilla-grid/03-data-manager-implementation.md.

Staleness is resolved per-manager, per-request — not per-grid-generation. There is no grid-level generation counter analogous to the old _totalRowsGeneration. Each DataManager subclass that makes a network round trip for the count (e.g. ODataDataManager) tracks its own monotonic version, bumped on every fetchRows() call; an in-flight count request captures that version and only calls _fireTotalRowCountChanged() if its captured version still matches the manager's current version when it resolves. Because the version lives on the manager and increments on every fetch (not on an externally-orchestrated reset), two concurrent reloads for the same grid never have a shared flag to race over — whichever fetch is actually current always gets its own honest attempt, and a superseded one is discarded regardless of why it was superseded (aborted, errored, or just slow).

A manager with no network round trip for its count (e.g. StaticDataManager, or a custom manager whose row response already bundles the total) simply calls _fireTotalRowCountChanged() synchronously inside fetchRows() — there is no staleness window to guard against.

8. Scroll Indicator Integration

_updateScrollIndicator() includes infinite-scroll awareness:

Default text format:

{firstRow} - {lastRow} {messages.of} {displayTotalRows}

Where displayTotalRows prefers known total when resolved.

9. Interaction with Selection and Sorting

10. Load-More Busy Signals

A page fetch produces two distinct affordances: positional skeleton rows at the tail of the data, and a position-independent busy state (container class + banner). The split exists because the user can scroll away from the tail mid-fetch — at which point the skeletons say nothing, and only the busy state can answer "is this grid still loading?".

_setLoadingMore(loading) — the busy-state choke point

Defined in interaction.feature.js. It is the only place _isLoadingMore is written during a fetch, and it keeps two things in lockstep:

  1. _isLoadingMore itself (also read by isBusy() and by <vn-grid-toolbar>'s busy state);
  2. the vn-grid-loading-more class on .vn-grid-table-container — a paint hook hosts can style.

Called with true immediately before onLoadMore() and with false in the promise's .finally(), which already covers the success, error and _disposed paths. setRows() and _resetInfiniteScrollState() also route through it so a reload cannot leave the class orphaned.

The class is state; the banner is presentation. The class tracks _isLoadingMore exactly — it is the same fact isBusy() reports, so delaying it would make it lie, and hosts who want an undelayed indicator need an honest hook to hang it on. The banner is raised on the shimmerDelay timer instead (see below) and only dropped by _setLoadingMore(false), which keeps the invariant that no affordance outlives the fetch however the fetch ended.

vn-grid-loading-more is deliberately not vn-grid-shimmer-loading. The latter is toggled from isLoading and means the dataset you are looking at is being replaced — rows are wiped, scrolling is frozen. A load-more append leaves every visible row valid and leaves scrolling unblocked, so it gets its own class.

The banner (_ensureLoadMoreBanner() / _showLoadMoreBanner() / _hideLoadMoreBanner() / _destroyLoadMoreBanner())

Defined in rendering.feature.js; lifecycle mirrors the freeze guide (lazy-create, toggle via a class, destroy on teardown).

_showLoadMoreSkeletons()

When a page fetch takes longer than infiniteScroll.shimmerDelay ms, skeleton rows are appended at the tail of the data.

_hideLoadMoreSkeletons()

Grace period

A setTimeout of infiniteScroll.shimmerDelay ms is started at the beginning of each fetch. If the promise resolves before the timer fires (fast backend), clearTimeout ensures neither the banner nor the skeletons ever appear. If the timer fires first, both show until the response arrives.

A negative shimmerDelay is the "never show" sentinel: the timer is not scheduled at all, so the grid paints no built-in load-more affordance and a host that wants one builds it off the vn-grid-loading-more class.

Drag-to-bottom visibility

When the user drags the custom scrollbar thumb to the bottom, _loadMoreIfNeeded(true) is triggered and the shimmer timer starts. After the grace period, _showLoadMoreSkeletons() inserts skeleton rows which grow viewport.scrollHeight beyond the data-only maxScrollTop.

Without a special guard, the pointermove handler would keep resetting viewport.scrollTop to fraction × metrics.maxScrollTop (computed from displayRows.length only), placing the skeleton rows just out of view below the viewport edge and making the grid look frozen.

To fix this, _onScrollbarThumbPointerMove uses the actual DOM maximum when the thumb is pinned at the bottom:

if (newThumbTop >= metrics.usableTrack - 1) {
    const domMax = Math.max(0, viewport.scrollHeight - viewportHeight);
    if (domMax > newScrollTop) newScrollTop = domMax;
}

This keeps the viewport scrolled far enough to show skeleton rows for the entire duration of the drag-hold, giving consistent shimmer visibility regardless of input method (wheel, swipe, or drag). The near-bottom gate on the auto-scroll above does not interfere: a thumb pinned at the end of the track satisfies _isNearBottomOfLoadedData().

Cleanup on reset

_invalidatePool() cancels any pending timer, resets _loadMoreSkeletonVisible, and hides the banner — every caller (showLoadingSkeletons(), and therefore setLoading(true); showEmpty(); showError()) is a state where "loading more rows" no longer applies. Skeleton rows themselves go with the cleared <tbody>. destroy() removes the banner element and drops the vn-grid-loading-more class.

11. Failure Safety and Reentrancy Guards

Important protections:

12. Example Host Configuration

const grid = new VanillaGrid({
  infiniteScroll: {
    enabled: true,
    pageSize: 500,
    onLoadMore: async (skip, top) => {
      const res = await fetch(`/api/items?$skip=${skip}&$top=${top}`);
      return res.json();
    },
  },
  // Top-level, not nested under infiniteScroll — fires whenever the total
  // changes. A host driving the grid directly (no DataManager) pushes the
  // total itself, e.g. after its own $count fetch: grid.setTotalRowCount(total).
  onTotalRowCountChanged: (count) => {
    console.log('Known total:', count);
  }
});

13. Web Component/DataManager Wiring

<vn-grid> integration (vanilla-grid-element.js) auto-wires infinite loading when a DataManager is present:

This allows declarative infinite scrolling with minimal custom glue code.