DataManager Implementation in Vanilla-Grid

This document explains the DataManager architecture in Vanilla-Grid, including the base contract, built-in implementations (ODataDataManager, GraphQLDataManager, StaticDataManager), and how <vn-grid> wires DataManager behavior into grid lifecycle.

1. Architecture Goal

The DataManager layer decouples data access from rendering.

This keeps data-source specifics out of core rendering logic.

2. Base Contract (data-manager.js)

DataManager defines the expected interface:

Default behavior is intentionally minimal/safe:

Subclasses override as needed.

3. <vn-grid> Integration Flow

VanillaGridElement.initializeGrid(...) performs DataManager-aware wiring:

  1. Read attached DataManager (this._dataManager).
  2. Resolve onSort callback:
    • explicit grid option first, else
    • dm.handleSort(...) bridge when available.
  3. If DataManager exists and infiniteScroll is enabled, auto-wire:
    • onLoadMore = dm.fetchMoreRows(skip, pageSize, context) (unless explicitly provided)
  4. If pageSize not explicitly set, use dm.getPageSize().

Separately, setDataManager() unconditionally wires dataManager.onTotalRowCountChanged = (count) => grid.setTotalRowCount(count) — this is passive state sync (not a reload trigger), so it is wired regardless of whether a DataManager or infiniteScroll is configured, and independent of setAutoReloadOnConfigChange(). See §8.

Selection and row-key attributes are merged into grid options at this same stage.

4. Context Model

VanillaGridElement._createContext(...) builds a context object passed to DataManager methods, including:

Per-call overrides are merged in for operation-specific context.

This context standardizes data-layer decisions without exposing low-level DOM internals.

5. Row Loading Lifecycle

VanillaGridElement.loadRowsAsync(options) with DataManager:

  1. Guard: throws if no DataManager is set.
  2. Call dm.cancel() — abort any previous in-flight fetch (last-wins semantics; see §5.1).
  3. Increment _loadGeneration counter — stamp this invocation.
  4. await this.ready() — ensures persisted column settings are applied.
  5. Emit lifecycle event: vn-grid-loading.
  6. Set the loading state (see §5.2) — setLoading(true) unconditionally; above the row-count threshold, additionally await a two-frame paint checkpoint so the skeleton is guaranteed to paint before fetching.
  7. Fetch rows via dm.fetchRows(context, options).
  8. Generation check after fetch — if another call started while this was in flight, silently discard the response and return (without clearing the loading state — the newer call owns it now).
  9. Optional transform via dm.transformRows(...).
  10. Generation check after transform.
  11. Clear the loading state (if set by this call) and push rows into grid via setData(rows).
  12. Optional side-effects via dm.onRowsLoaded(rows, context).
  13. Emit success event: vn-grid-loaded.
  14. On failure: if AbortError (or stale generation), return silently without touching the loading state. Otherwise clear it (if set by this call), emit vn-grid-error, and rethrow.

reload() delegates to loadRowsAsync().

5.1 Last-Wins / Cancel-and-Restart Semantics

When a column header is clicked during an active backend load (shimmer visible), a new loadRowsAsync() call races with the previous one. Without cancellation, both would eventually call setData(), causing a flicker or incorrect final state.

The solution is last-wins: each loadRowsAsync() call:

  1. Calls dm.cancel() to abort any in-flight fetch from the previous call.
  2. Stamps the invocation with a monotonically-increasing _loadGeneration counter.
  3. After every await, checks whether the generation still matches the latest value. If not, the response is silently discarded.

The AbortError thrown when the previous fetch is cancelled is caught inside loadRowsAsync() and treated as a no-op — it does not fire the vn-grid-error event and does not re-throw.

The loading state (see §5.2) remains set until the winning fetch completes, preventing a flash of empty content between cancellation and the new result. Only the call whose generation still matches _loadGeneration is ever allowed to clear it — a stale/superseded call (whether it resolves or errors) returns silently without touching it, since a newer, still-in-flight call may own it.

Select-all checkbox: while isLoading is true, the header checkbox is always disabled. This prevents confusing behaviour when the user clicks "select all" while rows are still arriving.

5.2 Automatic Loading State (and the Row-Count-Gated Forced Paint)

search(), setColumnFilter(s), clearColumnFilters(), clearColumnFiltersAndSorting(), and reload() all funnel into loadRowsAsync() (directly, or via reloadDataManager()) — as does the initial load. loadRowsAsync() brackets every load with VanillaGrid.setLoading(true)/setLoading(false) itself — success path, error path (a rejecting fetchRows() never leaves a stuck skeleton), generation-guarded against superseded calls (§5.1). Hosts must not wire setLoading into the manager fetch hooks: onBeforeFetch/onFetchResponse/onFetchError are pure app hooks (status text, raw-Response inspection, error UI) — a hand-rolled setLoading there duplicates the built-in bracketing and lacks its generation guard.

Above a configurable row-count threshold, the skeleton paint is additionally forced before dm.fetchRows() begins, so a slow synchronous scan (e.g. StaticDataManager's Array.filter() pass) never leaves the UI looking frozen with no feedback. Below the threshold the loading state is still set, but the paint is deliberately not forced — a fast same-task load resolves before the browser ever paints the skeleton, so no flicker appears.

StaticDataManager two-stage filter→search pipeline

_getFilteredRowsAsync() evaluates constraints as an ordered, independently-memoized pipeline — _rows → [Stage 1: column filters] → _columnFilteredRows → [Stage 2: search] → _filteredRows — rather than in one combined pass over the whole dataset. The point is the invalidation asymmetry: a search-term (or search-fields) edit re-runs only Stage 2 against the cached Stage-1 survivors; it does not re-evaluate the column filters. So searching after a selective filter has narrowed 5,000,000 rows to a few thousand scans only those few thousand — the frequent case (typing a search after filtering) never rebuilds a full-dataset search haystack. Because filter AND search is associative/commutative, the staged result is identical to a single combined pass and independent of the order the two constraints were applied (guarded by tests/node/static-staged-search.test.js).

In-thread search haystack cache (gated). _searchRowsInThread() can match each row against a cached lowercased haystack string (_ensureInThreadHaystackCache() — a WeakMap keyed by row object, mirroring the Worker path's resident haystacks but for the below-threshold in-thread path). The U+0001 field separator baked into _buildHaystack() means a haystack.includes(needle) hit can never span two fields, so a cached match is identical to the per-field _rowMatches() it replaces. Keyed by row object, one cache serves both the full-dataset and any survivor-subset search and survives across search-term keystrokes and column-filter changes (a row's haystack depends only on its own content and the searched fields).

This cache is a reuse optimization, not a free win — so it is gated by a reuse counter (IN_THREAD_HAYSTACK_CACHE_AFTER). Building it is a full up-front pass: it lowercases every searched field of every row (~91 ms for a 50k-row, 15-field dataset), more work than the un-cached lazy scan, which early-exits at the first matching field (~26 ms) and is the faster choice for a one-shot search. Every subsequent search over the same rows is then ~2.2× faster (~11.6 ms vs ~25.7 ms warm at 50k) with near-zero GC (vs ~140 minor GCs / 200 searches). Because the build only amortizes once several searches share a data load, _searchRowsInThread() stays on the lazy early-exit _rowMatches() path for the first IN_THREAD_HAYSTACK_CACHE_AFTER searches over a given (datasetVersion, searchFields) signature, and only builds + reuses the cache from that search on — so the first/one-shot search never pays the build, while a sustained interactive session still gets the fast cached path. The signature is tracked in _inThreadSearchSignature/_inThreadSearchCount; a setRows()/setSearchFields() change flips it, resetting both the counter and the cache (no setter has to clear it explicitly). The whole mechanism only applies below workerThreshold — above it the Worker owns extraction.

StaticDataManager Web Worker offload for filter/search

Above dataLoading.workerThreshold rows (forwarded to the attached StaticDataManager via its setWorkerOptions() method), StaticDataManager's filter/search stages (§ pipeline above) evaluate in a persistent Web Worker (StaticDataManager._workerSource) rather than a disposable one created fresh per call. Replies with a Uint32Array of matching row indices, reconstructed into row objects on the main thread. Falls back to the in-thread path (_getFilteredRowsSync() — the combined search+column reference implementation, also the worker-parity oracle in tests/node/data-loading-worker.test.js) when the runtime lacks Worker/Blob support, useWorker is disabled, or the row count is below the threshold — filtering results are identical either way.

Extraction is cached, not redone on every call. A naive "extract everything, ship it, discard it" design would re-scan the whole dataset and re-clone the result on every search keystroke or filter-value tweak — the actual scan itself was measured at ~31 seconds for a 5,000,000-row, 21-column dataset with no searchFields configured, and cloning that scan's result once was enough to hit an out-of-memory DataCloneError in isolation. Instead, _filterViaWorker() computes two independent signatures — _computeHaystackSignature(): (datasetVersion, searchFields); _computeColumnSignatures(): a Map<columnKey, signature>, one entry per active filter column — each independent of the search needle or filter condition values — and only (re)extracts and (re)sends haystacks/column values when the relevant signature changes. The two are tracked separately, not as one combined signature, because they're needed on unrelated schedules (a search-term-only edit never touches column shape and vice versa) and have wildly different per-row costs (see the chunking paragraph below):

Worker-resident haystacks: index-based subset search. The Worker holds THREE independently-resident slots, not one, so a survivor-subset search never evicts (or is evicted by) the full-dataset haystack:

Before this feature, subset searches reused the single haystack slot positionally, which meant every filter change re-extracted the survivor haystacks from scratch (repacking strings the Worker might already hold resident inside its full copy) and a survivor-subset search evicted the full-dataset haystack — clearing the filter while a search stayed active then re-extracted the whole dataset again, undoing whatever prewarmSearchIndex() had invested. With three independent slots, a filter tweak with an active search now ships only the needle (or the tiny index list) once the full haystack is resident, and clearing filters with a search still active is a needle-only cache hit.

The payload itself is compact, not a boxed array per row. Search haystacks are packed into a handful of moderately-sized "chunk blob" strings (one per FILTER_EXTRACTION_CHUNK_SIZE-row batch, e.g. ~2,500 for 5,000,000 rows) plus two Uint32Arrays of per-row start offsets/lengths within their chunk (_packHaystacksChunked()), instead of an Array<string> with one boxed string object per row — postMessage()'s structured-clone cost is dominated by per-object bookkeeping, not raw byte count, so a multi-million-entry Array<string> clones far worse than a few thousand larger strings plus two transferable typed arrays. Deliberately an array of chunk blobs rather than one single joined string for the whole dataset: joining a 5,000,000-row, 21-column dataset's combined haystack content into one string was measured to throw RangeError: Invalid string length (V8's single-string size ceiling), even though the identical content split across chunk blobs builds and clones without issue. Column filter values for number/uid/date/datetime/time types (the common, non-fan-out case) are similarly packed as a transferable Float64Array (_packColumnValuesChunked()) rather than a boxed Array<number> — except when a condition uses a type-agnostic unary operator (isEmpty/isNotEmpty/isTrue/isFalse), which tests the raw value's null/undefined/'' identity; those columns keep the boxed representation so an originally-empty value doesn't get silently converted to NaN and misread as non-empty. String/boolean columns and genuine multi-field fan-outs keep the original boxed-array shape (lower priority to compact, since the caching above already means they're only sent once per signature change, not once per keystroke).

The one unavoidable (re)extraction pass is chunked, not one synchronous sweep — at two different granularities. When the signature does change, _packHaystacksChunked()/_packColumnValuesChunked() process the dataset in batches, yielding to the event loop (_yieldToMainThread()) between batches — so even a first-ever search/filter over a multi-million-row dataset keeps the page responsive throughout, rather than blocking for the pass's whole duration in one synchronous sweep. The two extraction kinds do not share a batch size: building a multi-field search haystack costs roughly two orders of magnitude more per row than reading (or getter-calling) a single column's value, so a batch size sized for one starves the other. FILTER_EXTRACTION_CHUNK_SIZE (2,000 rows — measured to keep a single haystack-building batch under ~7ms on reference hardware, well inside a 60fps frame budget) governs _packHaystacksChunked(); FILTER_COLUMN_CHUNK_SIZE (100,000 rows — the cheap per-row cost keeps even this larger batch under ~6ms) governs _packColumnValuesChunked(). A shared 100,000-row batch size for both was the original (Phase 1) design; it was measured to let a single haystack-building batch run ~365ms synchronously on a 250,000-row dataset — worse than the in-thread (non-worker) filter path it was meant to outperform — which is what motivated the split.

How the pass yields matters as much as how it chunks. _yieldToMainThread() delegates to the shared VanillaGridYield singleton (src/vanilla-grid/main-thread-yield.js) — the one main-thread yield primitive in the component, also used by the sort Worker's per-column extraction pass (features/sorting-worker.feature.js), so both chunked passes schedule identically. StaticDataManager reads the global lazily and falls back to a plain macrotask when it is absent (partial bundle, or a standalone node test that requires only the data manager). The module resolves on the fastest available cooperative-yield channel — scheduler.yield() where the runtime has it, else a module-scoped MessageChannel macrotask, else setTimeout(…, 0) — and deliberately not on requestAnimationFrame. rAF resolves at the next frame boundary, which caps the pass at one batch per frame no matter how short the batch is: a ~5ms batch followed by a ~11ms wait ran the extraction at a ~30% duty cycle, so roughly two thirds of a large warm's wall clock was the main thread sitting idle. Measured on a 300,000-row, 21-field pack: 2.49s wall for 0.75s of work under rAF, versus 1.07s at a ~98% duty cycle through the fast macrotask — the same work, the same batch length, ~2.3× less elapsed time (extrapolating to ~42s → ~18s at 5,000,000 rows). The browser still interleaves rendering and input between batches, so responsiveness is preserved; what is reclaimed is pure idle time. Two smaller consequences of the same choice: a warm now continues in a backgrounded tab (rAF stops firing when a tab is hidden, which used to stall a warm indefinitely), and the fallback avoids the nested-setTimeout 4ms clamp. rAF remains the right primitive for work that must align with a paint, which is why dom-scheduler.js keeps it. Node is excluded from the MessageChannel branch (the module's internal yieldPort() returns null there): node's ports are worker_threads handles that keep the event loop alive and hang the test runner, and unref()ing them would risk exiting with a pack half-finished. The ladder and that guard are covered by tests/node/main-thread-yield.test.js.

Batch size is a row count; a slice is also capped by time. A row count is only a proxy for cost — the same 2,000 rows take ~5ms on a typical row and far longer on a very wide one or one behind an expensive filterValueGetter (the ~365ms slice above is exactly that failure). Both packers therefore end a slice at whichever comes first: the row cap, or SLICE_BUDGET_MS (5ms), tested every SLICE_PROBE_ROWS (256) rows. The budget only ever makes yields more frequent — the row caps remain the upper bound — and costs nothing measurable (~98% duty cycle either way). Measured on a pathological 4,000-row × 300-field dataset: one 116ms blocking slice before, fifteen slices with a 14.1ms worst case after. In _packHaystacksChunked() a budget-triggered yield does not close the current blob: the Worker addresses row i as blobs[Math.floor(i / chunkSize)], so blobs must stay exactly FILTER_EXTRACTION_CHUNK_SIZE rows long — yield points and blob boundaries are deliberately decoupled, and a short blob would silently misaddress every row after it (guarded by tests/node/static-search-warm-latency.test.js). _packColumnValuesChunked() has no such addressing to preserve, so there a budget-triggered yield simply ends the slice early.

See tests/node/data-loading-worker.test.js and tests/playwright/search-worker-performance.spec.js for the caching/cache-miss/compact-encoding/responsiveness test coverage; tests/node/static-staged-search.test.js for the two-stage pipeline's result-equality and Stage-1-reuse guarantees; tests/node/static-incremental-narrowing.test.js for the strict-narrowing classifier's soundness and the base-subset scan's correctness; tests/node/static-indexed-survivor-search.test.js for the index-based/packed-fallback subset dispatch, the no-eviction (thrashing) guard, the fused round trip, and _columnFilteredIndices invalidation; tests/node/static-column-payload-residency.test.js for per-column payload residency and the packing-determinant signature (type/forceBoxed/getter-identity) regressions; and tests/playwright/staged-search-survivor-worker.spec.js / tests/playwright/staged-search-clear-filters.spec.js for the survivor-subset Worker search and the domain switch on clear over a real browser Worker.

5.3 Lowering client-side memory (large offline datasets)

The remaining limit above is a memory ceiling, not a time one, and it is reached without any cross-origin isolation being available. Two host-controllable levers reduce it on a single thread (no SharedArrayBuffer, no COOP/COEP):

Both are opt-in and additive; neither changes the default { rows: [objects] } behavior. Guarded by tests/node/static-intern-and-advisory.test.js. A larger, still-isolation-free columnar/dictionary-encoded backing store (moving the ceiling ~5–10×) is a separate, deferred effort.

5.4 Pre-warming the search index (prewarmSearchIndex())

The first search on a large dataset is slow (§5.2: it pays the one-time haystack extraction) while every later search is instant (the haystacks are resident). That first-search cost lands on the user's first keystroke. StaticDataManager.prewarmSearchIndex() moves that extraction to a moment the host controls, so the first real search is already a cache hit. It is opt-in (never warms by default): warming spends CPU and memory up front, which is wasted on users who never open the search box, so the decision belongs to the host. The host either calls the method itself, or sets prewarmSearchIndex: true and lets the manager schedule it (below) — in both cases nothing warms unless asked.

Guarded by tests/node/static-search-prewarm.test.js (in-thread path + the dev-mode messages), tests/node/static-prewarm-search-index-option.test.js (the prewarmSearchIndex option's scheduling), tests/node/static-search-warm-latency.test.js (the subset-await ratio guard and the pre-warm-time advisory), tests/playwright/search-prewarm.spec.js (the real Worker prime path), tests/playwright/search-index-auto-prewarm.spec.js (the option end-to-end over a real Worker) and tests/playwright/search-warm-latency.spec.js (the browser yield primitive's duty cycle, slice length, and mid-warm search parity).

5.5 Cancelling superseded searches (cancel())

VanillaGridElement.loadRowsAsync() calls dm.cancel() before every load and discards stale results with a last-wins generation guard — but that guard runs after await dm.fetchRows(), so it prevents a stale render, not the stale work. For StaticDataManager the expensive work is the O(matches) main-thread result build in _filterViaWorker() (filtered[i] = rows[indices[i]]), which for an intermediate short term (e.g. typing/backspacing through "b" / "a") can be 250k–500k rows — built and then thrown away. Unlike ODataDataManager (which overrides cancel() to abort its fetch()), StaticDataManager has no request to abort; the cost is CPU.

So StaticDataManager overrides cancel() to bump a cooperative cancellation token (_loadToken). Every fetchRows() captures a fresh token (++_loadToken); cancel() — and the next fetchRows() — bump it. The pipeline checks the token at each await boundary and short-circuits with a FETCH_CANCELLED sentinel when superseded:

Guarded by tests/node/static-search-cancel.test.js (token mechanics, in-thread) and tests/playwright/search-cancel.spec.js (the Worker-path build-skip: a superseded search builds nothing).

6. ODataDataManager Internals

ODataDataManager extends base contract for OData-compatible endpoints.

6.1 Configuration

Constructor options include:

6.2 Request construction

_buildUrl(baseUrl, skip, top) serializes OData query options onto the URL. $skip and $top are only appended when non-null; passing null for either omits that parameter (used when loading all rows at once with infinite scroll disabled). Other clauses:

buildRequestHeaders(context) resolves from:

  1. dynamic header function
  2. static header object merged with Accept
  3. default Accept

6.3 Fetch behaviors

6.4 Sort delegation

handleSort(column, direction, sortState) builds an OData $orderby expression and stores it in _query.$orderby, emits the notification callback onSortChanged(orderBy, column, direction, sortState), and then fires _fireConfigChanged('handleSort') — sort is a query mutation like $filter/$search, so it goes through the same config-changed channel. A grid with setAutoReloadOnConfigChange(true) therefore re-fetches page 0 on sort automatically; onSortChanged is notification-only and no longer needs to trigger the reload.

When a multi-column sort chain is present (sortState.sortColumns), all entries are processed in order. For each entry, if the column has a sortFields array, each path is expanded into a separate $orderby clause with the same direction — enabling composite-field sorting via navigation properties. Dot notation is automatically converted to OData slash notation (Employee.FirstName → Employee/FirstName).

Example result for a column declared with sort-fields="Employee.FirstName,Employee.LastName":

$orderby=Employee/FirstName asc, Employee/LastName asc

Columns without sortFields generate a single clause from their key.

This is the key bridge for server-driven sorting. See Sorting §7.4 for the client-side counterpart.

6.5 GraphQLDataManager Internals

GraphQLDataManager extends the base contract for GraphQL endpoints. Where ODataDataManager can own the whole serialization pipeline (OData is a standardized query grammar, so contains(Name,'x') and Price gt 5 parses identically on every compliant service), GraphQL standardizes only transport and envelope: a POST of { query, operationName?, variables? } returning 200 OK with { data?, errors? } — a GraphQL error is an HTTP 200 with a populated errors array, not a 4xx/5xx. Everything a grid cares about (pagination shape, sort/filter/search argument names, where the rows live in data, whether a total exists) is schema-specific with no cross-server grammar to hardcode.

So the manager owns everything universal — transport, envelope/errors handling, cancellation, last-wins version guarding, total-count push, config-changed events, response unwrapping via a declared dot-path — and the host supplies the query document plus small builder hooks that translate grid state into the schema's variables. This is the extension pattern of §9, shipped as a well-behaved default. No VanillaGrid / <vn-grid> core changes are required — the grid already talks only to the DataManager interface.

6.5.1 Two-tier configuration

The manager always thinks in grid-native skip/top (matching the fetchMoreRows(skip, top) contract). The state object passed to the builders carries offset conveniences (page = floor(skip/top)+1, perPage = top) and cursor conveniences (after = stored endCursor, first = top) derived from them, so the host maps grid offsets to the schema's real pagination in one honest boundary.

6.5.2 Configuration

Constructor options include: endpoint (required), headers, query (required), operationName, variables; paginationStyle ('offset' | 'cursor'), pageSize, infiniteScroll, maxRows; dataPath, totalPath, pageInfoPath; countQuery, countPath, countVariables; variableNames, caseInsensitive; the builder hooks buildVariables / buildSort / buildFilter / buildSearch; and hooks onSortChanged, onRowsLoaded, onFetchResponse, onBeforeFetch, onFetchError, onGraphQLErrors.

6.5.3 Fetch behaviors

6.5.4 Sort / filter / search delegation

handleSort(column, direction, sortState) stores buildSort(column, direction, sortState) in _orderBy (or a default "key direction" string when no buildSort is given; multi-column chains read sortState.sortColumns), emits the notification callback onSortChanged(_orderBy, column, direction, sortState), then fires _fireConfigChanged('handleSort') for auto-reload. setSearchTerm recomputes _search = buildSearch(term) (default term || null); setColumnFilters (and the single-column / clear variants) normalize the model via window.VanillaGridFilterModel and recompute _filter = buildFilter(model, { caseInsensitive }) — without a buildFilter hook there is no universal GraphQL filter grammar, so _filter stays null. setSearchTerm/setColumnFilters fire _fireConfigChanged(...) with no-op change detection; handleSort fires it unconditionally (a header interaction is always a real change).

6.5.5 GraphQLError

GraphQLDataManager.GraphQLError is a small Error subclass carrying .graphQLErrors (the raw array) so hosts / onFetchError can inspect individual messages / paths. Thrown by the default onGraphQLErrors.

7. StaticDataManager Internals

StaticDataManager provides in-memory paging over a fixed array.

State:

Methods:

No network calls are involved. The full dataset stays in _rows; search + column filters produce a memoized derived array (_filteredRows), invalidated by setRows(), setSearchTerm(), setSearchFields(), setColumnFilters(s)/clearColumnFilters(), and setFilterValueGetters().

8. Sort + Infinite Scroll Composition

DataManager and grid collaborate as follows:

This creates a consistent flow for both remote and in-memory data sources.

9. Extension Pattern

To implement a custom DataManager:

  1. Extend DataManager.
  2. Override at least fetchRows(...).
  3. Add fetchMoreRows(...) for infinite mode.
  4. Call this._fireTotalRowCountChanged(count) from fetchRows(...) (or wherever the total becomes known) if known totals are needed — see §8. If the total arrives asynchronously via a separate network round trip, guard against a superseded call the way ODataDataManager._queryVersion does (§6.3).
  5. Implement handleSort(...) for server-sort integration — update your sort clause, then call this._fireConfigChanged('handleSort') so a grid with setAutoReloadOnConfigChange(true) re-fetches page 0 on sort (mirrors ODataDataManager / GraphQLDataManager). The manager never reloads itself; the reload is the grid's job.
  6. Optionally use transformRows(...) and lifecycle hooks.
  7. Override cancel() if your manager makes asynchronous network calls — create an AbortController per request, store it, and call .abort() in cancel(). This enables last-wins load semantics (see §5.1).

Because the contract is small and context-rich, custom backends can be integrated without changing VanillaGrid internals. GraphQLDataManager (§6.5) is a shipped, well-behaved default implementation of exactly this pattern for GraphQL endpoints: the host supplies the query document and builder hooks; the manager owns transport, envelope/errors handling, cancellation, version-guarded total-count push, and response unwrapping.

10. Example Wiring

const dm = new ODataDataManager({
  baseUrl: '/api/items',
  countUrl: '/api/items/$count',
  pageSize: 500
});

const el = document.getElementById('myGrid');
el.setDataManager(dm);
el.initializeGrid({ infiniteScroll: { enabled: true }, sorting: { serverSide: true } });
await el.loadRowsAsync();

This setup enables:

The GraphQL equivalent supplies a query document plus builder hooks instead of a structured query object (offset paging with an inline total, no separate count request):

const dm = new GraphQLDataManager({
  endpoint: 'https://graphql.anilist.co',
  query: ANILIST_QUERY,
  paginationStyle: 'offset',
  pageSize: 50,
  infiniteScroll: true,
  dataPath: 'Page.media',
  totalPath: 'Page.pageInfo.total',           // inline total — no second request
  buildVariables: (state) => ({
    page: state.page, perPage: state.perPage,
    search: state.search, sort: state.orderBy, ...state.filter,
  }),
  buildSort: (column, direction) => [mapToMediaSort(column, direction)],
  buildFilter: (model) => mapToAniListArgs(model),
});

el.setDataManager(dm);
el.initializeGrid({ infiniteScroll: { enabled: true }, sorting: { serverSide: true } });
await el.loadRowsAsync();

Search is a source-agnostic contract on the DataManager. Each manager interprets a free-text term in its own way, but hosts use a single API. The term reduces the rows the grid loads/displays; it composes with sort and (for OData) $filter.

11.1 Contract

dm.setSearchTerm('john');   // narrow rows; fires onConfigChanged('setSearchTerm')
dm.getSearchTerm();         // => 'john'
dm.setSearchTerm('');       // clear

Base DataManager.setSearchTerm() is a no-op, so custom managers without search support ignore the term. Both built-in managers override it. The mutator follows the same _fireConfigChanged convention as setFilter/setOrderBy, so it integrates with auto-reload. Setting the same term twice is a no-op (no spurious reload).

A manager may additionally restrict the term to a subset of fields:

dm.getSearchFields();   // string[] when restricted, or null when unrestricted

null means "unrestricted" — the base DataManager.getSearchFields() implementation, and the value returned whenever the field list isn't actually being enforced (see §11.4's searchMode caveat). <vn-grid>.getSearchFields() delegates to this, and <vn-grid-toolbar-search> consumes it to show a hover/focus tooltip listing the active fields — see vanilla-grid-toolbar's implementation doc.

11.2 Triggering a search from <vn-grid>

await gridElement.search('john');   // setSearchTerm + reloadDataManager

search() delegates to dm.setSearchTerm(term) then reloads:

reloadDataManager() runs reset() → _resetInfiniteScrollState() → loadRowsAsync(), so the filtered set loads from page 0 under last-wins semantics.

Persistence (opt-in). When persistence.searchTerm.enabled is true (default false), search() also writes the term to the storage provider. On the next load the grid parks the stored term and <vn-grid> re-applies it after the first data load — composed with any restored filter model in a single reload. See Local Storage Settings §4.5.

11.3 StaticDataManager — client-side search

Case-insensitive substring match against the raw field values (not the rendered/localized cell text — the manager has no column or formatter knowledge).

11.4 ODataDataManager — server-side search

Two modes, selected via searchMode:

Runtime accessors: setSearchTerm/getSearchTerm, setSearchMode/getSearchMode, setSearchFields/getSearchFields.

getSearchFields() is mode-aware. It returns the configured list only while searchMode: 'filter' is active and the list is non-empty; in the default searchMode: 'search' the list is stored but never serialized (the server owns $search's field scope), so getSearchFields() reports null there even if searchFields was passed at construction — consumers (e.g. the toolbar's search-fields tooltip, §11.1) must not treat a configured-but-inert list as an active restriction.

Count correctness: _refreshTotalRowCount() appends $filter and the serialized search term to countUrl, so the total reflects the constrained set. (Previously the count endpoint received no query clauses.)

11.5 Search-Match Highlighting Is a Rendering-Only Re-Match

<vn-grid highlightSearchMatches> (§1.17 of the virtualization doc) wraps matched text in <mark> purely by re-checking the active search term against each cell's already-formatted display text at render time — it has no access to, and does not consult, whichever matching semantics a manager actually used to decide row inclusion above.

This is a deliberate decoupling, not an oversight: StaticDataManager matches against raw field values (§11.3), which can differ from the rendered text a date/number/currency-typed column shows; ODataDataManager and GraphQLDataManager push the term to the server and receive back only already-filtered rows, with no signal at all about which field(s) matched or why (server-side full-text ranking, stemming, and fuzzy matching are opaque to the client).

Consequence, and expected behavior: a row can appear in the result set with no cell highlighted — it matched a hidden/non-displayed field, matched its raw value where the formatted value differs, or matched via server-side relevance ranking a plain case-insensitive substring check can't reproduce. This is not a bug to fix; it is the accepted tradeoff for a purely visual affordance that needs zero manager-specific plumbing and works identically across every DataManager, including infinite-scroll pages landing after the initial load.

12. Column Filters

Structured, type-aware per-column filtering. Same architectural spine as search (config setter → onConfigChanged → reload), composing with search and the base $filter. Full reference: 07-column-filters-implementation.md.

12.1 Contract

dm.setColumnFilters({ salary: { type: 'number', conditions: [ { operator: 'between', value: 30000, valueTo: 50000 } ] } });
dm.getColumnFilters();                  // → normalized clone
dm.setColumnFilter('country', { type: 'string', conditions: [ { operator: 'equals', value: 'Italy' } ] });
dm.clearColumnFilters();

All four are no-ops on the base class. Setters store the model normalized via window.VanillaGridFilterModel.normalizeFilterModel(), fire onConfigChanged('setColumnFilters'), and are no-ops when the normalized model is unchanged. The model persists across reset() (like search and $filter).

12.2 StaticDataManager — client-side predicate

Compiles the model into a memoized (row) => boolean (cross-column AND, per-column combinator) applied after search in _getFilteredRowsAsync()'s in-thread path (_getFilteredRowsSync()) — so paging and count use the filtered set. Above workerThreshold rows, the same search+filter logic is instead evaluated in a Web Worker (see §5.2) — results are identical either way. Options: caseInsensitive (default true), columnFilters, filterValueGetters ({ key: (row) => value }) for computed columns, and useWorker/workerThreshold for the worker offload.

12.3 ODataDataManager — server-side $filter

_composeEffectiveFilter() AND-merges base $filter + column filters + 'filter'-mode search disjunction; both _buildUrl() and _refreshTotalRowCount() use it. Type-aware literal formatting (strings quoted/escaped, numbers/dates/booleans bare). Options: caseInsensitive, columnFilters.

12.4 From <vn-grid>

grid.setColumnFilters(model) / setColumnFilter(key, cf) / clearColumnFilters() delegate then reload (with the same auto-reload guard as search()). The element drops non-filterable columns (filtering.enabled === false or column.filterable === false) and enriches kept entries with field/type from the column def. See 07-column-filters-implementation.md.