Scope

A host’s scope says which of its items are indexed and how deeply. This page is the exact document; choose what is indexed has the common cases.

The document

{
  "rev": 1,
  "window": { "by": "mtime", "within": "1y", "outside": "catalog" },
  "rules": [
    { "when": { "path": { "glob": "**/node_modules/**" } }, "then": "exclude" },
    { "when": { "size": { "gt": 26214400 } }, "then": "catalog" },
    { "then": "embed" }
  ],
  "limits": { "maxObjectBytes": 268435456, "maxTextBytes": 2097152 }
}

field
rev	integer, set by the server on every save. Leave it out when you PUT
window.by	a dimension with an ordering: mtime on folders and the pad
window.within	a duration: ^(\d+)(h|d|w|m|y)$; m is 30 days, y is 365
window.since	an RFC 3339 time instead of within
window.outside	the most an item outside the window gets. An item with no value for by counts as inside
rules	ordered; the first rule whose every condition matches wins. The last rule has no when
limits.maxObjectBytes	items larger than this are never read: they stay at catalog
limits.maxTextBytes	text is cut at this many bytes of UTF-8

Missing fields take the defaults shown above. The depths, from least to most: exclude, catalog, embed.

Dimensions

Every host has these five. A host may add its own; GET /v1/hosts/{id}/scope lists them.

dimension	type	value
path	path	the path within the host
ext	string	the lowercased suffix after the last . of the file name, without the dot; empty when none
mime	string	the host’s type, else guessed from ext
size	bytes	the item’s size
mtime	datetime	the item’s modification time

Conditions

type	conditions
string, path	{"eq": "…"}, {"in": ["…"]}, {"prefix": "…"}, {"glob": "…"}
bytes, integer	{"gt": n}, {"gte": n}, {"lt": n}, {"lte": n}; any subset, all required
datetime	{"within": "90d"} (value is at least now minus the duration), {"before": "RFC 3339"}, {"after": "RFC 3339"}
boolean	{"eq": true} or {"eq": false}

Globs: * matches any run of characters except /; ** any run including /; ? one character except /; {a,b} alternatives; everything else is literal. A pattern without / is matched against the file name only, so *.log matches a/b/c.log. A pattern with / is matched against the whole path.

A condition on a dimension the item has no value for does not match, and the rule is skipped, except {"eq": false} on a boolean, which matches a missing value.

Evaluation

depth  = the first rule whose every condition matches, else the last rule's depth
if the window is set, the item has a value for `by`, and that value is before the boundary:
       depth = the lesser of depth and window.outside
if size > limits.maxObjectBytes and depth is above catalog:
       depth = catalog

The result is applied when the host walks its items. An item already indexed deeper than its new result is reduced on the next walk; an item whose result rose is read again then.

Defaults

A folder on a device:

{ "window": { "by": "mtime", "within": "1y", "outside": "catalog" },
  "rules": [
    { "when": { "path": { "glob": "**/{node_modules,.git,.hg,.svn,.cache,__pycache__,.venv,venv,target,dist,build,.next,.turbo,DerivedData,Pods,.gradle,.idea,.vscode,.DS_Store,Library,.Trash}/**" } }, "then": "exclude" },
    { "when": { "path": { "glob": "**/.*" } }, "then": "exclude" },
    { "when": { "path": { "glob": "**/.*/**" } }, "then": "exclude" },
    { "when": { "ext": { "in": ["log", "tmp", "lock", "sqlite", "sqlite-wal", "sqlite-shm", "db", "pyc", "o", "a", "so", "dylib", "class", "jar", "bin", "iso", "dmg", "pkg"] } }, "then": "exclude" },
    { "when": { "size": { "gt": 26214400 } }, "then": "catalog" },
    { "then": "embed" } ] }

The pad: the same window, size > 25 MiB at catalog, everything else at embed.

Coverage

A host reports what its scope produced: {"items": {"catalog": 10, "embed": 900}, "evaluatedAt": "…", "scopeRev": 1}. It is on the host (GET /v1/hosts/{id}) and in every search answer, so a tool can tell how much of a host is searchable by text.