Search, indexes & vectorization

In one line. How a document becomes findable — chunked, embedded and indexed on arrival — and how a question becomes an answer scoped to exactly the knowledge you attached. You’ll be able to. Reason about retrieval quality: why an agent finds a passage or misses it, and which kind of question needs the vector index versus the datasheet.

Two kinds of question, two engines

Everything a user asks lands on one of two engines, and knowing which is which explains most retrieval behavior:

Question	Engine	Example
Reading — “what does it say about X?”	The vector index: the query is embedded and KNN-matched against document chunks	“What is our retention policy for contractor records?”
Counting — “how many / which ones?”	The datasheet: SQL over structured fields	“How many contracts expire in Q3?”

A great assistant handles both — which is why a Knowledge collection can carry a Knowledge schema feeding a Datasheet alongside its vectorized text.

What happens when a document lands

For a Knowledge collection, indexing is part of arrival, not a separate chore:

Extract — text comes out of the file (PDF, Office formats included). Standalone images and text-free scans go through vision/OCR page summaries when those are enabled in Knowledge settings — otherwise they fail with a clear “no readable text” reason rather than indexing garbage.
Chunk — the text is split into overlapping passages sized for retrieval.
Embed — each chunk becomes a vector using the collection’s configured Search model.
Index — vectors land in the search index, keyed to their project and document.

From then on the document is query-ready: any agent with the collection attached can retrieve its passages. Updates re-index; deletions evict their vectors, and a scheduled reconciler heals gaps so the index converges with the collection even after failures.

Index Health — watching the corpus

A Knowledge collection’s Index Health tab is where you check that arrival actually worked. It leads with the reassuring number — “N of M documents indexed and searchable” — over a segmented ready / processing / failed bar, plus the Search model in use and a warning if anything has been processing for more than 30 minutes.

Failures are never dumped raw. A Needs attention list groups failed documents by cause — model access issue, no Search model selected, no readable text, rate-limited, timed out — each group with one plain “what to do” line and the real file names (no error dumps, no document ids). Fix the cause, then Retry a document, a group, or Retry all. Admins additionally get a View logs link per document that jumps to the Crew step-log timeline, where every retry attempt is visible with its full error (Crew & evaluations).

Reindex collection re-chunks and re-embeds every document. It now asks for confirmation first, stating exactly how many documents will be re-processed: search stays available while it runs, but it re-consumes embedding quota and can take a while — reach for it only after fixing a model/config issue or to rebuild the index.

Scope is the safety property

Retrieval is never global. When an agent answers, the KNN match is filtered to the collections attached on its Knowledge tab — an agent literally cannot retrieve from content you didn’t attach. This is the same boundary that makes per-audience agents safe: separate collections, separate agents, no leakage by construction.

Why an agent misses a passage — a diagnosis table

Symptom	Likely cause	Fix
“The documents don’t contain that” — but they do	The passage exists but scored below the retrieved top-K; question phrased far from the text	Ask more specifically; keep collections focused so relevant chunks aren’t crowded out
Answers from the wrong document	Multiple similar documents attached; retrieval is honest, attachment is too broad	Split collections by audience or domain; attach precisely
A freshly added document isn’t found	Indexing still in flight — or it failed	Check Index Health: wait if it’s processing, follow the grouped cause + Retry if it failed
Counting question answered vaguely	It went to the vector index, not the datasheet	Add a Knowledge schema so facts land in the Datasheet

Practical tuning

Focused collections retrieve better. Ten collections attached “just in case” dilute every query; attach what the agent’s job needs.
Titles and structure matter. Well-titled sections chunk into self-explanatory passages; a 40-page wall of text chunks into ambiguity.
Test retrieval in the playground. Ask the exact questions your users will ask and click the citations — the fastest way to see what the index is actually returning.

Where to go next

The collection category that makes all this automatic: D2 · Collection types.
The user-facing surface on top: E4 · Chat & search experience.
Structured facts and SQL answers: D5 · Drive & datasheet.