Vectorizer

View as Markdown

An internal RocketRide ingestion filter that chunks flagged text and tables, creates embeddings through configured components, and persists the resulting documents to a configured store.

What it does

The vectorizer runs only for objects marked with FLAGS.VECTORIZE. It passes each non-empty text or table value through a configured preprocessor, embeds the resulting chunks, and either writes them directly to the store or sends them downstream for an endpoint store driver. Choose it for the internal ingestion path that needs RocketRide's preprocessor, embedding, and store components together; it is not a canvas-configured document parser or an agent tool.

Configuration

This node declares no fields or profiles of its own. Outside configuration-open mode, it looks for optional multi-provider preprocessor, embedding, and store sections in the connection configuration and initializes each present component. Vectorization requires the processing path to supply the components it uses: text is split with the preprocessor, chunks are encoded with the embedding component, and they are persisted or rendered through the store.

Notes

Chunk metadata and modes

Chunk and table counters reset for every object. Each chunk carries its chunk ID, whether it came from a table, its table ID, a false deletion flag, and the object permission ID (or -1 if absent). In instance mode, chunks are added directly to the store and the object is marked vectorized on close. In transform mode, chunks are written downstream as documents instead.

Rendering

For an object whose vectorBatchId is non-zero, the node asks the store to render the saved content into the text writer and prevents the default render path. Objects that have not been marked vectorized are left to the default renderer.

Schema

No configuration fields.