Metadata filters

Mastra provides a unified metadata filtering syntax across all vector stores, based on MongoDB/Sift query syntax. Each vector store translates these filters into its native query format. For example, PgVector uses PostgreSQL JSONB predicates, while OracleDB stores metadata as Oracle JSON and compiles filters to JSON_VALUE, JSON_EXISTS, REGEXP_LIKE, and LIKE predicates with bound values.

Basic example

import { PgVector } from '@mastra/pg'

const store = new PgVector({
  id: 'pg-vector',
  connectionString,
})

const results = await store.query({
  indexName: 'my_index',
  queryVector: queryVector,
  topK: 10,
  filter: {
    category: 'electronics', // Simple equality
    price: { $gt: 100 }, // Numeric comparison
    tags: { $in: ['sale', 'new'] }, // Array membership
  },
})

Supported operators

Basic Comparison

$eq

Matches values equal to specified value

{ age: { $eq: 25 } }

Supported by: All except Couchbase

$ne

Matches values not equal

{ status: { $ne: 'inactive' } }

Supported by: All except Couchbase

$gt

Greater than

{ price: { $gt: 100 } }

Supported by: All except Couchbase

$gte

Greater than or equal

{ rating: { $gte: 4.5 } }

Supported by: All except Couchbase

$lt

Less than

{ stock: { $lt: 20 } }

Supported by: All except Couchbase

$lte

Less than or equal

{ priority: { $lte: 3 } }

Supported by: All except Couchbase

Array Operators

$in

Matches any value in array

{ category: { $in: ["A", "B"] } }

Supported by: All except Couchbase

$nin

Matches none of the values

{ status: { $nin: ["deleted", "archived"] } }

Supported by: All except Couchbase

$all

Matches arrays containing all elements

{ tags: { $all: ["urgent", "high"] } }

Supported by: Astra, Pinecone, Upstash, MongoDB, OracleDB

$elemMatch

Matches array elements meeting criteria

{ scores: { $elemMatch: { $gt: 80 } } }

Supported by: libSQL, PgVector, MongoDB, OracleDB

Logical Operators

$and

Logical AND

{ $and: [{ price: { $gt: 100 } }, { stock: { $gt: 0 } }] }

Supported by: All except Vectorize, Couchbase

$or

Logical OR

{ $or: [{ status: "active" }, { priority: "high" }] }

Supported by: All except Vectorize, Couchbase

$not

Logical NOT

{ price: { $not: { $lt: 100 } } }

Supported by: Astra, Qdrant, Upstash, PgVector, libSQL, MongoDB, OracleDB

$nor

Logical NOR

{ $nor: [{ status: "deleted" }, { archived: true }] }

Supported by: Qdrant, Upstash, PgVector, libSQL, MongoDB, OracleDB

Element Operators

$exists

Matches documents with field

{ rating: { $exists: true } }

Supported by: All except Vectorize, Chroma, Couchbase

Custom Operators

$contains

Text contains substring

{ description: { $contains: "sale" } }

Supported by: Upstash, libSQL, PgVector, OracleDB

$regex

Regular expression match

{ name: { $regex: "^test" } }

Supported by: Qdrant, PgVector, Upstash, MongoDB, OracleDB

$size

Array length check

{ tags: { $size: 3 } }

Supported by: Astra, libSQL, PgVector, MongoDB, OracleDB

$geo

Geospatial query

{ location: { $geo: { type: "radius", ... } } }

Supported by: Qdrant

$datetime

Datetime range query

{ created: { $datetime: { range: { gt: "2024-01-01" } } } }

Supported by: Qdrant

$hasId

Vector ID existence check

{ $hasId: ["id1", "id2"] }

Supported by: Qdrant

$hasVector

Vector existence check

{ $hasVector: true }

Supported by: Qdrant

Common rules and restrictions

1. Field names can't:

   • Contain dots (.) unless referring to nested fields
   • Start with $ or contain null characters
   • Be empty strings

2. Values must be:

   • Valid JSON types (string, number, boolean, object, array)
   • Not undefined
   • Properly typed for the operator (e.g., numbers for numeric comparisons)

3. Logical operators:

   • Must contain valid conditions
   • Can't be empty
   • Must be properly nested
   • Can only be used at top level or nested within other logical operators
   • Can't be used at field level or nested inside a field
   • Can't be used inside an operator
   • Valid: { "$and": [{ "field": { "$gt": 100 } }] }
   • Valid: { "$or": [{ "$and": [{ "field": { "$gt": 100 } }] }] }
   • Invalid: { "field": { "$and": [{ "$gt": 100 }] } }
   • Invalid: { "field": { "$gt": { "$and": [{...}] } } }

4. $not operator:

   • Must be an object
   • Can't be empty
   • Can be used at field level or top level
   • Valid: { "$not": { "field": "value" } }
   • Valid: { "field": { "$not": { "$eq": "value" } } }

5. Operator nesting:

   • Logical operators must contain field conditions, not direct operators
   • Valid: { "$and": [{ "field": { "$gt": 100 } }] }
   • Invalid: { "$and": [{ "$gt": 100 }] }

Store-specific notes

Astra

• Nested field queries are supported using dot notation
• Array fields must be explicitly defined as arrays in the metadata
• Metadata values are case-sensitive

ChromaDB

• Where filters only return results where the filtered field exists in metadata
• Empty metadata fields aren't included in filter results
• Metadata fields must be present for negative matches (e.g., $ne won't match documents missing the field)

Cloudflare Vectorize

• Requires explicit metadata indexing before filtering can be used
• Use createMetadataIndex() to index fields you want to filter on
• Up to 10 metadata indexes per Vectorize index
• String values are indexed up to first 64 bytes (truncated on UTF-8 boundaries)
• Number values use float64 precision
• Filter JSON must be under 2048 bytes
• Field names can't contain dots (.) or start with $
• Field names limited to 512 characters
• Vectors must be re-upserted after creating new metadata indexes to be included in filtered results
• Range queries may have reduced accuracy with very large datasets (~10M+ vectors)

libSQL

• Supports nested object queries with dot notation
• Array fields are validated to ensure they contain valid JSON arrays
• Numeric comparisons maintain proper type handling
• Empty arrays in conditions are handled gracefully
• Metadata is stored in a JSONB column for efficient querying

OracleDB

• Metadata is stored as Oracle JSON alongside each VECTOR row
• Scalar comparisons use JSON_VALUE, while array, existence, and element-match checks use JSON_EXISTS
• $regex uses Oracle REGEXP_LIKE; string $contains uses case-insensitive LIKE
• Nested fields are supported with dot notation and are converted to quoted Oracle JSON paths
• User-provided metadata values are bound as parameters instead of interpolated into SQL

PgVector

• Full support for PostgreSQL's native JSON querying capabilities
• Efficient handling of array operations using native array functions
• Proper type handling for numbers, strings, and booleans
• Nested field queries use PostgreSQL's JSON path syntax internally
• Metadata is stored in a JSONB column for efficient indexing

Pinecone

• Metadata field names are limited to 512 characters
• Numeric values must be within the range of ±1e38
• Arrays in metadata are limited to 64KB total size
• Nested objects are flattened with dot notation
• Metadata updates replace the entire metadata object

Qdrant

• Supports advanced filtering with nested conditions
• Payload (metadata) fields must be explicitly indexed for filtering
• Use createPayloadIndex() to index fields you want to filter on:

// Index a field before filtering on it
await store.createPayloadIndex({
  indexName: 'my_index',
  fieldName: 'source',
  fieldSchema: 'keyword', // 'keyword' | 'integer' | 'float' | 'geo' | 'text' | 'bool' | 'datetime' | 'uuid'
})

// Now filtering works
const results = await store.query({
  indexName: 'my_index',
  queryVector: queryVector,
  filter: { source: 'document-a' },
})

• Efficient handling of geo-spatial queries
• Special handling for null and empty values
• Vector-specific filtering capabilities
• Datetime values must be in RFC 3339 format

Upstash

• 512-character limit for metadata field keys
• Query size is limited (avoid large IN clauses)
• No support for null/undefined values in filters
• Translates to SQL-like syntax internally
• Case-sensitive string comparisons
• Metadata updates are atomic

MongoDB

• Full support for MongoDB/Sift query syntax for metadata filters
• Supports all standard comparison, array, logical, and element operators
• Supports nested fields and arrays in metadata
• Filtering can be applied to both metadata and the original document content using the filter and documentFilter options, respectively
• filter applies to the metadata object; documentFilter applies to the original document fields
• No artificial limits on filter size or complexity (subject to MongoDB query limits)
• Indexing metadata fields is recommended for optimal performance

Couchbase

• Currently doesn't have support for metadata filters. Filtering must be done client-side after retrieving results or by using the Couchbase SDK's Search capabilities directly for more complex queries.

Amazon S3 Vectors

• Equality values must be primitives (string/number/boolean). null/undefined, arrays, objects, and Date aren't allowed for equality. Range operators accept numbers or Date (Dates are normalized to epoch ms).
• $in/$nin require non-empty arrays of primitives; Date elements are allowed and normalized to epoch ms. Array equality isn't supported.
• Implicit AND is canonicalized ({a:1,b:2} → {$and:[{a:1},{b:2}]). Logical operators must contain field conditions and use non-empty arrays. They may appear only at the root or within other logical operators (not inside field values).
• Keys listed in nonFilterableMetadataKeys at index creation are stored but not filterable. This setting is immutable.
• $exists requires a boolean value.
• undefined/null/empty filters are treated as no filter.
• Each metadata key name limited to 63 characters.
• Total metadata per vector: Up to 40 KB (filterable + non-filterable)
• Total metadata keys per vector: Up to 10
• Filterable metadata per vector: Up to 2 KB
• Non-filterable metadata keys per vector index: Up to 10

Related

• Astra
• Chroma
• Cloudflare Vectorize
• libSQL
• MongoDB
• OracleDB
• PgStore
• Pinecone
• Qdrant
• Upstash
• Amazon S3 Vectors