How to Index Vector Embeddings for Vector Search

How to Index Vector Embeddings for Vector Search

For the complete MongoDB documentation index, see www.mongodb.com/docs/llms.txt

How to Index Vector Embeddings for Vector Search

You can use the knnVector type to index vector embeddings. The vector field can be represented as an array of [numbers](https://www.mongodb.com/docs/search/indexes/field-types/number-type.md#std-label-bson-data-types-number of the following types:

BSON (Binary Javascript Object Notation) int32, int64, or double data types for querying using the [knnBeta](https://www.mongodb.com/docs/search/query/operators-collectors/knn-beta.md#std-label-knn-beta-ref operator.
BSON (Binary Javascript Object Notation) double data type for querying using the [$vectorSearch](https://www.mongodb.com/docs/vector-search/query/aggregation-stages/vector-search-stage.md#mongodb-pipeline-pipe.-vectorSearch stage.

You can use the [knnBeta](https://www.mongodb.com/docs/search/query/operators-collectors/knn-beta.md#std-label-knn-beta-ref operator, which is now deprecated, and the [$vectorSearch](https://www.mongodb.com/docs/vector-search/query/aggregation-stages/vector-search-stage.md#mongodb-pipeline-pipe.-vectorSearch stage in your aggregation pipeline to query fields indexed as knnVector.

Note:

You can't use the MongoDB Search Visual Editor in the Atlas UI to configure fields of type knnVector. Instead, use the MongoDB Search JSON Editor to configure fields of type knnVector.

You can also use MongoDB Vector Search with local Atlas deployments that you create with the Atlas CLI. To learn more, see Create a Local Atlas Deployment.

`knnVector` Type Limitations

You can't index fields inside arrays of documents or fields inside arrays of objects (MongoDB Search embeddedDocuments as knnVector type.

Define the Index for the `knnVector` Type

The following is the JSON (Javascript Object Notation) syntax for the knnVector type. Replace the default index definition with the following. To learn more about the fields, see [Field Properties](https://www.mongodb.com/docs/search/indexes/field-types/vector-type.md#std-label-fts-vector-field-type-properties.

{
  "mappings": {
    "dynamic": true|false,
    "fields": {
      "<field-name>": {
        "type": "knnVector",
        "dimensions": <number-of-dimensions>,
        "similarity": "euclidean | cosine | dotProduct"
      }
    }
  }
}

Configure `knnVector` Field Properties

The knnVector type has the following options:

Option	Type	Necessity	Purpose
`type`	string	Required	Human-readable label that identifies this field type. Value must be `knnVector`.
`dimensions`	int	Required	Number of vector dimensions, which MongoDB Search enforces at index- and query-time. This value can't be greater than `8192`.
`similarity`	string	Required	Vector similarity function to use to search for top K-nearest neighbors. Value can be one of the following: `euclidean` - measures the distance between ends of vectors. This allows you to measure similarity based on varying dimensions. To learn more, see Euclidean.; `cosine` - measures similarity based on the angle between vectors. This allows you to measure similarity that isn't scaled by magnitude. You can't use zero magnitude vectors with `cosine`. To measure cosine similarity, we recommend that you normalize your vectors and use `dotProduct` instead. To learn more, see Cosine.; `dotProduct` - measures similarity like `cosine`, but takes into account the magnitude of the vector. This allows you to efficiently measure similarity based on both angle and magnitude. To learn more, see Dot Product.If you normalize the magnitude, `cosine` and `dotProduct` are almost identical in measuring similarity.

Try an Example for the `knnVector` Type

The following index definition for the sample_mflix.embedded_movies collection dynamically indexes all the dynamically indexable fields in the collection and statically indexes plot_embedding field as the knnVector type. The plot_embedding field contains embeddings created using OpenAI's text-embedding-ada-002 embeddings model. The index definition specifies 1536 vector dimensions and measures similarity using euclidean.

{
  "mappings": {
    "dynamic": true,
    "fields": {
      "plot_embedding": {
        "type": "knnVector",
        "dimensions": 1536,
        "similarity": "euclidean"
      }
    }
  }
}

If you load the [sample data](https://www.mongodb.com/docs/manual/sample-data/sample-mflix.md#std-label-mflix-embedded_movies on your cluster and create the preceding MongoDB Search index for this collection, you can run [$vectorSearch](https://www.mongodb.com/docs/vector-search/query/aggregation-stages/vector-search-stage.md#mongodb-pipeline-pipe.-vectorSearch queries against this collection. To learn more about the sample queries that you can run, see [$vectorSearch Examples.](https://www.mongodb.com/docs/vector-search/query/aggregation-stages/vector-search-stage.md#std-label-vectorSearch-agg-pipeline-egs

Migrate to `vector` Type

To migrate your MongoDB Search indexes to the [vector](https://www.mongodb.com/docs/search/indexes/field-types/vector-type.md#std-label-bson-data-types-vector type, replace the knnVector type in your index definition for the vector field with the vector type.

Example:

To use the vector type for the plot_embedding field in the [sample index definition](https://www.mongodb.com/docs/search/indexes/field-types/knn-vector.md#std-label-knn-vector-egs for the sample_mflix.embedded_movies collection, replace the index definition with the following:

{
  "mappings": {
    "dynamic": true,
    "fields": {
      "plot_embedding": {
        "type": "vector",
        "dimensions": 1536,
        "similarity": "euclidean"
      }
    }
  }
}

To run queries against the vector type field, you must use the [vectorSearch (MongoDB Search Operator)](https://www.mongodb.com/docs/search/query/operators-collectors/vectorsearch.md#std-label-fts-vectorSearch-ref instead of the [knnBeta Operator.](https://www.mongodb.com/docs/search/query/operators-collectors/knn-beta.md#std-label-knn-beta-ref

Note:

Your existing [knnBeta Operator](https://www.mongodb.com/docs/search/query/operators-collectors/knn-beta.md#std-label-knn-beta-ref queries can continue to run against the vector type index. After migrating the index, we recommend immediately migrating your knnBeta operator queries to the [vectorSearch (MongoDB Search Operator).](https://www.mongodb.com/docs/search/query/operators-collectors/vectorsearch.md#std-label-fts-vectorSearch-ref

Get Started with Network Egress

How to Index Vector Embeddings for Vector Search