Optimize and Measure Performance

For the complete MongoDB documentation index, see www.mongodb.com/docs/llms.txt

Optimize and Measure Performance

AI applications can often start small in terms of compute, data, and monetary costs. As production applications scale due to increased user engagement, key factors such as the cost associated with storing and retrieving large volumes of data become critical optimization opportunities. These challenges can be addressed by focusing on:

• Efficient Vector Search Algorithms

• Automated Quantization Processes

• Optimized Embedding Strategies

Both Retrieval-Augmented Generation (RAG) and agent-based systems rely on vector data—numerical representations of data objects like images, videos, and text—to perform semantic similarity searches. Systems that use RAG or agent-driven workflows must efficiently handle massive, high-dimensional data sets to maintain fast response times, minimize retrieval latency, and control infrastructure costs.

About the Tutorial

This tutorial equips you with the techniques needed to design, deploy, and manage advanced AI workloads at scale, ensuring optimal performance and cost efficiency.

Specifically, in this tutorial, you'll learn how to:

• Generate embeddings using Voyage AI's voyage-3-large, a general purpose, multilingual embedding model that is also quantization aware, and ingest them into a MongoDB database.

• Automatically quantize the embeddings to lower precision data types, optimizing both memory usage and query latency.

• Run a query that compares float32, int8, and binary embeddings, weighing data type precision against efficiency and retrieval accuracy.

• Measure the recall (also referred to as retention) of the quantized embeddings, which evaluates how effectively the quantized ANN (Approximate Nearest Neighbor) search retrieves the same documents as a full-precision ENN (Exact Nearest Neighbor) search.

Note:

• Binary quantization is optimal for scenarios demanding reduced resource consumption, though it may require a rescoring pass to address any loss in accuracy.

• Scalar quantization offers a practical middle ground, suitable for most use cases that need to balance performance and precision.

• Float32 ensures maximum fidelity but has the steepest performance and memory overhead, making it less ideal for large-scale or latency-sensitive systems.

Work with a runnable version of this tutorial as a Python notebook.

Prerequisites

To complete this tutorial, you must have the following:

• An M20 or higher cluster with 2 or more search nodes using High-CPU S20 or higher search tier.

• An environment to run interactive Python notebooks such as VS Code or Colab.

• A paid-tier Voyage AI API key. A free-tier key does not support the request volume required for this tutorial.

We also recommend:

• A Hugging Face access token. Although not a requirement, using a Hugging Face access token increases the download speed of the Wikipedia datasets this tutorial uses.

Procedure

1. Import required libraries and set the environment variables.

Create an interactive Python notebook by saving a file with the .ipynb extension.

Install the libraries.

For this tutorial, you must import the following libraries:

To install the libraries, run the following:

text pip install --quiet -U pymongo voyageai pandas datasets matplotlib

Securely get and set environment variables.

The following set_env_securely helper function gets and sets environment variables securely. Copy, paste, and run the following code and when prompted, set secret values such as your Voyage AI API (Application Programming Interface) key and your cluster connection string. You are also prompted for an optional Hugging Face access token. Providing a token increases rate limits and speeds up dataset downloads. Press Enter to skip.

```python import getpass import os import voyageai import huggingface_hub

# Function to securely get and set environment variables def set_env_securely(var_name, prompt): value = getpass.getpass(prompt) os.environ[var_name] = value

# Environment Variables set_env_securely("VOYAGE_API_KEY", "Enter your Voyage API Key: ") set_env_securely("MONGO_URI", "Enter your MongoDB URI: ") MONGO_URI = os.environ.get("MONGO_URI") if not MONGO_URI: raise ValueError("MONGO_URI not set in environment variables.")

# Voyage Client voyage_client = voyageai.Client() # Hugging Face Token (optional) # Providing a token increases rate limits and speeds up dataset downloads. # Press Enter to skip. hf_token = getpass.getpass("Enter your Hugging Face token (optional, press Enter to skip): ") if hf_token: os.environ["HF_TOKEN"] = hf_token huggingface_hub.login(hf_token) print("Hugging Face token set.") else: print("No Hugging Face token provided. You may experience slower download speeds.") ```

1. Ingest the data into your cluster.

In this step, you load up to 250000 documents from the following datasets:

##### wikipedia-22-12-en-voyage-embed

###### Contains the wikipedia data with the embeddings for each document.

The wikipedia-22-12-en-voyage-embed dataset contains Wikipedia article fragments with pre-generated 1024-dimensional float32 embeddings from Voyage AI's voyage-3-large model. This is the primary document collection with metadata. This dataset serves as a diverse vector corpus for testing vector quantization effects in semantic search. Each document in this dataset contains the following fields:

##### wikipedia-22-12-en-annotation

###### Contains the annotation data for the wikipedia data.

The wikipedia-22-12-en-annotation dataset contains the annotated reference data for recall measurement function. This data is used as benchmark dataset for accuracy validation and to evaluate quantization impact on retrieval quality. Each document in this dataset contains the following fields, which are the ground truth used to evaluate the performance of the vector search:

Define the functions to load the data into your cluster.

Copy, paste, and run the following code in your notebook. The sample code defines the following functions:

• generate_bson_vector to convert the embeddings in the dataset to BSON (Binary Javascript Object Notation) binary vectors for efficient storage and processing of your vectors.

• get_mongo_client to get your cluster connection string.

• insert_dataframe_into_collection to ingest data into the cluster.

```python import pandas as pd from datasets import load_dataset from bson.binary import Binary, BinaryVectorDtype import pymongo

# Connect to Cluster def get_mongo_client(uri): """Connect to MongoDB and confirm the connection.""" client = pymongo.MongoClient(uri) if client.admin.command("ping").get("ok") == 1.0: print("Connected to MongoDB successfully.") return client print("Failed to connect to MongoDB.") return None

# Generate BSON Vector def generate_bson_vector(array, data_type): """Convert an array to BSON vector format.""" array = [float(val) for val in eval(array)] return Binary.from_vector(array, BinaryVectorDtype(data_type))

# Load Datasets def load_and_prepare_data(dataset_name, amount): """Load and prepare streaming datasets for DataFrame.""" print(f"Loading {amount} records from {dataset_name}...") data = load_dataset(dataset_name, streaming=True, split="train").take(amount) return pd.DataFrame(data)

# Insert datasets into MongoDB Collection def insert_dataframe_into_collection(df, collection): """Insert Dataset records into MongoDB collection.""" collection.insert_many(df.to_dict("records")) print(f"Inserted {len(df)} records into '{collection.name}' collection.") ```

Load the data into your cluster.

Copy, paste, and run the following code in your notebook to load the dataset into your cluster. This code performs the following actions:

• Fetches the datasets.

• Converts the embeddings to BSON (Binary Javascript Object Notation) format.

• Creates collections in your cluster and inserts the data.

```python import pandas as pd from bson.binary import Binary, BinaryVectorDtype from pymongo.errors import CollectionInvalid

wikipedia_data_df = load_and_prepare_data("MongoDB/wikipedia-22-12-en-voyage-embed", amount=250000) wikipedia_annotation_data_df = load_and_prepare_data("MongoDB/wikipedia-22-12-en-annotation", amount=250000) wikipedia_annotation_data_df.drop(columns=["_id"], inplace=True)

# Convert embeddings to BSON format wikipedia_data_df["embedding"] = wikipedia_data_df["embedding"].apply( lambda x: generate_bson_vector(x, BinaryVectorDtype.FLOAT32) )

# MongoDB Setup mongo_client = get_mongo_client(MONGO_URI) DB_NAME = "testing_datasets" db = mongo_client[DB_NAME]

collections = { "wikipedia-22-12-en": wikipedia_data_df, "wikipedia-22-12-en-annotation": wikipedia_annotation_data_df, }

# Create Collections and Insert Data for collection_name, df in collections.items(): if collection_name not in db.list_collection_names(): try: db.create_collection(collection_name) print(f"Collection '{collection_name}' created successfully.") except CollectionInvalid: print(f"Error creating collection '{collection_name}'.") else: print(f"Collection '{collection_name}' already exists.")

   # Clear collection and insert fresh data
   collection = db[collection_name]
   collection.delete_many({})
   insert_dataframe_into_collection(df, collection)

```

Output:

shell Loading 250000 records from MongoDB/wikipedia-22-12-en-voyage-embed... Loading 250000 records from MongoDB/wikipedia-22-12-en-annotation... Connected to MongoDB successfully. Collection 'wikipedia-22-12-en' created successfully. Inserted 250000 records into 'wikipedia-22-12-en' collection. Collection 'wikipedia-22-12-en-annotation' created successfully. Inserted 87200 records into 'wikipedia-22-12-en-annotation' collection.

IMPORTANT: It might take some time to convert embeddings to BSON (Binary Javascript Object Notation) vectors and ingest the datasets into your cluster.

Verify that the datasets loaded successfully by logging into your cluster and visually inspecting the collections in [Data Explorer.](https://www.mongodb.com/docs/atlas/atlas-ui.md#std-label-atlas-ui

1. Create MongoDB Vector Search indexes on the collection.

In this step, you create the following three indexes on the embedding field:

Define the function to create MongoDB Vector Search index.

Copy, paste, and run the following in your notebook:

```python import time from pymongo.operations import SearchIndexModel

def setup_vector_search_index(collection, index_definition, index_name="vector_index"): new_vector_search_index_model = SearchIndexModel( definition=index_definition, name=index_name, type="vectorSearch" )

   # Create the new index
   try:
       result = collection.create_search_index(model=new_vector_search_index_model)
       print(f"Creating index '{index_name}'...")

       # Wait for initial sync to complete
       print("Polling to check if the index is ready. This might take a few minutes.")
       predicate=None
       if predicate is None:
           predicate = lambda index: index.get("queryable") is True
           while True:
               indices = list(collection.list_search_indexes(result))
               if len(indices) and predicate(indices[0]):
                   break
               time.sleep(5)
       print(f"Index '{index_name}' is ready for querying.")
       return result

   except Exception as e:
       print(f"Error creating new vector search index '{index_name}': {e!s}")
       return None

```

Define the indexes.

The following index configurations implement a different quantization strategy:

The automatic quantization happens transparently when these indexes are created, with MongoDB Vector Search handling the conversion from float32 to the specified quantized format during index creation and search operations.

The vector_index_definition_float32_ann index configuration indexes full fidelity vectors of 1024 dimensions by using the cosine similarity function.

python # Scalar Quantization vector_index_definition_scalar_quantized = { "fields": [ { "type": "vector", "path": "embedding", "quantization": "scalar", "numDimensions": 1024, "similarity": "cosine", } ] } # Binary Quantization vector_index_definition_binary_quantized = { "fields": [ { "type": "vector", "path": "embedding", "quantization": "binary", "numDimensions": 1024, "similarity": "cosine", } ] } # Float32 Embeddings vector_index_definition_float32_ann = { "fields": [ { "type": "vector", "path": "embedding", "numDimensions": 1024, "similarity": "cosine", } ] }

Create the scalar, binary, and float32 indexes by using the setup_vector_search_index function.

Set the collection and index names for the indexes.

python wiki_data_collection = db["wikipedia-22-12-en"] wiki_annotation_data_collection = db["wikipedia-22-12-en-annotation"] vector_search_scalar_quantized_index_name = "vector_index_scalar_quantized" vector_search_binary_quantized_index_name = "vector_index_binary_quantized" vector_search_float32_ann_index_name = "vector_index_float32_ann"

Create the MongoDB Vector Search indexes.

python setup_vector_search_index( wiki_data_collection, vector_index_definition_scalar_quantized, vector_search_scalar_quantized_index_name, ) setup_vector_search_index( wiki_data_collection, vector_index_definition_binary_quantized, vector_search_binary_quantized_index_name, ) setup_vector_search_index( wiki_data_collection, vector_index_definition_float32_ann, vector_search_float32_ann_index_name, )

Output:

shell Creating index 'vector_index_scalar_quantized'... Polling to check if the index is ready. This might take a few minutes. Index 'vector_index_scalar_quantized' is ready for querying. Creating index 'vector_index_binary_quantized'... Polling to check if the index is ready. This might take a few minutes. Index 'vector_index_binary_quantized' is ready for querying. Creating index 'vector_index_float32_ann'... Polling to check if the index is ready. This might take a few minutes. Index 'vector_index_float32_ann' is ready for querying. vector_index_float32_ann'

IMPORTANT: The operation might take a few minutes to complete. Indexes must be in Ready state to use them in queries.

Verify that the index creation succeeded by logging into your cluster and visually inspecting the indexes in Search & Vector Search page.

1. Define the functions to generate embeddings and query a collection using the MongoDB Vector Search indexes.

This code defines the following functions:

• The get_embedding() function generates 1024 dimension embeddings for the given text by using Voyage AI's voyage-3-large embedding model.

• The custom_vector_search function takes the following input parameters and returns the results of the vector search operation.

  user_query	Query text string for which to generate embeddings.
  collection	MongoDB collection to search.
  embedding_path	Field in the collection that contains the embeddings.
  vector_search_index_name	Name of index to use in the query.
  top_k	Number of top documents in the results to return.
  num_candidates	Number of candidates to consider.
  use_full_precision	Flag to perform ANN (Approximate Nearest Neighbor), if False, or ENN (Exact Nearest Neighbor), if True, search. The use_full_precision value is set to False by default for an ANN (Approximate Nearest Neighbor) search. Set the use_full_precision value to True to perform an ENN (Exact Nearest Neighbor) search. Specifically, this function performs the following actions: Generates the embeddings for the query text; Constructs the [$vectorSearch](https://www.mongodb.com/docs/vector-search/query/aggregation-stages/vector-search-stage.md#mongodb-pipeline-pipe.-vectorSearch stage; Configures the type of search; Specifies the fields in the collection to return; Executes the pipeline after gathering performance statistics; Returns the results

```python

def get_embedding(text, task_prefix="document"): """Fetch embedding for a given text using Voyage AI.""" if not text.strip(): print("Empty text provided for embedding.") return [] result = voyage_client.embed([text], model="voyage-3-large", input_type=task_prefix) return result.embeddings[0]

def custom_vector_search( user_query, collection, embedding_path, vector_search_index_name="vector_index", top_k=5, num_candidates=25, use_full_precision=False, ): """Perform vector search on a MongoDB collection using specified index.""" # Generate embedding for the user query query_embedding = get_embedding(user_query, task_prefix="query")

   if query_embedding is None or not query_embedding:
       return {
           "error": "Invalid query or embedding generation failed.",
           "execution_time_ms": None,
           "results": [],
       }

   # Define the vector search stage
   vector_search_stage = {
       "$vectorSearch": {
           "index": vector_search_index_name,
           "queryVector": query_embedding,
           "path": embedding_path,
           "limit": top_k,
       }
   }

   # Configure search precision approach
   if not use_full_precision:
       # For approximate nearest neighbor (ANN) search
       vector_search_stage["$vectorSearch"]["numCandidates"] = num_candidates
   else:
       # For exact nearest neighbor (ENN) search
       vector_search_stage["$vectorSearch"]["exact"] = True

   # Project stage to fetch desired fields and vector search score
   project_stage = {
       "$project": {
           "_id": 0,
           "title": 1,
           "text": 1,
           "wiki_id": 1,
           "url": 1,
           "score": {"$meta": "vectorSearchScore"},
       }
   }

   # Define the aggregate pipeline
   pipeline = [vector_search_stage, project_stage]

   try:
       # Execute the explain command to measure latency
       explain_result = collection.database.command(
           "explain",
           {"aggregate": collection.name, "pipeline": pipeline, "cursor": {}},
           verbosity="executionStats",
       )

       # Extract the execution time
       vector_search_explain = explain_result["stages"][0]["$vectorSearch"]
       execution_time_ms = vector_search_explain["explain"]["query"]["stats"]["context"]["millisElapsed"]

       # Execute the actual aggregate query
       results = list(collection.aggregate(pipeline))

       return {
           "results": results,
           "execution_time_ms": execution_time_ms,
       }
   except Exception as e:
       print(f"Error during vector search: {e}")
       return {
           "error": str(e),
           "execution_time_ms": None,
           "results": [],
       }

```

1. Run a MongoDB Vector Search query to evaluate the search performance.

The following query performs the vector searches across different quantization strategies, measuring performance metrics for scalar quantized, binary quantized, and full precision (float32) vectors while capturing latency measurements at each precision level and standardizing the result format for analytical comparison. It uses embeddings generated using Voyage AI for the query string “How do I increase my productivity for maximum output”.

The query stores key essential performance indicators in the results variable including precision level (Scalar, Binary, Float32), result set size (top_k), query latency in milliseconds, and retrieved document content, providing comprehensive metrics for evaluating search performance across different quantization strategies.

```python vector_search_indices = [ vector_search_float32_ann_index_name, vector_search_scalar_quantized_index_name, vector_search_binary_quantized_index_name, ]

# Random query user_query = "How do I increase my productivity for maximum output" test_top_k = 5 test_num_candidates = 25

# Result is a list of dictionaries with the following headings: precision, top_k, latency_ms, results results = []

for vector_search_index in vector_search_indices: # Conduct a vector search operation using scalar quantized vector_search_results = custom_vector_search( user_query, wiki_data_collection, embedding_path="embedding", vector_search_index_name=vector_search_index, top_k=test_top_k, num_candidates=test_num_candidates, use_full_precision=False, ) # Include the precision in the results precision = vector_search_index.split("vector_index")[1] precision = precision.replace("quantized", "").capitalize()

   results.append(
       {
           "precision": precision,
           "top_k": test_top_k,
           "num_candidates": test_num_candidates,
           "latency_ms": vector_search_results["execution_time_ms"],
           "results": vector_search_results["results"][0],  # Just taking the first result, modify this to include more results if needed
       }
   )

# Conduct a vector search operation using full precision precision = "Float32_ENN" vector_search_results = custom_vector_search( user_query, wiki_data_collection, embedding_path="embedding", vector_search_index_name="vector_index_scalar_quantized", top_k=test_top_k, num_candidates=test_num_candidates, use_full_precision=True, )

results.append( { "precision": precision, "top_k": test_top_k, "num_candidates": test_num_candidates, "latency_ms": vector_search_results["execution_time_ms"], "results": vector_search_results["results"][0], # Just taking the first result, modify this to include more results if needed } )

# Convert the results to a pandas DataFrame with the headings: precision, top_k, latency_ms results_df = pd.DataFrame(results) results_df.columns = ["precision", "top_k", "num_candidates", "latency_ms", "results"]

# To display the results: results_df.head() ```

Output:

python precision top_k num_candidates latency_ms results 0 _float32_ann 5 25 1659.498601 {'title': 'Henry Ford', 'text': 'Ford had deci... 1 _scalar_ 5 25 951.537687 {'title': 'Gross domestic product', 'text': 'F... 2 _binary_ 5 25 344.585193 {'title': 'Great Depression', 'text': 'The fir... 3 Float32_ENN 5 25 0.231693 {'title': 'Great Depression', 'text': 'The fir...

The performance metrics in the results show latency differences across precision levels. This demonstrates that while quantization provides substantial performance improvements, there's a clear trade-off between precision and retrieval speed, with full-precision float32 operations requiring notably more computational time compared to their quantized counterparts.

1. Measure latency with varying top-k and num_candidates values.

The following query introduces a systematic latency measurement framewwork that evaluates vector search performance across different precision levels and retrieval scales. The parameter top-k not only determines the number of results to return but also sets the numCandidates parameter in MongoDB's HNSW (Hierarchical Navigable Small World) graph search.

The numCandidates value influences how many nodes in the HNSW (Hierarchical Navigable Small World) graph MongoDB Vector Search explores during the ANN (Approximate Nearest Neighbor) search. Here, a higher value increases the likelihood of finding the true nearest neighbors but requires more computation time.

Define the function to format the latency_ms to a human-readable format.

```python from datetime import timedelta

def format_time(ms): """Convert milliseconds to a human-readable format""" delta = timedelta(milliseconds=ms)

   # Extract minutes, seconds, and milliseconds with more precision
   minutes = delta.seconds // 60
   seconds = delta.seconds % 60
   milliseconds = round(ms % 1000, 3)  # Keep 3 decimal places for milliseconds

   # Format based on duration
   if minutes > 0:
       return f"{minutes}m {seconds}.{milliseconds:03.0f}s"
   elif seconds > 0:
       return f"{seconds}.{milliseconds:03.0f}s"
   else:
       return f"{milliseconds:.3f}ms"

```

Define the function to measure the latency of the vector search query.

The following function takes a user_query, a collection, a vector_search_index_name, a use_full_precision value, a top_k_values value, and a num_candidates_values value as input and returns the results of the vector search. Here, make a note of the following:

• The latency increases as the top_k and num_candidates values increase because the vector search operation uses a larger number of documents and causes the search to take longer.

• The latency is higher for full fidelity search (use_full_precision=True) than for approximate search (use_full_precision=False) because full fidelity search takes longer than approximate search as it searches the entire dataset, using the full precision float32 vectors.

• The latency of quantized search is lower than full fidelity search because the quantized search uses the approximate search and the quantized vectors.

```python def measure_latency_with_varying_topk( user_query, collection, vector_search_index_name="vector_index_scalar_quantized", use_full_precision=False, top_k_values=[5, 10, 100], num_candidates_values=[25, 50, 100, 200, 500, 1000, 2000, 5000, 10000], ): results_data = []

   # Conduct vector search operation for each (top_k, num_candidates) combination
   for top_k in top_k_values:
       for num_candidates in num_candidates_values:
           # Skip scenarios where num_candidates < top_k
           if num_candidates < top_k:
               continue

           # Construct the precision name
           precision_name = vector_search_index_name.split("vector_index")[1]
           precision_name = precision_name.replace("quantized", "").capitalize()

           # If use_full_precision is true, then the precision name is "_float32_"
           if use_full_precision:
               precision_name = "_float32_ENN"

           # Perform the vector search
           vector_search_results = custom_vector_search(
               user_query=user_query,
               collection=collection,
               embedding_path="embedding",
               vector_search_index_name=vector_search_index_name,
               top_k=top_k,
               num_candidates=num_candidates,
               use_full_precision=use_full_precision,
           )

           # Extract the execution time (latency)
           latency_ms = vector_search_results["execution_time_ms"]

           # Append the results to the list
           results_data.append(
               {
                   "precision": precision_name,
                   "top_k": top_k,
                   "num_candidates": num_candidates,
                   "latency_ms": latency_ms,
               }
           )

           print(f"Top-K: {top_k}, NumCandidates: {num_candidates}, "
                 f"Latency: {format_time(latency_ms)}, Precision: {precision_name}")

   return results_data

```

Run the MongoDB Vector Search query to measure the latency.

The latency evaluation operation conducts a comprehensive performance analysis by executing searches across all quantization strategies, testing multiple result set sizes, capturing standardized performance metrics, and aggregating results for comparative analysis, enabling detailed evaluation of vector search behavior under different configurations and retrieval loads.

```python # Run the measurements user_query = "How do I increase my productivity for maximum output" top_k_values = [5, 10, 50, 100] num_candidates_values = [25, 50, 100, 200, 500, 1000, 2000, 5000, 10000]

latency_results = []

for vector_search_index in vector_search_indices: latency_results.append( measure_latency_with_varying_topk( user_query, wiki_data_collection, vector_search_index_name=vector_search_index, use_full_precision=False, top_k_values=top_k_values, num_candidates_values=num_candidates_values, ) )

# Conduct vector search operation using full precision latency_results.append( measure_latency_with_varying_topk( user_query, wiki_data_collection, vector_search_index_name="vector_index_scalar_quantized", use_full_precision=True, top_k_values=top_k_values, num_candidates_values=num_candidates_values, ) )

```

Output:

python Top-K: 5, NumCandidates: 25, Latency: 1.673s, Precision: _float32_ann ... Top-K: 100, NumCandidates: 10000, Latency: 184.905ms, Precision: _float32_ann Top-K: 5, NumCandidates: 25, Latency: 828.459ms, Precision: _scalar_ ... Top-K: 100, NumCandidates: 10000, Latency: 214.200ms, Precision: _scalar_ Top-K: 5, NumCandidates: 25, Latency: 400.160ms, Precision: _binary_ ... Top-K: 100, NumCandidates: 10000, Latency: 360.909ms, Precision: _binary_ Top-K: 5, NumCandidates: 25, Latency: 0.239ms, Precision: _float32_ENN ... Top-K: 100, NumCandidates: 10000, Latency: 0.179ms, Precision: _float32_ENN

The latency measurements reveal a clear performance hierarchy across precision types, where binary quantization demonstrates the fastest retrieval times, followed by scalar quantization. The full precision float32 ANN (Approximate Nearest Neighbor) operations show significantly higher latencies. The performance gap between quantized and full precision searches become more pronounced as Top-K values increase. The float32 ENN (Exact Nearest Neighbor) operations are the slowest, but they provide the highest precision results.

Plot the search latency against various top-k values.

```python import matplotlib.pyplot as plt

# Map your precision field to the labels and colors you want in the legend precision_label_map = { "scalar": "scalar", "binary": "binary", "_float32_ann": "float32_ann", "_float32_ENN": "float32_ENN", }

precision_color_map = { "scalar": "orange", "binary": "red", "_float32_ann": "blue", "_float32_ENN": "purple", }

# Flatten all measurements and find the unique top_k values all_measurements = [m for precision_list in latency_results for m in precision_list] unique_topk = sorted(set(m["top_k"] for m in all_measurements))

# For each top_k, create a separate plot for k in unique_topk: plt.figure(figsize=(10, 6))

   # For each precision type, filter out measurements for the current top_k value
   for measurements in latency_results:
       # Filter measurements with top_k equal to the current k
       filtered = [m for m in measurements if m["top_k"] == k]
       if not filtered:
           continue

       # Extract x (num_candidates) and y (latency) values
       x = [m["num_candidates"] for m in filtered]
       y = [m["latency_ms"] for m in filtered]

       # Determine the precision, label, and color from the first measurement in this filtered list
       precision = filtered[0]["precision"]
       label = precision_label_map.get(precision, precision)
       color = precision_color_map.get(precision, "blue")

       # Plot the line for this precision type
       plt.plot(x, y, marker="o", color=color, label=label)

   # Label axes and add title including the top_k value
   plt.xlabel("Number of Candidates")
   plt.ylabel("Latency (ms)")
   plt.title(f"Search Latency vs Num Candidates for Top-K = {k}")

   # Add a legend and grid, then show the plot
   plt.legend()
   plt.grid(True)
   plt.show()

```

The code returns the following latency charts, which illustrate how vector search document retrieval performs with different embedding precision types, binary, scalar, and float32, as the top-k (the number of results retrieved) increases:

##### Search Latency vs Num Candidates for Top-K = 5

###### The search latency for each precision type with the measurements for the top 5 results.

##### Search Latency vs Num Candidates for Top-K = 10

###### The search latency for each precision type with the measurements for the top 10 results.

##### Search Latency vs Num Candidates for Top-K = 50

###### The search latency for each precision type with the measurements for the top 50 results.

##### Search Latency vs Num Candidates for Top-K = 100

###### The search latency for each precision type with the measurements for the top 100 results.

1. Measure the representation capacity and retention.

The following query measures how effectively MongoDB Vector Search retrieves relevant documents from the ground truth dataset. It is calculated as the ratio of correctly found relevant documents to the total number of relevant documents in the ground truth (Found/Total). For example, if a query has 5 relevant documents in the ground truth and MongoDB Vector Search finds 4 of them, the recall would be 0.8 or 80%.

Define a function to measure the representational capacity and retention of the vector search operation. This function does the following:

Creates the baseline search using the full precision float32 vectors and ENN (Exact Nearest Neighbor) search.

Creates the quantized search using the quantized vectors and ANN (Approximate Nearest Neighbor) search.

Computes the retention of the quantized search compared to the baseline search.

The retention must be maintained within a reasonable range for the quantized search. If the representational capacity is low, it means that the vector search operation is not able to capture the semantic meaning of the query and the results might not be accurate. This indicates that the quantization is not effective and the initial embedding model used is not effective for the quantization process. We recommend utilizing embedding models that are quantization aware, meaning that during the training process, the model is specifically optimized to produce embeddings that maintain their semantic properties even after quantization.

```python def measure_representational_capacity_retention_against_float_enn( ground_truth_collection, collection, quantized_index_name, # This is used for both the quantized search and (with use_full_precision=True) for the baseline. top_k_values, # List/array of top-k values to test. num_candidates_values, # List/array of num_candidates values to test. num_queries_to_test=1, ): retention_results = {"per_query_retention": {}} overall_retention = {} # overall_retention[top_k][num_candidates] = [list of retention values]

   # Initialize overall retention structure
   for top_k in top_k_values:
       overall_retention[top_k] = {}
       for num_candidates in num_candidates_values:
           if num_candidates < top_k:
               continue
           overall_retention[top_k][num_candidates] = []

   # Extract and store the precision name from the quantized index name.
   precision_name = quantized_index_name.split("vector_index")[1]
   precision_name = precision_name.replace("quantized", "").capitalize()
   retention_results["precision_name"] = precision_name
   retention_results["top_k_values"] = top_k_values
   retention_results["num_candidates_values"] = num_candidates_values

   # Load ground truth annotations
   ground_truth_annotations = list(
       ground_truth_collection.find().limit(num_queries_to_test)
   )
   print(f"Loaded {len(ground_truth_annotations)} ground truth annotations")

   # Process each ground truth annotation
   for annotation in ground_truth_annotations:
       # Use the ground truth wiki_id from the annotation.
       ground_truth_wiki_id = annotation["wiki_id"]

       # Process only queries that are questions.
       for query_type, queries in annotation["queries"].items():
           if query_type.lower() not in ["question", "questions"]:
               continue

           for query in queries:
               # Prepare nested dict for this query
               if query not in retention_results["per_query_retention"]:
                   retention_results["per_query_retention"][query] = {}

               # For each valid combination of top_k and num_candidates
               for top_k in top_k_values:
                   if top_k not in retention_results["per_query_retention"][query]:
                       retention_results["per_query_retention"][query][top_k] = {}
                   for num_candidates in num_candidates_values:
                       if num_candidates < top_k:
                           continue

                       # Baseline search: full precision using ENN (Float32)
                       baseline_result = custom_vector_search(
                           user_query=query,
                           collection=collection,
                           embedding_path="embedding",
                           vector_search_index_name=quantized_index_name,
                           top_k=top_k,
                           num_candidates=num_candidates,
                           use_full_precision=True,
                       )
                       baseline_ids = {
                           res["wiki_id"] for res in baseline_result["results"]
                       }

                       # Quantized search:
                       quantized_result = custom_vector_search(
                           user_query=query,
                           collection=collection,
                           embedding_path="embedding",
                           vector_search_index_name=quantized_index_name,
                           top_k=top_k,
                           num_candidates=num_candidates,
                           use_full_precision=False,
                       )
                       quantized_ids = {
                           res["wiki_id"] for res in quantized_result["results"]
                       }

                       # Compute

（本文超出站点单文档体积上限，已截断）