Tutorial: Queryable Encryption with Mongoose

Tutorial: Queryable Encryption with Mongoose

For the complete MongoDB documentation index, see www.mongodb.com/docs/llms.txt

Tutorial: Queryable Encryption with Mongoose

Overview

In this guide, you can learn how to create an application that uses Mongoose to implement MongoDB's Queryable Encryption feature.

Queryable Encryption allows you to automatically encrypt and decrypt document fields. You can use Queryable Encryption to encrypt sensitive data in your application, store data fields as randomized encrypted data on the server, and query the encrypted fields. This tutorial shows you how to set up Queryable Encryption by using Mongoose, which provides an Object Document Mapper (ODM) library for data interaction.

Prerequisites

Before you begin this tutorial, complete the following prerequisite tasks:

Create a MongoDB Atlas account and configure a cluster. Ensure that your cluster runs on MongoDB Server version 7.0 or later. To learn more, see the MongoDB Get Started guide.
Download the Automatic Encryption Shared Library. To view instructions, see the [Install and Configure a Query Analysis Component](https://www.mongodb.com/docs/manual/core/queryable-encryption/install-library/?encryption-component=crypt_shared#procedure guide. These instructions show how to navigate to the MongoDB Download Center and fill out the form required to download the library.
Install Node.js v16.20.1 or later.

Tutorial

This tutorial shows how to create a Queryable Encryption application that uses Mongoose. The application encrypts and decrypts patient medical records and queries encrypted medical data.

Tip: Complete Application

To view the complete sample application for this tutorial, see the mongoose-qe-app folder on GitHub.

Set Up Your Project

Follow the steps in this section to install the project dependencies, configure your environment, and create the application structure.

Set up your environment.

Run the following commands in your terminal to initialize your project and install the necessary dependencies:

bash mkdir mongoose-qe-app cd mongoose-qe-app npm init -y npm pkg set main="queryable-encryption-tutorial.js" npm pkg set type="module" npm pkg set scripts.start="node queryable-encryption-tutorial.js" npm i mongoose mongodb dotenv mongodb-client-encryption

These commands create a mongoose-qe-app project directory and install the following dependencies:

Mongoose, the Node.js ODM
Node.js driver
Dotenv, a module for loading environment variables
mongodb-client-encryption, the encryption package
Specify your environment variables.

In your project root, create an .env file and paste the following code:

```bash # MongoDB Connection URI and Shared Library Path

MONGODB_URI="" SHARED_LIB_PATH=""

# AWS Credentials

AWS_ACCESS_KEY_ID="" AWS_SECRET_ACCESS_KEY="" AWS_KEY_REGION="" AWS_KEY_ARN=""

# Azure Credentials

AZURE_TENANT_ID="" AZURE_CLIENT_ID="" AZURE_CLIENT_SECRET="" AZURE_KEY_NAME="" AZURE_KEY_VERSION="" AZURE_KEY_VAULT_ENDPOINT=""

# GCP Credentials

GCP_EMAIL="" GCP_PRIVATE_KEY=""

GCP_PROJECT_ID="" GCP_LOCATION="" GCP_KEY_RING="" GCP_KEY_NAME="" GCP_KEY_VERSION=""

# KMIP Credentials

KMIP_KMS_ENDPOINT="" KMIP_TLS_CA_FILE="" KMIP_TLS_CERT_FILE="" ```

Replace the <connection URI> placeholder with the connection URI that connects to your cluster, and replace the <Automatic Encryption Shared Library path> with the full path to your Automatic Encryption Shared Library. Ensure the path points to lib/mongo_crypt_v1.dylib inside your downloaded package.

If you use one of the Key Management Systems (KMS) listed in the .env template, replace the corresponding placeholder values. Otherwise, you can leave these values unassigned and store your Customer Master Key locally. This application creates the local Customer Master Key file for you.

Warning: Local CMK Storage

If you store your Customer Master Key locally, do not use this application in production. Without a remote KMS, you risk unauthorized access to the encryption key or loss of the key needed to decrypt your data.

Tip: Key Management Systems

To learn more about KMS, see the Key Management Wikipedia entry.

Create your application files.

Navigate to your mongoose-qe-app directory and create a file named queryable-encryption-tutorial.js, which will store your application logic. Paste the following code into this file:

```javascript import 'dotenv/config'; import mongoose from 'mongoose'; import * as qeHelper from './queryable-encryption-helpers.js'; import { MongoClient, ClientEncryption } from 'mongodb';

async function runExample() {

 // Paste initial application variables below

 // Paste credential and options variables below

 // Paste connection and client configuration below

 // Paste data key creation code below

 // Paste encryption schema below

 // Paste the model below

 // Paste connection code below

 // Paste the insertion operation below

 // Paste the encrypted query below

 await connection.close();
 console.log('Connection closed.');

}

runExample().catch(console.dir);

```

Then, create a file named queryable-encryption-helpers.js and paste the following code:

```javascript import 'dotenv/config'; import { writeFileSync, readFileSync, existsSync } from 'fs'; import { randomBytes } from 'crypto';

export async function dropExistingDatabase(client, databaseName) { const database = client.db(databaseName); await database.dropDatabase(); }

// Paste helper methods below ```

This file contains a dropExistingDatabase() helper function for your application. Future steps in this tutorial instruct you to add additional helper functions.

Assign your application variables.

Specify the initial database and encryption variables by pasting the following code in the runExample() function of your queryable-encryption-tutorial.js file:

```javascript const kmsProviderName = '';

const uri = process.env.MONGODB_URI; // Your connection URI

const keyVaultDatabaseName = 'encryption'; const keyVaultCollectionName = '__keyVault'; const keyVaultNamespace = ${keyVaultDatabaseName}.${keyVaultCollectionName}; const encryptedDatabaseName = 'medicalRecords'; const encryptedCollectionName = 'patients'; ```

Replace the '<KMS provider>' placeholder with your key provider: 'aws', 'azure', 'gcp', or 'kmip'. To store your Customer Master Key locally, set this value to 'local'.

The code pre-populates the following variables:

keyVaultDatabaseName - The database in MongoDB that stores your data encryption keys (DEKs). This tutorial uses the encryption database.
keyVaultCollectionName - The collection in MongoDB that stores your DEKs. The code sets this variable to __keyVault, prefixed with an underscore to distinguish it from a user collection.
keyVaultNamespace - The namespace in MongoDB that stores your DEKs. This variable consists of your keyVaultDatabaseName and keyVaultCollectionName variables, separated by a period.
encryptedDatabaseName - The database in MongoDB that stores your encrypted data. This tutorial uses the medicalRecords database.
encryptedCollectionName - The collection in MongoDB that stores your encrypted data. This tutorial uses the patients collection.

Configure Encryption Credentials

After setting up your project structure, follow the steps in this section to configure your KMS provider credentials and encryption options.

Retrieve your KMS provider credentials.

Add the getKMSProviderCredentials() function to your queryable-encryption-helpers.js file. This function retrieves the credentials for your Key Management System provider, or it creates a local Customer Master Key file if you do not use a KMS provider.

Paste the following code after the dropExistingDatabase() function:

``javascript export function getKMSProviderCredentials(kmsProviderName) { let kmsProviders; switch (kmsProviderName) { case 'aws': kmsProviders = { aws: { accessKeyId: process.env.AWS_ACCESS_KEY_ID, // Your AWS access key ID secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY, // Your AWS secret access key }, }; return kmsProviders; case 'azure': kmsProviders = { azure: { tenantId: process.env.AZURE_TENANT_ID, // Your Azure tenant ID clientId: process.env.AZURE_CLIENT_ID, // Your Azure client ID clientSecret: process.env.AZURE_CLIENT_SECRET, // Your Azure client secret }, }; return kmsProviders; case 'gcp': kmsProviders = { gcp: { email: process.env.GCP_EMAIL, // Your GCP email privateKey: process.env.GCP_PRIVATE_KEY, // Your GCP private key }, }; return kmsProviders; case 'kmip': kmsProviders = { kmip: { endpoint: process.env.KMIP_KMS_ENDPOINT, // Your KMIP KMS endpoint }, }; return kmsProviders; case 'local': (function () { if (!existsSync('./customer-master-key.txt')) { try { writeFileSync('customer-master-key.txt', randomBytes(96)); } catch (err) { throw new Error(Unable to write Customer Master Key to file due to the following error: ${err}` ); } } })(); try { // WARNING: Do not use a local key file in a production application const localMasterKey = readFileSync('./customer-master-key.txt');

       if (localMasterKey.length !== 96) {
         throw new Error(
           'Expected the customer master key file to be 96 bytes.'
         );
       }

       kmsProviders = {
         local: {
           key: localMasterKey,
         },
       };
     } catch (err) {
       throw new Error(
         `Unable to read the Customer Master Key due to the following error: ${err}`
       );
     }
     return kmsProviders;

   default:
     throw new Error(
       `Unrecognized value for KMS provider name \'${kmsProviderName}\' encountered while retrieving KMS credentials.`
     );
 }

} ```

This getKMSProviderCredentials() function supports multiple KMS providers, including AWS, Azure, GCP, KMIP, and local key storage.

Retrieve your Customer Master Key credentials.

Add the getCustomerMasterKeyCredentials() function to your queryable-encryption-helpers.js file. This function retrieves Customer Master Key credentials based on your KMS provider.

Paste the following code after the getKMSProviderCredentials() function:

javascript export function getCustomerMasterKeyCredentials(kmsProviderName) { let customerMasterKeyCredentials; switch (kmsProviderName) { case 'aws': customerMasterKeyCredentials = { key: process.env.AWS_KEY_ARN, // Your AWS Key ARN region: process.env.AWS_KEY_REGION, // Your AWS Key Region }; return customerMasterKeyCredentials; case 'azure': customerMasterKeyCredentials = { keyVaultEndpoint: process.env.AZURE_KEY_VAULT_ENDPOINT, // Your Azure Key Vault Endpoint keyName: process.env.AZURE_KEY_NAME, // Your Azure Key Name }; return customerMasterKeyCredentials; case 'gcp': customerMasterKeyCredentials = { projectId: process.env.GCP_PROJECT_ID, // Your GCP Project ID location: process.env.GCP_LOCATION, // Your GCP Key Location keyRing: process.env.GCP_KEY_RING, // Your GCP Key Ring keyName: process.env.GCP_KEY_NAME, // Your GCP Key Name }; return customerMasterKeyCredentials; case 'kmip': case 'local': customerMasterKeyCredentials = {}; return customerMasterKeyCredentials; default: throw new Error( `Unrecognized value for KMS provider name \'${kmsProviderName}\' encountered while retrieving Customer Master Key credentials.` ); } }

This function configures the corresponding Customer Master Key credentials for your chosen KMS provider.

Retrieve your automatic encryption options.

Add the getAutoEncryptionOptions() function to your queryable-encryption-helpers.js file. This function configures the automatic encryption options for your application.

Paste the following code after the getCustomerMasterKeyCredentials() function:

```javascript export async function getAutoEncryptionOptions( kmsProviderName, keyVaultNamespace, kmsProviders ) { if (kmsProviderName === 'kmip') { const tlsOptions = { kmip: { tlsCAFile: process.env.KMIP_TLS_CA_FILE, // Path to your TLS CA file tlsCertificateKeyFile: process.env.KMIP_TLS_CERT_FILE, // Path to your TLS certificate key file }, }; const extraOptions = { cryptSharedLibPath: process.env.SHARED_LIB_PATH, // Path to your Automatic Encryption Shared Library };

   const autoEncryptionOptions = {
     keyVaultNamespace,
     kmsProviders,
     extraOptions,
     tlsOptions,
   };
   return autoEncryptionOptions;
 } else {
   const extraOptions = {
     cryptSharedLibPath: process.env.SHARED_LIB_PATH, // Path to your Automatic Encryption Shared Library
   };

   const autoEncryptionOptions = {
     keyVaultNamespace,
     kmsProviders,
     extraOptions,
   };
   return autoEncryptionOptions;
 }

} ```

Assign your credentials variables.

Now that you've defined helper functions, you can use them to access your credentials from the main application file.

Navigate to your queryable-encryption-tutorial.js file and paste the following code after the // Paste credential and options variables below code comment:

```javascript const kmsProviderCredentials = qeHelper.getKMSProviderCredentials(kmsProviderName); const customerMasterKeyCredentials = qeHelper.getCustomerMasterKeyCredentials(kmsProviderName);

const autoEncryptionOptions = await qeHelper.getAutoEncryptionOptions( kmsProviderName, keyVaultNamespace, kmsProviderCredentials ); ```

This code calls your helper functions to retrieve the KMS provider credentials, Customer Master Key credentials, and automatic encryption options used to configure Queryable Encryption.

Encrypt and Connect to MongoDB

After configuring encryption settings, follow the steps in this section to set up your MongoDB connection, create data keys, and define your encrypted schema.

Configure your MongoDB connection and client.

To set up your MongoDB connection and client, navigate to your queryable-encryption-tutorial.js file and paste the following code after the // Paste connection and client configuration below code comment:

```javascript const connection = mongoose.createConnection();

const client = new MongoClient(uri); const clientEncryption = new ClientEncryption( client, autoEncryptionOptions );

await qeHelper.dropExistingDatabase(client, encryptedDatabaseName); await qeHelper.dropExistingDatabase(client, keyVaultDatabaseName); ```

This code creates a Mongoose connection for encrypted operations, a MongoDB client for key management, and a ClientEncryption instance for creating data keys. It also drops any existing databases to ensure a clean setup for the tutorial.

Create your data keys.

To create the necessary data keys, paste the following code after the // Paste data key creation code below code comment:

javascript const keyId1 = await clientEncryption.createDataKey( kmsProviderName, { masterKey: customerMasterKeyCredentials, }); const keyId2 = await clientEncryption.createDataKey( kmsProviderName, { masterKey: customerMasterKeyCredentials, }); const keyId3 = await clientEncryption.createDataKey( kmsProviderName, { masterKey: customerMasterKeyCredentials, });

This code creates three data encryption keys used encrypt different fields in your patient documents. Each encrypted field requires its own data key.

Define your encryption schema.

To create the encrypted schema definition, paste the following code after the // Paste encryption schema below code comment:

javascript const patientSchema = new mongoose.Schema({ patientName: { type: String, required: true }, patientId: { type: Number, required: true }, patientRecord: { ssn: { type: String, encrypt: { keyId: keyId1, queries: { queryType: 'equality' } } }, billing: { type: { type: String, encrypt: { keyId: keyId2, } }, number: { type: String, encrypt: { keyId: keyId3 } } }, billAmount: Number } }, { encryptionType: 'queryableEncryption', collection: encryptedCollectionName });

This schema defines the structure for documents in the patients collection and specifies the following encrypted fields:

patientRecord.ssn: Encrypted and configured for equality queries
patientRecord.billing.type: Encrypted, but not queryable
patientRecord.billing.number: Encrypted, but not queryable
Register the encryption schema on a model.

Create a new model named Patient to represent the patients collection and register your encryption schema on the model. To create the model, paste the following code after the // Paste the model below code comment:

javascript const Patient = connection.model('Patient', patientSchema);

This code creates a Mongoose model that handles automatic encryption and decryption of fields when you perform database operations.

Establish your database connection.

After registering your model, you can establish the database connection by pasting the following code after the // Paste connection code below code comment:

javascript await connection.openUri(uri, { autoEncryption: autoEncryptionOptions, dbName: encryptedDatabaseName });

This establishes the connection to your medicalRecords database, on which automatic encryption is enabled.

Perform Encrypted Operations

After configuring your application and database connection, follow the steps in this section to insert and query encrypted documents.

Insert encrypted data.

To insert a document that has encrypted fields, navigate to your queryable-encryption-tutorial.js file and paste the following code after the // Paste the insertion operation below code comment:

```javascript const patientDocument = { patientName: 'Jon Doe', patientId: 12345678, patientRecord: { ssn: '987-65-4320', billing: { type: 'Visa', number: '4111111111111111', }, billAmount: 1500, }, };

const result = await Patient.create(patientDocument);

if (result) { console.log('Successfully inserted the patient document.'); console.log('Document ID:', result._id); } ```

Queryable Encryption automatically encrypts the patientRecord.ssn and patientRecord.billing fields before storing the document in MongoDB.

Query encrypted data.

To query an encrypted field, paste the following code after the // Paste the encrypted query below code comment:

javascript const findResult = await Patient.findOne({ 'patientRecord.ssn': '987-65-4320', }); console.log('Found patient:'); console.log(findResult);

This code performs an equality query on the encrypted patientRecord.ssn field.

Run your application.

Finally, you can run the encrypted operations defined in the previous steps by running the following command from your mongoose-qe-app directory:

bash npm start

If successful, your command output resembles the following example:

javascript Successfully inserted the patient document. Document ID: new ObjectId('...') Found patient: { patientRecord: { billing: { type: 'Visa', number: '4111111111111111' }, ssn: '987-65-4320', billAmount: 1500 }, _id: new ObjectId('...'), patientName: 'Jon Doe', patientId: 12345678, __v: 0, __safeContent__: [ Binary.createFromBase64('EGzhQwBpf1B6W9udskSxJ8kwEEnF5P+SJPZ6ygQ9Ft8=', 0) ] } Connection closed.

Next Steps

Congratulations on completing the Mongoose Queryable Encryption tutorial! You now have a sample Mongoose application that uses Queryable Encryption to automatically encrypt and decrypt document fields. Your application encrypts sensitive data on the server side and queries the data on the client side.

To learn more about Queryable Encryption and Mongoose, visit the following resources:

View more driver Queryable Encryption tutorials in the MongoDB Server manual.
Learn how to create an application that uses Mongoose without Queryable Encryption in the [Mongoose Get Started](https://www.mongodb.com/docs/drivers/node/current/integrations/mongoose/mongoose-get-started.md#std-label-node-mongoose-get-started tutorial.
Learn more about Mongoose in the Mongoose documentation.

Include and Link the Driver in Your Program

Tutorial: Queryable Encryption with Mongoose