token-type
- token-type
- How to Index Token Fields
- token Type Limitations
- Review the Behavior of the token Type
- Define the Index for the token Type
- Configure token Field Properties
- Try an Example for the token Type
- token Type Limitations
- Review the Behavior of the token Type
- Define the Index for the token Type
- Configure token Field Properties
- Try an Example for the token Type
- token Type Limitations
- Review the Behavior of the token Type
- Define the Index for the token Type
- Configure token Field Properties
- Try an Example for the token Type
- token Type Limitations
- Review the Behavior of the token Type
- Define the Index for the token Type
- Configure token Field Properties
- Try an Example for the token Type
- token Type Limitations
- Review the Behavior of the token Type
- Define the Index for the token Type
- Configure token Field Properties
- Try an Example for the token Type
- token Type Limitations
- Review the Behavior of the token Type
- Define the Index for the token Type
For the complete MongoDB documentation index, see www.mongodb.com/docs/llms.txt
<!-- Tab options on this page. Append to the .md URL to filter: ?tabs=<id,...> select specific tabs (e.g. ?tabs=nodejs,shell) ?allTabs=true include every tab (no param) default: one tab per tabset
Available tabs: other tabs: token-type-only, multiple-types, vib, jsonib, cloud, local, maven bom, gradle bom, maven, gradle -->
How to Index Token Fields
Use the MongoDB Search token field type to index string values for sorting, faceting, and querying. After you index a field as token, you can sort and group results by the indexed field, and query it using the [equals](https://www.mongodb.com/docs/search/query/operators-collectors/equals.md#std-label-equals-ref, [in](https://www.mongodb.com/docs/search/query/operators-collectors/in.md#std-label-in-ref, and [range](https://www.mongodb.com/docs/search/query/operators-collectors/range.md#std-label-range-ref operators.
You must use the MongoDB Search token type to index [string](https://www.mongodb.com/docs/manual/reference/bson-types.md#std-label-document-bson-type-string fields to enable sorting and faceting on these fields. Then, you can perform the following actions:
-
Use the
sortoption in your query to sort the results by the indexed field. To learn more, see [Sort MongoDB Search Results.](https://www.mongodb.com/docs/search/query/sort.md#std-label-sort-ref -
Use the
facetcollector in your query to group the results by the indexed field. To learn more, see [How to Use Facets with MongoDB Search.](https://www.mongodb.com/docs/search/tutorial/facet-tutorial.md#std-label-facet-tutorial
Additionally, you must index string fields as the token type in order to use the [equals](https://www.mongodb.com/docs/search/query/operators-collectors/equals.md#std-label-equals-ref, [in](https://www.mongodb.com/docs/search/query/operators-collectors/in.md#std-label-in-ref and [range](https://www.mongodb.com/docs/search/query/operators-collectors/range.md#std-label-range-ref operators. To learn more, see the documentation for each respective [operator.](https://www.mongodb.com/docs/search/query/operators-collectors/overview.md#std-label-operators-ref
Note:
You can also use the token type to index string fields for pre-filtering your data for [$vectorSearch](https://www.mongodb.com/docs/vector-search/query/aggregation-stages/vector-search-stage.md#mongodb-pipeline-pipe.-vectorSearch queries. To learn more, see [MongoDB Vector Search Overview.](https://www.mongodb.com/docs/vector-search.md#std-label-fts-vector-search
token Type Limitations
When you index a field as the token type, you must index that field as string type also to query the text value using operators such as [text](https://www.mongodb.com/docs/search/query/operators-collectors/text.md#std-label-text-ref, [phrase](https://www.mongodb.com/docs/search/query/operators-collectors/phrase.md#std-label-phrase-ref, etc. For the following operators, you don't need to index the field as string type also to query the text value in the field:
-
[equals](https://www.mongodb.com/docs/search/query/operators-collectors/equals.md#std-label-equals-ref
-
[in](https://www.mongodb.com/docs/search/query/operators-collectors/in.md#std-label-in-ref
-
[range](https://www.mongodb.com/docs/search/query/operators-collectors/range.md#std-label-range-ref
Note:
We do not recommend indexing a field with both the stringFacet (deprecated) and token types, unless you require a different normalizer for the token type for a use case other than faceting. With this index definition, MongoDB Search uses the stringFacet definition for faceting.
Review the Behavior of the token Type
When you index a field as token type, MongoDB Search indexes the terms in the string as a single token (searchable term) and stores them in a columnar storage for efficient filtering or sort operations. You can use a [normalizer](https://www.mongodb.com/docs/search/indexes/analyzers/overview.md#std-label-ref-built-in-normalizers to transform the token. By default, the normalizer is set to none and so MongoDB Search indexes strings in their original form.
The major difference between the MongoDB Search string and token types is that MongoDB Search creates one or more tokens for fields indexed as string type whereas MongoDB Search creates only a single token for fields indexed as the token type.
If a string being indexed as a token field type exceeds 8181 characters, MongoDB Search truncates it to 8181 characters before indexing.
Define the Index for the token Type
- Retrieve your Service Account access token for the project.
Use your service account to generate an access token, which authenticates your requests to the Atlas Administration API. To learn how to generate an access token, see [Make an API (Application Programming Interface) Request.](https://www.mongodb.com/docs/atlas/configure-api-access.md#std-label-example-api-request
- Make an API (Application Programming Interface) call.
Replace the following values in the curl command below. This command sends a POST request to your cluster to create a MongoDB Search index with the token field type.
-
Replace {
ACCESS-TOKEN} with the output from the preceding step -
Replace {
groupId} with the project ID of the project where you want to create the MongoDB Search index -
Replace {
clusterName} with the name of the cluster where you want to create the MongoDB Search index
shell
curl --header "Authorization: Bearer ${ACCESS_TOKEN}" \
--header "Accept: application/vnd.atlas.2025-03-12+json" \
--header "Content-Type: application/json" \
-X POST "https://cloud.mongodb.com/api/atlas/v2/groups/{groupId}/clusters/{clusterName}/search/indexes" \
-d '{ "collectionName": "movies",
"database": "sample_mflix",
"name": "default",
"type": "search",
"definition": {
"mappings": {
"dynamic": false,
"fields": {
"genres": [
{
"type": "string"
},
{
"type": "token"
}
]
}
}
}
}'
Configure token Field Properties
The MongoDB Search token type takes the following parameters:
| Option | Type | Necessity | Description | Default |
|---|---|---|---|---|
type |
string | Required | Human-readable label that identifies this field type. Value must be token. |
|
normalizer |
string | Optional | Type of transformation to perform on the field value. Value can be one of the following: lowercase - to transform text values in string fields to lowercase.; none - to not perform any transformation. If you don't set this option explicitly, it defaults to none. |
none |
Try an Example for the token Type
The following index definition examples use collections in the [sample data](https://www.mongodb.com/docs/manual/sample-data.md#std-label-available-sample-datasets. After you [load the sample data](https://www.mongodb.com/docs/atlas/import/load-sample-data.md#std-label-load-sample-data, you can use these examples to index fields as the token type.
To learn how to set up and run the following example code, see [Define the Index for the token Type](https://www.mongodb.com/docs/search/indexes/field-types/token-type.md#std-label-define-index-token-admin-api.
Index as token Type Only Example
The following index definition indexes string values in the title field as MongoDB Search token type and converts the field value to lowercase, which allows you to do the following:
-
Perform case-insensitive sort, as specified by the
normalizer, on thetitlefield. -
Run exact match queries on the
titlefield using the following operators: -
[equals](https://www.mongodb.com/docs/search/query/operators-collectors/equals.md#std-label-equals-ref
-
[in](https://www.mongodb.com/docs/search/query/operators-collectors/in.md#std-label-in-ref
-
[range](https://www.mongodb.com/docs/search/query/operators-collectors/range.md#std-label-range-ref
curl --header "Authorization: Bearer ${ACCESS_TOKEN}" \
--header "Accept: application/vnd.atlas.2025-03-12+json" \
--header "Content-Type: application/json" \
-X POST "https://cloud.mongodb.com/api/atlas/v2/groups/{groupId}/clusters/{clusterName}/search/indexes" \
-d '{ "collectionName": "movies",
"database": "sample_mflix",
"name": "default",
"type": "search",
"definition": {
"mappings": {
"dynamic": false,
"fields": {
"title": {
"type": "token",
"normalizer": "lowercase"
}
}
}
}
}'
Output:
{"collectionName":"comments","database":"sample_mflix","indexID":"687a8d52a3b4982b8c228383","latestDefinition":{"mappings":{"dynamic":true,"fields":{"title":{"normalizer":"lowercase","type":"token"}}}},"latestDefinitionVersion":{"version":0},"name":"default_15","queryable":false,"status":"PENDING","type":"search"}
You must use the MongoDB Search token type to index [string](https://www.mongodb.com/docs/manual/reference/bson-types.md#std-label-document-bson-type-string fields to enable sorting and faceting on these fields. Then, you can perform the following actions:
-
Use the
sortoption in your query to sort the results by the indexed field. To learn more, see [Sort MongoDB Search Results.](https://www.mongodb.com/docs/search/query/sort.md#std-label-sort-ref -
Use the
facetcollector in your query to group the results by the indexed field. To learn more, see [How to Use Facets with MongoDB Search.](https://www.mongodb.com/docs/search/tutorial/facet-tutorial.md#std-label-facet-tutorial
Additionally, you must index string fields as the token type in order to use the [equals](https://www.mongodb.com/docs/search/query/operators-collectors/equals.md#std-label-equals-ref, [in](https://www.mongodb.com/docs/search/query/operators-collectors/in.md#std-label-in-ref and [range](https://www.mongodb.com/docs/search/query/operators-collectors/range.md#std-label-range-ref operators. To learn more, see the documentation for each respective [operator.](https://www.mongodb.com/docs/search/query/operators-collectors/overview.md#std-label-operators-ref
Note:
You can also use the token type to index string fields for pre-filtering your data for [$vectorSearch](https://www.mongodb.com/docs/vector-search/query/aggregation-stages/vector-search-stage.md#mongodb-pipeline-pipe.-vectorSearch queries. To learn more, see [MongoDB Vector Search Overview.](https://www.mongodb.com/docs/vector-search.md#std-label-fts-vector-search
token Type Limitations
When you index a field as the token type, you must index that field as string type also to query the text value using operators such as [text](https://www.mongodb.com/docs/search/query/operators-collectors/text.md#std-label-text-ref, [phrase](https://www.mongodb.com/docs/search/query/operators-collectors/phrase.md#std-label-phrase-ref, etc. For the following operators, you don't need to index the field as string type also to query the text value in the field:
-
[equals](https://www.mongodb.com/docs/search/query/operators-collectors/equals.md#std-label-equals-ref
-
[in](https://www.mongodb.com/docs/search/query/operators-collectors/in.md#std-label-in-ref
-
[range](https://www.mongodb.com/docs/search/query/operators-collectors/range.md#std-label-range-ref
Note:
We do not recommend indexing a field with both the stringFacet (deprecated) and token types, unless you require a different normalizer for the token type for a use case other than faceting. With this index definition, MongoDB Search uses the stringFacet definition for faceting.
Review the Behavior of the token Type
When you index a field as token type, MongoDB Search indexes the terms in the string as a single token (searchable term) and stores them in a columnar storage for efficient filtering or sort operations. You can use a [normalizer](https://www.mongodb.com/docs/search/indexes/analyzers/overview.md#std-label-ref-built-in-normalizers to transform the token. By default, the normalizer is set to none and so MongoDB Search indexes strings in their original form.
The major difference between the MongoDB Search string and token types is that MongoDB Search creates one or more tokens for fields indexed as string type whereas MongoDB Search creates only a single token for fields indexed as the token type.
If a string being indexed as a token field type exceeds 8181 characters, MongoDB Search truncates it to 8181 characters before indexing.
Define the Index for the token Type
- Create a
search-index.jsonconfiguration file and specify thetokentype.
json
{
"name": "default",
"collectionName": "<collection>",
"database": "<database>",
"definition": {
"mappings": {
"dynamic": true|false,
"fields": {
"<field-name>": {
"type": "token",
"normalizer": "lowercase|none"
}
}
}
}
}
- Connect to the Atlas CLI
In your terminal, connect to your cluster from the Atlas CLI. To learn more, see Connect from the Atlas CLI.
- Create a MongoDB Search index.
Run the atlas clusters search indexes create command in your terminal, replacing <clusterName> with the name of the cluster, and <path-to-file> with the path to the search-index.json file:
shell
atlas clusters search indexes create --clusterName <cluster-name> --file <path-to-file>
Configure token Field Properties
The MongoDB Search token type takes the following parameters:
| Option | Type | Necessity | Description | Default |
|---|---|---|---|---|
type |
string | Required | Human-readable label that identifies this field type. Value must be token. |
|
normalizer |
string | Optional | Type of transformation to perform on the field value. Value can be one of the following: lowercase - to transform text values in string fields to lowercase.; none - to not perform any transformation. If you don't set this option explicitly, it defaults to none. |
none |
Try an Example for the token Type
The following index definition examples use collections in the [sample data](https://www.mongodb.com/docs/manual/sample-data.md#std-label-available-sample-datasets. After you [load the sample data](https://www.mongodb.com/docs/atlas/import/load-sample-data.md#std-label-load-sample-data, you can use these examples to index fields as the token type.
To learn how to set up and run the following example code, see [Define the Index for the token Type](https://www.mongodb.com/docs/search/indexes/field-types/token-type.md#std-label-define-index-token-admin-api.
Index as token Type Only Example
The following index definition indexes string values in the title field as MongoDB Search token type and converts the field value to lowercase, which allows you to do the following:
-
Perform case-insensitive sort, as specified by the
normalizer, on thetitlefield. -
Run exact match queries on the
titlefield using the following operators: -
[equals](https://www.mongodb.com/docs/search/query/operators-collectors/equals.md#std-label-equals-ref
-
[in](https://www.mongodb.com/docs/search/query/operators-collectors/in.md#std-label-in-ref
-
[range](https://www.mongodb.com/docs/search/query/operators-collectors/range.md#std-label-range-ref
{
"name": "default",
"collectionName": "movies",
"database": "sample_mflix",
"definition": {
"mappings": {
"dynamic": false,
"fields": {
"title": {
"type": "token",
"normalizer": "lowercase"
}
}
}
}
}
You must use the MongoDB Search token type to index [string](https://www.mongodb.com/docs/manual/reference/bson-types.md#std-label-document-bson-type-string fields to enable sorting and faceting on these fields. Then, you can perform the following actions:
-
Use the
sortoption in your query to sort the results by the indexed field. To learn more, see [Sort MongoDB Search Results.](https://www.mongodb.com/docs/search/query/sort.md#std-label-sort-ref -
Use the
facetcollector in your query to group the results by the indexed field. To learn more, see [How to Use Facets with MongoDB Search.](https://www.mongodb.com/docs/search/tutorial/facet-tutorial.md#std-label-facet-tutorial
Additionally, you must index string fields as the token type in order to use the [equals](https://www.mongodb.com/docs/search/query/operators-collectors/equals.md#std-label-equals-ref, [in](https://www.mongodb.com/docs/search/query/operators-collectors/in.md#std-label-in-ref and [range](https://www.mongodb.com/docs/search/query/operators-collectors/range.md#std-label-range-ref operators. To learn more, see the documentation for each respective [operator.](https://www.mongodb.com/docs/search/query/operators-collectors/overview.md#std-label-operators-ref
Note:
You can also use the token type to index string fields for pre-filtering your data for [$vectorSearch](https://www.mongodb.com/docs/vector-search/query/aggregation-stages/vector-search-stage.md#mongodb-pipeline-pipe.-vectorSearch queries. To learn more, see [MongoDB Vector Search Overview.](https://www.mongodb.com/docs/vector-search.md#std-label-fts-vector-search
token Type Limitations
When you index a field as the token type, you must index that field as string type also to query the text value using operators such as [text](https://www.mongodb.com/docs/search/query/operators-collectors/text.md#std-label-text-ref, [phrase](https://www.mongodb.com/docs/search/query/operators-collectors/phrase.md#std-label-phrase-ref, etc. For the following operators, you don't need to index the field as string type also to query the text value in the field:
-
[equals](https://www.mongodb.com/docs/search/query/operators-collectors/equals.md#std-label-equals-ref
-
[in](https://www.mongodb.com/docs/search/query/operators-collectors/in.md#std-label-in-ref
-
[range](https://www.mongodb.com/docs/search/query/operators-collectors/range.md#std-label-range-ref
Note:
We do not recommend indexing a field with both the stringFacet (deprecated) and token types, unless you require a different normalizer for the token type for a use case other than faceting. With this index definition, MongoDB Search uses the stringFacet definition for faceting.
Review the Behavior of the token Type
When you index a field as token type, MongoDB Search indexes the terms in the string as a single token (searchable term) and stores them in a columnar storage for efficient filtering or sort operations. You can use a [normalizer](https://www.mongodb.com/docs/search/indexes/analyzers/overview.md#std-label-ref-built-in-normalizers to transform the token. By default, the normalizer is set to none and so MongoDB Search indexes strings in their original form.
The major difference between the MongoDB Search string and token types is that MongoDB Search creates one or more tokens for fields indexed as string type whereas MongoDB Search creates only a single token for fields indexed as the token type.
If a string being indexed as a token field type exceeds 8181 characters, MongoDB Search truncates it to 8181 characters before indexing.
Define the Index for the token Type
- Connect to your cluster using MongoDB Compass.
Open Compass and connect to your cluster. For detailed instructions, see [Connect to a Cluster via Compass.](https://www.mongodb.com/docs/atlas/compass-connection.md#std-label-atlas-connect-via-compass
- Specify the database and collection.
On the Database screen, click the name of the database, then click the name of the collection.
- Create the MongoDB Search index.
Click the Indexes tab, then select Search Indexes.
Click Create Atlas Search Index to open the index creation dialog box.
Specify a name for the index.
Your MongoDB Search index is named default by default. If you keep this name, then your index will be the default Search index for any MongoDB Search query that does not specify a different index option in its [operators](https://www.mongodb.com/docs/search/query/operators-collectors/overview.md#std-label-fts-operators. If you are creating multiple indexes, we recommend that you maintain a consistent, descriptive naming convention across your indexes.
Specify the JSON (Javascript Object Notation) MongoDB Search index definition.
json
{
"mappings": {
"dynamic": true|false,
"fields": {
"<field-name>": {
"type": "token",
"normalizer": "lowercase|none"
}
}
}
}
Click Create Search Index.
Configure token Field Properties
The MongoDB Search token type takes the following parameters:
| Option | Type | Necessity | Description | Default |
|---|---|---|---|---|
type |
string | Required | Human-readable label that identifies this field type. Value must be token. |
|
normalizer |
string | Optional | Type of transformation to perform on the field value. Value can be one of the following: lowercase - to transform text values in string fields to lowercase.; none - to not perform any transformation. If you don't set this option explicitly, it defaults to none. |
none |
Try an Example for the token Type
The following index definition examples use collections in the [sample data](https://www.mongodb.com/docs/manual/sample-data.md#std-label-available-sample-datasets. After you [load the sample data](https://www.mongodb.com/docs/atlas/import/load-sample-data.md#std-label-load-sample-data, you can use these examples to index fields as the token type.
To learn how to set up and run the following example code, see [Define the Index for the token Type](https://www.mongodb.com/docs/search/indexes/field-types/token-type.md#std-label-define-index-token-admin-api.
Index as token Type Only Example
The following index definition indexes string values in the title field as MongoDB Search token type and converts the field value to lowercase, which allows you to do the following:
-
Perform case-insensitive sort, as specified by the
normalizer, on thetitlefield. -
Run exact match queries on the
titlefield using the following operators: -
[equals](https://www.mongodb.com/docs/search/query/operators-collectors/equals.md#std-label-equals-ref
-
[in](https://www.mongodb.com/docs/search/query/operators-collectors/in.md#std-label-in-ref
-
[range](https://www.mongodb.com/docs/search/query/operators-collectors/range.md#std-label-range-ref
{
"mappings": {
"dynamic": false,
"fields": {
"title": {
"type": "token",
"normalizer": "lowercase"
}
}
}
}
You must use the MongoDB Search token type to index [string](https://www.mongodb.com/docs/manual/reference/bson-types.md#std-label-document-bson-type-string fields to enable sorting and faceting on these fields. Then, you can perform the following actions:
-
Use the
sortoption in your query to sort the results by the indexed field. To learn more, see [Sort MongoDB Search Results.](https://www.mongodb.com/docs/search/query/sort.md#std-label-sort-ref -
Use the
facetcollector in your query to group the results by the indexed field. To learn more, see [How to Use Facets with MongoDB Search.](https://www.mongodb.com/docs/search/tutorial/facet-tutorial.md#std-label-facet-tutorial
Additionally, you must index string fields as the token type in order to use the [equals](https://www.mongodb.com/docs/search/query/operators-collectors/equals.md#std-label-equals-ref, [in](https://www.mongodb.com/docs/search/query/operators-collectors/in.md#std-label-in-ref and [range](https://www.mongodb.com/docs/search/query/operators-collectors/range.md#std-label-range-ref operators. To learn more, see the documentation for each respective [operator.](https://www.mongodb.com/docs/search/query/operators-collectors/overview.md#std-label-operators-ref
Note:
You can also use the token type to index string fields for pre-filtering your data for [$vectorSearch](https://www.mongodb.com/docs/vector-search/query/aggregation-stages/vector-search-stage.md#mongodb-pipeline-pipe.-vectorSearch queries. To learn more, see [MongoDB Vector Search Overview.](https://www.mongodb.com/docs/vector-search.md#std-label-fts-vector-search
token Type Limitations
When you index a field as the token type, you must index that field as string type also to query the text value using operators such as [text](https://www.mongodb.com/docs/search/query/operators-collectors/text.md#std-label-text-ref, [phrase](https://www.mongodb.com/docs/search/query/operators-collectors/phrase.md#std-label-phrase-ref, etc. For the following operators, you don't need to index the field as string type also to query the text value in the field:
-
[equals](https://www.mongodb.com/docs/search/query/operators-collectors/equals.md#std-label-equals-ref
-
[in](https://www.mongodb.com/docs/search/query/operators-collectors/in.md#std-label-in-ref
-
[range](https://www.mongodb.com/docs/search/query/operators-collectors/range.md#std-label-range-ref
Note:
We do not recommend indexing a field with both the stringFacet (deprecated) and token types, unless you require a different normalizer for the token type for a use case other than faceting. With this index definition, MongoDB Search uses the stringFacet definition for faceting.
Review the Behavior of the token Type
When you index a field as token type, MongoDB Search indexes the terms in the string as a single token (searchable term) and stores them in a columnar storage for efficient filtering or sort operations. You can use a [normalizer](https://www.mongodb.com/docs/search/indexes/analyzers/overview.md#std-label-ref-built-in-normalizers to transform the token. By default, the normalizer is set to none and so MongoDB Search indexes strings in their original form.
The major difference between the MongoDB Search string and token types is that MongoDB Search creates one or more tokens for fields indexed as string type whereas MongoDB Search creates only a single token for fields indexed as the token type.
If a string being indexed as a token field type exceeds 8181 characters, MongoDB Search truncates it to 8181 characters before indexing.
Define the Index for the token Type
- Connect to the deployment by using
mongosh.
In your terminal, connect to your Atlas cloud-hosted deployment or local deployment from [mongosh](https://www.mongodb.com/docs/mongodb-shell.md#mongodb-binary-bin.mongosh. For detailed instructions on how to connect, see Connect to a Deployment.
- Switch to the database that contains the collection for which you want to create the index.
shell
use <database>
Output:
shell
switched to db <database>
- Run the
db.collection.createSearchIndex()method to create the index.
shell
db.<collection>.createSearchIndex(
"default",
{
"mappings": {
"dynamic": true|false,
"fields": {
"<field-name>": {
"type": "token",
"normalizer": "lowercase|none"
}
}
}
}
)
Output:
text
default
Configure token Field Properties
The MongoDB Search token type takes the following parameters:
| Option | Type | Necessity | Description | Default |
|---|---|---|---|---|
type |
string | Required | Human-readable label that identifies this field type. Value must be token. |
|
normalizer |
string | Optional | Type of transformation to perform on the field value. Value can be one of the following: lowercase - to transform text values in string fields to lowercase.; none - to not perform any transformation. If you don't set this option explicitly, it defaults to none. |
none |
Try an Example for the token Type
The following index definition examples use collections in the [sample data](https://www.mongodb.com/docs/manual/sample-data.md#std-label-available-sample-datasets. After you [load the sample data](https://www.mongodb.com/docs/atlas/import/load-sample-data.md#std-label-load-sample-data, you can use these examples to index fields as the token type.
To learn how to set up and run the following example code, see [Define the Index for the token Type](https://www.mongodb.com/docs/search/indexes/field-types/token-type.md#std-label-define-index-token-admin-api.
Index as token Type Only Example
The following index definition indexes string values in the title field as MongoDB Search token type and converts the field value to lowercase, which allows you to do the following:
-
Perform case-insensitive sort, as specified by the
normalizer, on thetitlefield. -
Run exact match queries on the
titlefield using the following operators: -
[equals](https://www.mongodb.com/docs/search/query/operators-collectors/equals.md#std-label-equals-ref
-
[in](https://www.mongodb.com/docs/search/query/operators-collectors/in.md#std-label-in-ref
-
[range](https://www.mongodb.com/docs/search/query/operators-collectors/range.md#std-label-range-ref
db.movies.createSearchIndex(
"default",
{
"mappings": {
"dynamic": false,
"fields": {
"title": {
"type": "token",
"normalizer": "lowercase"
}
}
}
}
)
You must use the MongoDB Search token type to index [string](https://www.mongodb.com/docs/manual/reference/bson-types.md#std-label-document-bson-type-string fields to enable sorting and faceting on these fields. Then, you can perform the following actions:
-
Use the
sortoption in your query to sort the results by the indexed field. To learn more, see [Sort MongoDB Search Results.](https://www.mongodb.com/docs/search/query/sort.md#std-label-sort-ref -
Use the
facetcollector in your query to group the results by the indexed field. To learn more, see [How to Use Facets with MongoDB Search.](https://www.mongodb.com/docs/search/tutorial/facet-tutorial.md#std-label-facet-tutorial
Additionally, you must index string fields as the token type in order to use the [equals](https://www.mongodb.com/docs/search/query/operators-collectors/equals.md#std-label-equals-ref, [in](https://www.mongodb.com/docs/search/query/operators-collectors/in.md#std-label-in-ref and [range](https://www.mongodb.com/docs/search/query/operators-collectors/range.md#std-label-range-ref operators. To learn more, see the documentation for each respective [operator.](https://www.mongodb.com/docs/search/query/operators-collectors/overview.md#std-label-operators-ref
Note:
You can also use the token type to index string fields for pre-filtering your data for [$vectorSearch](https://www.mongodb.com/docs/vector-search/query/aggregation-stages/vector-search-stage.md#mongodb-pipeline-pipe.-vectorSearch queries. To learn more, see [MongoDB Vector Search Overview.](https://www.mongodb.com/docs/vector-search.md#std-label-fts-vector-search
token Type Limitations
When you index a field as the token type, you must index that field as string type also to query the text value using operators such as [text](https://www.mongodb.com/docs/search/query/operators-collectors/text.md#std-label-text-ref, [phrase](https://www.mongodb.com/docs/search/query/operators-collectors/phrase.md#std-label-phrase-ref, etc. For the following operators, you don't need to index the field as string type also to query the text value in the field:
-
[equals](https://www.mongodb.com/docs/search/query/operators-collectors/equals.md#std-label-equals-ref
-
[in](https://www.mongodb.com/docs/search/query/operators-collectors/in.md#std-label-in-ref
-
[range](https://www.mongodb.com/docs/search/query/operators-collectors/range.md#std-label-range-ref
Note:
We do not recommend indexing a field with both the stringFacet (deprecated) and token types, unless you require a different normalizer for the token type for a use case other than faceting. With this index definition, MongoDB Search uses the stringFacet definition for faceting.
Review the Behavior of the token Type
When you index a field as token type, MongoDB Search indexes the terms in the string as a single token (searchable term) and stores them in a columnar storage for efficient filtering or sort operations. You can use a [normalizer](https://www.mongodb.com/docs/search/indexes/analyzers/overview.md#std-label-ref-built-in-normalizers to transform the token. By default, the normalizer is set to none and so MongoDB Search indexes strings in their original form.
The major difference between the MongoDB Search string and token types is that MongoDB Search creates one or more tokens for fields indexed as string type whereas MongoDB Search creates only a single token for fields indexed as the token type.
If a string being indexed as a token field type exceeds 8181 characters, MongoDB Search truncates it to 8181 characters before indexing.
Define the Index for the token Type
Choose your preferred configuration method in the Atlas UI and then select the database and collection.
Visual Editor
-
Click Refine Your Index to configure your index.
-
In the Field Mappings section, click Add Field Mapping to open the Add Field Mapping window.
-
Click Customized Configuration.
-
Select the field to index from the Field Name dropdown.
Note:
You can't index fields that contain the dollar ($) sign at the start of the field name.
-
Click the Data Type dropdown and select Token.
-
Configure the field properties for the
tokentype. To learn more, see [Field Properties](https://www.mongodb.com/docs/search/indexes/field-types/uuid-type.md#std-label-fts-field-types-uuid-options. -
Click Add.
Configure token Field Properties
The MongoDB Search token type takes the following parameters:
| Option | Type | Necessity | Description | Default |
|---|---|---|---|---|
type |
string | Required | Human-readable label that identifies this field type. Value must be token. |
|
normalizer |
string | Optional | Type of transformation to perform on the field value. Value can be one of the following: lowercase - to transform text values in string fields to lowercase.; none - to not perform any transformation. If you don't set this option explicitly, it defaults to none. |
none |
Try an Example for the token Type
The following index definition examples use collections in the [sample data](https://www.mongodb.com/docs/manual/sample-data.md#std-label-available-sample-datasets. After you [load the sample data](https://www.mongodb.com/docs/atlas/import/load-sample-data.md#std-label-load-sample-data, you can use these examples to index fields as the token type.
To learn how to set up and run the following example code, see [Define the Index for the token Type](https://www.mongodb.com/docs/search/indexes/field-types/token-type.md#std-label-define-index-token-admin-api.
Index as token Type Only Example
The following index definition indexes string values in the title field as MongoDB Search token type and converts the field value to lowercase, which allows you to do the following:
-
Perform case-insensitive sort, as specified by the
normalizer, on thetitlefield. -
Run exact match queries on the
titlefield using the following operators: -
[equals](https://www.mongodb.com/docs/search/query/operators-collectors/equals.md#std-label-equals-ref
-
[in](https://www.mongodb.com/docs/search/query/operators-collectors/in.md#std-label-in-ref
-
[range](https://www.mongodb.com/docs/search/query/operators-collectors/range.md#std-label-range-ref
{
"mappings": {
"dynamic": false,
"fields": {
"title": {
"type": "token",
"normalizer": "lowercase"
}
}
}
}
You must use the MongoDB Search token type to index [string](https://www.mongodb.com/docs/manual/reference/bson-types.md#std-label-document-bson-type-string fields to enable sorting and faceting on these fields. Then, you can perform the following actions:
-
Use the
sortoption in your query to sort the results by the indexed field. To learn more, see [Sort MongoDB Search Results.](https://www.mongodb.com/docs/search/query/sort.md#std-label-sort-ref -
Use the
facetcollector in your query to group the results by the indexed field. To learn more, see [How to Use Facets with MongoDB Search.](https://www.mongodb.com/docs/search/tutorial/facet-tutorial.md#std-label-facet-tutorial
Additionally, you must index string fields as the token type in order to use the [equals](https://www.mongodb.com/docs/search/query/operators-collectors/equals.md#std-label-equals-ref, [in](https://www.mongodb.com/docs/search/query/operators-collectors/in.md#std-label-in-ref and [range](https://www.mongodb.com/docs/search/query/operators-collectors/range.md#std-label-range-ref operators. To learn more, see the documentation for each respective [operator.](https://www.mongodb.com/docs/search/query/operators-collectors/overview.md#std-label-operators-ref
Note:
You can also use the token type to index string fields for pre-filtering your data for [$vectorSearch](https://www.mongodb.com/docs/vector-search/query/aggregation-stages/vector-search-stage.md#mongodb-pipeline-pipe.-vectorSearch queries. To learn more, see [MongoDB Vector Search Overview.](https://www.mongodb.com/docs/vector-search.md#std-label-fts-vector-search
token Type Limitations
When you index a field as the token type, you must index that field as string type also to query the text value using operators such as [text](https://www.mongodb.com/docs/search/query/operators-collectors/text.md#std-label-text-ref, [phrase](https://www.mongodb.com/docs/search/query/operators-collectors/phrase.md#std-label-phrase-ref, etc. For the following operators, you don't need to index the field as string type also to query the text value in the field:
-
[equals](https://www.mongodb.com/docs/search/query/operators-collectors/equals.md#std-label-equals-ref
-
[in](https://www.mongodb.com/docs/search/query/operators-collectors/in.md#std-label-in-ref
-
[range](https://www.mongodb.com/docs/search/query/operators-collectors/range.md#std-label-range-ref
Note:
We do not recommend indexing a field with both the stringFacet (deprecated) and token types, unless you require a different normalizer for the token type for a use case other than faceting. With this index definition, MongoDB Search uses the stringFacet definition for faceting.
Review the Behavior of the token Type
When you index a field as token type, MongoDB Search indexes the terms in the string as a single token (searchable term) and stores them in a columnar storage for efficient filtering or sort operations. You can use a [normalizer](https://www.mongodb.com/docs/search/indexes/analyzers/overview.md#std-label-ref-built-in-normalizers to transform the token. By default, the normalizer is set to none and so MongoDB Search indexes strings in their original form.
The major difference between the MongoDB Search string and token types is that MongoDB Search creates one or more tokens for fields indexed as string type whereas MongoDB Search creates only a single token for fields indexed as the token type.
If a string being indexed as a token field type exceeds 8181 characters, MongoDB Search truncates it to 8181 characters before indexing.
Define the Index for the token Type
- Set up the C project.
In your terminal, navigate to where you want to create your application, then run the following command to create a directory called atlas-search-project for this project:
shell
# Create a new directory for the project
mkdir atlas-search-project && cd atlas-search-project
Add the C driver to your project by following the instructions in the MongoDB C Driver documentation.
- Define the index.
Create a create_index.c file in your project directory, and copy and paste the following code into the file.
```c
#include
int main (void) { mongoc_client_t client = NULL; mongoc_collection_t collection = NULL; mongoc_database_t *database = NULL; bson_error_t error; bson_t cmd = BSON_INITIALIZER; bool ok = true;
mongoc_init();
// Connect to your Atlas deployment
client = mongoc_client_new("<connectionString>");
if (!client) {
fprintf(stderr, "Failed to create a MongoDB client.\n");
ok = false;
goto cleanup;
}
// Access your database and collection
database = mongoc_client_get_database(client, "<databaseName>");
collection = mongoc_database_get_collection(database, "<collectionName>");
// Specify the command and the new index
const char *cmd_str = BSON_STR({
"createSearchIndexes" : "<collectionName>",
"indexes" : [ {
"name" : "default",
"definition" : {
"mappings": {
"dynamic": true|false,
"fields": {
"<fieldName>" : {
"type": "token",
"normalizer": "lowercase|none"
}
}
},
}
} ]
});
// Convert your command to BSON
if (!bson_init_from_json(&cmd, cmd_str, -1, &error)) {
fprintf(stderr, "Failed to parse command: %s\n", error.message);
ok = false;
goto cleanup;
}
// Create the MongoDB Search index by running the command
if (!mongoc_collection_command_simple (collection, &cmd, NULL, NULL, &error)) {
fprintf(stderr, "Failed to run createSearchIndexes: %s\n", error.message);
ok = false;
goto cleanup;
}
printf ("Index created!\n");
cleanup: mongoc_collection_destroy (collection); mongoc_client_destroy (client); mongoc_database_destroy (database); bson_destroy (&cmd); mongoc_cleanup (); return ok ? EXIT_SUCCESS : EXIT_FAILURE; }
```
Replace <connection-string> with the connection string for your Atlas cluster or local Atlas deployment.
To learn more about finding your connection string, see [Connect to a Cluster via Client Libraries.](https://www.mongodb.com/docs/atlas/driver-connection.md#std-label-connect-via-driver
Note:
For a local or self-managed deployment, your connection string should use the following format:
text
mongodb://localhost:<port-number>/?directConnection=true
To learn more, see [Connection Strings.](https://www.mongodb.com/docs/manual/reference/connection-string.md#std-label-mongodb-uri
- Set up a CMake application
To configure your application, create a CMakeLists.txt file in your project directory. Then, add the following code to the file:
```txt
cmake_minimum_required(VERSION 3.11)
project(atlas-search-index LANGUAGES C)
add_executable (index.out create_index.c)
find_package(mongoc
```
The preceding code performs the following actions:
-
Configures a C project.
-
Creates a
index.outexecutable for your application. -
Finds and requires the C driver. Replace the
<version>placeholder with your C driver version token, such as2.0.0. -
Links the program to the
libmongoclibrary.
Note:
In the sample CMakeLists.txt file, the `mongoc::mon
(本文超出站点单文档体积上限,已截断)
