token-filters

For the complete MongoDB documentation index, see www.mongodb.com/docs/llms.txt

<!-- Tab options on this page. Append to the .md URL to filter: ?tabs=<id,...> select specific tabs (e.g. ?tabs=nodejs,shell) ?allTabs=true include every tab (no param) default: one tab per tabset

Available tabs: other tabs: visual-editor, json-editor, vib, jsoneditor -->

Token Filters for Custom Analyzers

A token filter performs operations such as the following:

• Stemming, which reduces related words, such as “talking”, “talked”, and “talks” to their root word “talk”.

• Redaction, the removal of sensitive information from public documents.

Token Filters require a type field, and some take additional options as well.

"tokenFilters": [
  {
    "type": "<token-filter-type>",
    "<additional-option>": <value>
  }
]

The sample index definitions and queries on this page use the [sample collection](https://www.mongodb.com/docs/search/indexes/analyzers/custom.md#std-label-custom-analyzers-eg-coll named minutes. To try these examples, load the minutes collection on your cluster and navigate to the Create a Search Index page in the Atlas UI following the steps in the [Create a MongoDB Search Index](https://www.mongodb.com/docs/search/tutorial.md#std-label-create-fts-index tutorial. Then, select the minutes collection as your data source, and follow the example procedure to create an index from the Atlas UI or using [mongosh.](https://www.mongodb.com/docs/mongodb-shell.md#mongodb-binary-bin.mongosh

MongoDB Search supports the following types of token filters:

asciiFolding

The asciiFolding token filter converts alphabetic, numeric, and symbolic unicode characters that are not in the Basic Latin Unicode block to their ASCII (American Standard Code for Information Interchange) equivalents, if available.

Attributes

It has the following attributes:

Example

The following index definition indexes the page_updated_by.first_name field in the [minutes](https://www.mongodb.com/docs/search/indexes/analyzers/custom.md#std-label-custom-analyzers-eg-coll collection using a custom analyzer named asciiConverter. The custom analyzer specifies the following:

Apply the [standard tokenizer](https://www.mongodb.com/docs/search/indexes/analyzers/tokenizers.md#std-label-standard-tokenizer-ref to create tokens based on word break rules.

Apply the asciiFolding token filter to convert the field values to their ASCII (American Standard Code for Information Interchange) equivalent.

Visual Editor

1. In the Custom Analyzers section, click Add Custom Analyzer.

2. Select the Create Your Own radio button and click Next.

3. Type asciiConverter in the Analyzer Name field.

4. Expand Tokenizer if it's collapsed.

5. Select standard from the dropdown.

6. Expand Token Filters and click Add token filter.

7. Select asciiFolding from the dropdown.

8. Click Add token filter to add the token filter to your custom analyzer.

9. Click Add to create the custom analyzer.

10. In the Field Mappings section, click Add Field Mapping to apply the custom analyzer on the page_updated_by.first_name field.

11. Select page_updated_by.first_name from the Field Name dropdown and String from the Data Type dropdown.

12. In the properties section for the data type, select asciiConverter from the Index Analyzer and Search Analyzer dropdowns.

13. Click Add, then Save Changes.

db.minutes.createSearchIndex(
  "default",
  {
    "mappings": {
      "dynamic": false,
      "fields": {
        "page_updated_by": {
          "type": "document",
          "dynamic": false,
          "fields": {
            "first_name": {
              "type": "string",
              "analyzer": "asciiConverter"
            }
          }
        }
      }
    },
    "analyzers": [
      {
        "name": "asciiConverter",
        "tokenizer": {
          "type": "standard"
        },
        "tokenFilters": [
          {
            "type": "asciiFolding"
          }
        ]
      }
    ]
  }
)

The following query searches the first_name field in the [minutes](https://www.mongodb.com/docs/search/indexes/analyzers/custom.md#std-label-custom-analyzers-eg-coll collection for names using their ASCII equivalent.

1. Click the Query button for your index.

2. Click Edit Query to edit the query.

3. Click on the query bar and select the database and collection.

4. Replace the default query with the following and click Search:

{
  "$search": {
    "index": "default",
    "text": {
      "query": "Sian",
      "path": "page_updated_by.first_name"
    }
  }
}

Output:

SCORE: 0.5472603440284729 _id:  "1"
  message: "try to siGn-In"
  page_updated_by: Object
    last_name: "AUERBACH"
    first_name: "Siân"
    email: "auerbach@example.com"
    phone: "(123)-456-7890"
  text: Object
    en_US: "<head> This page deals with department meetings.</head>"
    sv_FI: "Cette page traite des réunions de département"

db.minutes.aggregate([
  {
    "$search": {
      "index": "default",
      "text": {
        "query": "Sian",
        "path": "page_updated_by.first_name"
      }
    }
  },
  {
    "$project": {
      "_id": 1,
      "page_updated_by.last_name": 1,
      "page_updated_by.first_name": 1
    }
  }
])

Output:

[
 {
    _id: 1,
    page_updated_by: { last_name: 'AUERBACH', first_name: 'Siân'}
 }
]

MongoDB Search returns document with _id: 1 in the results because MongoDB Search created the following tokens (searchable terms) for the page_updated_by.first_name field in the document, which it then used to match to the query term Sian:

daitchMokotoffSoundex

The daitchMokotoffSoundex token filter creates tokens for words that sound the same based on the Daitch-Mokotoff Soundex phonetic algorithm. This filter can generate multiple encodings for each input, where each encoded token is a 6 digit number.

Note:

Don't use the daitchMokotoffSoundex token filter in:

• [Synonym](https://www.mongodb.com/docs/search/indexes/synonyms.md#std-label-synonyms-ref or [autocomplete](https://www.mongodb.com/docs/search/indexes/field-types/autocomplete-type.md#std-label-bson-data-types-autocomplete type mapping definitions.

• Operators where fuzzy is enabled. MongoDB Search supports the fuzzy option for the following operators:

• [autocomplete](https://www.mongodb.com/docs/search/query/operators-collectors/autocomplete.md#std-label-autocomplete-ref

• [text](https://www.mongodb.com/docs/search/query/operators-collectors/text.md#std-label-text-ref

Attributes

It has the following attributes:

Example

The following index definition indexes the page_updated_by.last_name field in the [minutes](https://www.mongodb.com/docs/search/indexes/analyzers/custom.md#std-label-custom-analyzers-eg-coll collection using a custom analyzer named dmsAnalyzer. The custom analyzer specifies the following:

Apply the [standard tokenizer](https://www.mongodb.com/docs/search/indexes/analyzers/tokenizers.md#std-label-standard-tokenizer-ref to create tokens based on word break rules.

Apply the daitchMokotoffSoundex token filter to encode the tokens for words that sound the same.

Visual Editor

1. In the Custom Analyzers section, click Add Custom Analyzer.

2. Select the Create Your Own radio button and click Next.

3. Type dmsAnalyzer in the Analyzer Name field.

4. Expand Tokenizer if it's collapsed.

5. Select standard from the dropdown.

6. Expand Token Filters and click Add token filter.

7. Select daitchMokotoffSoundex from the dropdown and select the value shown in the following table for the originalTokens field:

1. Click Add token filter to add the token filter to your custom analyzer.

2. Click Add to create the custom analyzer.

3. In the Field Mappings section, click Add Field Mapping to apply the custom analyzer on the page_updated_by.last_name field.

4. Select page_updated_by.last_name from the Field Name dropdown and String from the Data Type dropdown.

5. In the properties section for the data type, select dmsAnalyzer from the Index Analyzer and Search Analyzer dropdowns.

6. Click Add, then Save Changes.

db.minutes.createSearchIndex(
  "default",
  {
    "mappings": {
      "dynamic": false,
      "fields": {
        "page_updated_by": {
          "type": "document",
          "dynamic": false,
          "fields": {
            "last_name": {
              "type": "string",
              "analyzer": "dmsAnalyzer"
            }
          }
        }
      }
    },
    "analyzers": [
      {
        "name": "dmsAnalyzer",
        "tokenizer": {
          "type": "standard"
        },
        "tokenFilters": [
          {
            "type": "daitchMokotoffSoundex",
            "originalTokens": "include"
          }
        ]
      }
    ]
  }
)

The following query searches for terms that sound similar to AUERBACH in the page_updated_by.last_name field of the [minutes](https://www.mongodb.com/docs/search/indexes/analyzers/custom.md#std-label-custom-analyzers-eg-coll collection.

1. Click the Query button for your index.

2. Click Edit Query to edit the query.

3. Click on the query bar and select the database and collection.

4. Replace the default query with the following and click Search:

{
  "$search": {
    "index": "default",
    "text": {
      "query": "AUERBACH",
      "path": "page_updated_by.last_name"
    }
  }
}

Output:

SCORE: 0.568153440952301  _id:  "1"
  message: "try to siGn-In"
  page_updated_by: Object
    last_name: "AUERBACH"
    first_name: "Siân"
    email: "auerbach@example.com"
    phone: "(123)-456-7890"
  text: Object
    en_US: "<head> This page deals with department meetings.</head>"
    sv_FI: "Den här sidan behandlar avdelningsmöten"
    fr_CA: "Cette page traite des réunions de département"

SCORE: 0.521163284778595  _id:  "2"
  message: "do not forget to SIGN-IN. See ① for details."
  page_updated_by: Object
    last_name: "OHRBACH"
    first_name: "Noël"
    email: "ohrbach@example.com"
    phone: "(123) 456 0987"
  text: Object
    en_US: "The head of the sales department spoke first."
    fa_IR: "ابتدا رئیس بخش فروش صحبت کرد"
    sv_FI: "Först talade chefen för försäljningsavdelningen"

db.minutes.aggregate([
  {
    "$search": {
     "index": "default",
     "text": {
        "query": "AUERBACH",
        "path": "page_updated_by.last_name"
      }
    }
  },
  {
    "$project": {
      "_id": 1,
      "page_updated_by.last_name": 1
    }
  }
])

Output:

[
 { "_id" : 1, "page_updated_by" : { "last_name" : "AUERBACH" } }
 { "_id" : 2, "page_updated_by" : { "last_name" : "OHRBACH" } }
]

MongoDB Search returns documents with _id: 1 and _id: 2 because the terms in both documents are phonetically similar, and are coded using the same six digit numbers (097400 and 097500). The following table shows the tokens (searchable terms and six digit encodings) that MongoDB Search creates for the documents in the results:

doubleMetaphone

The doubleMetaphone token filter creates tokens for words that sound the same based on the Double Metaphone phonetic algorithm. Unlike the [daitchMokotoffSoundex](https://www.mongodb.com/docs/search/indexes/analyzers/token-filters.md#std-label-daitchmokotoffsoundex-tf-ref token filter, which generates 6 digit numeric codes, this filter generates an alphabetic code for each input to account for variations in pronunciation.

Attributes

The doubleMetaphone token filter has the following attributes:

Example

The following index definition indexes the page_updated_by.last_name field in the [minutes](https://www.mongodb.com/docs/search/indexes/analyzers/custom.md#std-label-custom-analyzers-eg-coll collection using a custom analyzer named doubleMetaphoneAnalyzer. The custom analyzer specifies the following:

Apply the [standard tokenizer](https://www.mongodb.com/docs/search/indexes/analyzers/tokenizers.md#std-label-standard-tokenizer-ref to create tokens based on word break rules.

Apply the doubleMetaphone token filter to encode the tokens for words that sound the same.

Visual Editor

The doubleMetaphone token filter is not supported in the Atlas UI Visual Editor.

db.minutes.createSearchIndex(
  "default",
  {
    "mappings": {
      "dynamic": false,
      "fields": {
        "page_updated_by": {
          "type": "document",
          "dynamic": false,
          "fields": {
            "last_name": {
              "type": "string",
              "analyzer": "doubleMetaphoneAnalyzer"
            }
          }
        }
      }
    },
    "analyzers": [
      {
        "name": "doubleMetaphoneAnalyzer",
        "tokenizer": {
          "type": "standard"
        },
        "tokenFilters": [
          {
            "type": "doubleMetaphone",
            "originalTokens": "include"
          }
        ]
      }
    ]
  }
)

The following query searches for terms that sound similar to AUERBACH in the page_updated_by.last_name field of the [minutes](https://www.mongodb.com/docs/search/indexes/analyzers/custom.md#std-label-custom-analyzers-eg-coll collection.

1. Click the Query button for your index.

2. Click Edit Query to edit the query.

3. Click on the query bar and select the database and collection.

4. Replace the default query with the following and click Search:

{
  "$search": {
    "index": "default",
    "text": {
      "query": "AUERBACH",
      "path": "page_updated_by.last_name"
    }
  }
}

Output:

SCORE: 0.5041070580482483  _id:  "1"
  message: "try to siGn-In"
  page_updated_by: Object
    last_name: "AUERBACH"
    first_name: "Siân"
    email: "auerbach@example.com"
    phone: "(123)-456-7890"
  text: Object
    en_US: "<head> This page deals with department meetings.</head>"
    sv_FI: "Den här sidan behandlar avdelningsmöten"
    fr_CA: "Cette page traite des réunions de département"
  title: "The team's weekly meeting"

SCORE: 0.39608412981033325  _id:  "2"
  message: "do not forget to SIGN-IN. See ① for details."
  page_updated_by: Object
    last_name: "OHRBACH"
    first_name: "Noël"
    email: "ohrbach@example.com"
    phone: "(123) 456 0987"
  text: Object
    en_US: "The head of the sales department spoke first."
    fa_IR: "ابتدا رئیس بخش فروش صحبت کرد"
    sv_FI: "Först talade chefen för försäljningsavdelningen"
  title: "The check-in with sales team"

db.minutes.aggregate([
  {
    "$search": {
     "index": "default",
     "text": {
        "query": "AUERBACH",
        "path": "page_updated_by.last_name"
      }
    }
  },
  {
    "$project": {
      "_id": 1,
      "page_updated_by.last_name": 1
    }
  }
])

Output:

[
 { "_id" : 1, "page_updated_by" : { "last_name" : "AUERBACH" } }
 { "_id" : 2, "page_updated_by" : { "last_name" : "OHRBACH" } }
]

MongoDB Search returns documents with _id: 1 and _id: 2 because the terms in both documents are phonetically similar, and are encoded using the same phonetic code (ARPK). The following table shows the tokens (searchable terms and phonetic code) that MongoDB Search creates for the documents in the results:

edgeGram

The edgeGram token filter tokenizes input from the left side, or “edge”, of a text input into n-grams of configured sizes.

Note:

Typically, token filters operate similarly to a pipeline, with each input token yielding no more than one output token that is then inputted into the subsequent token. The edgeGram token filter, by contrast, is a graph-producing filter that yields multiple output tokens from a single input token.

Because synonym.

For querying with [regex (MongoDB Search Operator)](https://www.mongodb.com/docs/search/query/operators-collectors/regex.md#std-label-regex-ref or [wildcard Operator](https://www.mongodb.com/docs/search/query/operators-collectors/wildcard.md#std-label-wildcard-ref, you can't use wordDelimiterGraph token filter as the searchAnalyzer as it produces more than one output token per input token. Specify a different analyzer as the searchAnalyzer in your index definition.

Attributes

It has the following attributes:

Example

The following index definition indexes the title field in the [minutes](https://www.mongodb.com/docs/search/indexes/analyzers/custom.md#std-label-custom-analyzers-eg-coll collection using a custom analyzer named titleAutocomplete. The custom analyzer specifies the following:

Apply the [standard tokenizer](https://www.mongodb.com/docs/search/indexes/analyzers/tokenizers.md#std-label-standard-tokenizer-ref to create tokens based on word break rules.

Apply the following filters on the tokens:

• icuFolding token filter to apply character foldings to the tokens.

• edgeGram token filter to create 4 to 7 character long tokens from the left side.

Visual Editor

1. In the Custom Analyzers section, click Add Custom Analyzer.

2. Select the Create Your Own radio button and click Next.

3. Type titleAutocomplete in the Analyzer Name field.

4. Expand Tokenizer if it's collapsed.

5. Select standard from the dropdown.

6. Expand Token Filters and click Add token filter.

7. Select icuFolding from the dropdown and click Add token filter to add the token filter to your custom analyzer.

8. Click Add token filter to add another token filter.

9. Select edgeGram from the dropdown and type the value shown in the following table for the fields:

1. Click Add token filter to add the token filter to your custom analyzer.

2. Click Add to create the custom analyzer.

3. In the Field Mappings section, click Add Field Mapping to apply the custom analyzer on the title field.

4. Select title from the Field Name dropdown and String from the Data Type dropdown.

5. In the properties section for the data type, select titleAutocomplete from the Index Analyzer and Search Analyzer dropdowns.

6. Click Add, then Save Changes.

db.minutes.createSearchIndex(
  "default",
  {
    "analyzer": "titleAutocomplete",
    "mappings": {
      "dynamic": false,
      "fields": {
        "title": {
          "type": "string",
          "analyzer": "titleAutocomplete"
        }
      }
    },
    "analyzers": [
      {
        "name": "titleAutocomplete",
        "charFilters": [],
        "tokenizer": {
          "type": "standard"
        },
        "tokenFilters": [
          {
            "type": "icuFolding"
          },
          {
            "type": "edgeGram",
            "minGram": 4,
            "maxGram": 7
          }
        ]
      }
    ]
  }
)

The following query searches the title field of the [minutes](https://www.mongodb.com/docs/search/indexes/analyzers/custom.md#std-label-custom-analyzers-eg-coll collection for terms that begin with mee, followed by any number of other characters.

1. Click the Query button for your index.

2. Click Edit Query to edit the query.

3. Click on the query bar and select the database and collection.

4. Replace the default query with the following and click Search:

{
  "$search": {
    "wildcard": {
      "query": "mee*",
      "path": "title",
      "allowAnalyzedField": true
    }
  }
}

Output:

SCORE: 1  _id:  "1"
  message: "try to siGn-In"
  page_updated_by: Object
    last_name: "AUERBACH"
    first_name: "Siân"
    email: "auerbach@example.com"
    phone: "(123)-456-7890"
  text: Object
    en_US: "<head> This page deals with department meetings.</head>"
    sv_FI: "Den här sidan behandlar avdelningsmöten"
    fr_CA: "Cette page traite des réunions de département"

SCORE: 1  _id:  "3"
  message: "try to sign-in"
  page_updated_by: Object
    last_name: "LEWINSKY"
    first_name: "Brièle"
    email: "lewinsky@example.com"
    phone: "(123).456.9870"
  text: Object
    en_US: "<body>We'll head out to the conference room by noon.</body>"

db.minutes.aggregate([
  {
    "$search": {
      "wildcard": {
        "query": "mee*",
        "path": "title",
        "allowAnalyzedField": true
      }
    }
  },
  {
    "$project": {
      "_id": 1,
      "title": 1
    }
  }
])

Output:

[
  { _id: 1, title: 'The team's weekly meeting' },
  { _id: 3, title: 'The regular board meeting' }
]

MongoDB Search returns documents with _id: 1 and _id: 3 because the documents contain the term meeting, which matches the query criteria. Specifically, MongoDB Search creates the following 4 to 7 character tokens (searchable terms) for the documents in the results, which it then matches to the query term mee*:

englishPossessive

The englishPossessive token filter removes possessives (trailing 's) from words.

Attributes

It has the following attributes:

Example

The following index definition indexes the title field in the [minutes](https://www.mongodb.com/docs/search/indexes/analyzers/custom.md#std-label-custom-analyzers-eg-coll collection using a custom analyzer named englishPossessiveStemmer. The custom analyzer specifies the following:

Apply the [standard](https://www.mongodb.com/docs/search/indexes/analyzers/tokenizers.md#std-label-standard-tokenizer-ref tokenizer to create tokens (search terms) based on word break rules.

Apply the [englishPossessive](https://www.mongodb.com/docs/search/indexes/analyzers/token-filters.md#std-label-englishPossessive-tf-ref token filter to remove possessives (trailing 's) from the tokens.

Visual Editor

1. In the Custom Analyzers section, click Add Custom Analyzer.

2. Select the Create Your Own radio button and click Next.

3. Type englishPossessiveStemmer in the Analyzer Name field.

4. Expand Tokenizer if it's collapsed.

5. Select standard from the dropdown.

6. Expand Token Filters and click Add token filter.

7. Select englishPossessive from the dropdown.

8. Click Add token filter to add the token filter to your custom analyzer.

9. Click Add to create the custom analyzer.

10. In the Field Mappings section, click Add Field Mapping to apply the custom analyzer on the title field.

11. Select title from the Field Name dropdown and String from the Data Type dropdown.

12. In the properties section for the data type, select englishPossessiveStemmer from the Index Analyzer and Search Analyzer dropdowns.

13. Click Add, then Save Changes.

db.minutes.createSearchIndex(
  "default",
  {
    "mappings": {
      "fields": {
        "title": {
          "type": "string",
          "analyzer": "englishPossessiveStemmer"
        }
      }
    },
    "analyzers": [
      {
        "name": "englishPossessiveStemmer",
        "charFilters": [],
        "tokenizer": {
          "type": "standard"
        },
        "tokenFilters": [
          {
            "type": "englishPossessive"
          }
        ]
      }
    ]
  }
)

The following query searches the title field in the [minutes](https://www.mongodb.com/docs/search/indexes/analyzers/custom.md#std-label-custom-analyzers-eg-coll collection for the term team.

1. Click the Query button for your index.

2. Click Edit Query to edit the query.

3. Click on the query bar and select the database and collection.

4. Replace the default query with the following and click Search:

{
  "$search": {
    "index": "default",
    "text": {
      "query": "team",
      "path": "title"
    }
  }
}

Output:

SCORE: 0.34314215183258057  _id:  "1"
  message: "try to siGn-In"
  page_updated_by: Object
  text: Object

SCORE: 0.29123833775520325  _id:  "2"
  message: "do not forget to SIGN-IN. See ① for details."
  page_updated_by: Object
  text: Object

db.minutes.aggregate([
  {
    "$search": {
      "index": "default",
      "text": {
        "query": "team",
        "path": "title"
      }
    }
  },
  {
    "$project": {
      "_id": 1,
      "title": 1
    }
  }
])

Output:

[
  {
    _id: 1,
    title: 'The team's weekly meeting'
  },
  {
    _id: 2,
    title: 'The check-in with sales team'
  }
]

MongoDB Search returns results that contain the term team in the title field. MongoDB Search returns the document with _id: 1 because MongoDB Search transforms team's in the title field to the token team during analysis. Specifically, MongoDB Search creates the following tokens (searchable terms) for the documents in the results, which it then matches to the query term:

englishMinimalStemming

The englishMinimalStemming token filter stems plural English words to their singular form. Unlike more aggressive stemming filters such as [kStemming](https://www.mongodb.com/docs/search/indexes/analyzers/token-filters.md#std-label-kStemming-tf-ref, [porterStemming](https://www.mongodb.com/docs/search/indexes/analyzers/token-filters.md#std-label-porterStemming-tf-ref, or [snowballStemming](https://www.mongodb.com/docs/search/indexes/analyzers/token-filters.md#std-label-snowballstemming-tf-ref, this filter only removes plural suffixes and doesn't stem other morphological variations. This preserves non-plural words in their original form, which helps when you need exact matches for non-plural terms while still matching singular and plural forms.

Attributes

It has the following attributes:

Example

The following index definition indexes the text.en_US field in the [minutes](https://www.mongodb.com/docs/search/indexes/analyzers/custom.md#std-label-custom-analyzers-eg-coll collection using a custom analyzer named minimalStemmer. The custom analyzer specifies the following:

Apply the [standard](https://www.mongodb.com/docs/search/indexes/analyzers/tokenizers.md#std-label-standard-tokenizer-ref tokenizer to create tokens based on word break rules.

Apply the following token filters on the tokens:

• [lowercase](https://www.mongodb.com/docs/search/indexes/analyzers/token-filters.md#std-label-lowercase-tf-ref token filter to convert the tokens to lowercase.

• [englishMinimalStemming](https://www.mongodb.com/docs/search/indexes/analyzers/token-filters.md#std-label-englishMinimalStemming-tf-ref token filter to stem plural words to their singular form without stemming other word variations.

Visual Editor

1. In the Custom Analyzers section, click Add Custom Analyzer.

2. Select the Create Your Own radio button and click Next.

3. Type minimalStemmer in the Analyzer Name field.

4. Expand Tokenizer if it's collapsed.

5. Select standard from the dropdown.

6. Expand Token Filters and click Add token filter to add the following token filters from the dropdown.

7. lowercase

8. englishMinimalStemming

9. Click Add token filter to add the token filter to your custom analyzer.

10. Click Add to create the custom analyzer.

11. In the Field Mappings section, click Add Field Mapping to apply the custom analyzer on the text.en_US field.

12. Select text.en_US from the Field Name dropdown and String from the Data Type dropdown.

13. In the properties section for the data type, select minimalStemmer from the Index Analyzer and Search Analyzer dropdowns.

14. Click Add, then Save Changes.

db.minutes.createSearchIndex(
  "default",
  {
    "mappings": {
      "dynamic": true,
      "fields": {
        "text": {
          "fields": {
            "en_US": {
              "analyzer": "minimalStemmer",
              "searchAnalyzer": "minimalStemmer",
              "type": "string"
            }
          },
          "type": "document"
        }
      }
    },
    "analyzers": [
      {
        "name": "minimalStemmer",
        "tokenFilters": [
          {
            "type": "lowercase"
          },
          {
            "type": "englishMinimalStemming"
          }
        ],
        "tokenizer": {
          "type": "standard"
        }
      }
    ]
  }
)

The following query searches the text.en_US field in the [minutes](https://www.mongodb.com/docs/search/indexes/analyzers/custom.md#std-label-custom-analyzers-eg-coll collection for the term meeting.

1. Click the Query button for your index.

2. Click Edit Query to edit the query.

3. Click on the query bar and select the database and collection.

4. Replace the default query with the following and click Search:

json [{ "$search": { "index": "default", "text": { "query": "meeting", "path": "text.en_US" } } }]

Output:

js SCORE: 0.5960260629653931 _id: "1" message: "try to siGn-In" page_updated_by: Object last_name: "AUERBACH" first_name: "Siân" email: "auerbach@example.com" phone: "(123)-456-7890" text: Object en_US: "<head> This page deals with department meetings.</head>" sv_FI: "Den här sidan behandlar avdelningsmöten" fr_CA: "Cette page traite des réunions de département"

db.minutes.aggregate([
  {
    "$search": {
      "index": "default",
      "text": {
        "query": "meeting",
        "path": "text.en_US"
      }
    }
  },
  {
    "$project": {
      "_id": 1,
      "title": 1,
      "text.en_US": 1
    }
  }
])

Output:

[
  {
    _id: 1,
    title: "The team's weekly meeting",
    text: { en_US: '<head> This page deals with department meetings.</head>' }
  }
]

MongoDB Search returns the document with _id: 1 because the [lowercase](https://www.mongodb.com/docs/search/indexes/analyzers/token-filters.md#std-label-lowercase-tf-ref token filter normalizes token text to lowercase and the englishMinimalStemming token filter stems the plural meetings to the singular meeting, which MongoDB Search matches to the query term. Specifically, MongoDB Search creates the following tokens (searchable terms) for the document in the results, which it then matches to the query term meeting:

flattenGraph

The flattenGraph token filter transforms a token filter graph into a flat form suitable for indexing. If you use the [wordDelimiterGraph](https://www.mongodb.com/docs/search/indexes/analyzers/token-filters.md#std-label-wordDelimiterGraph-tf-ref token filter, use this filter after the [wordDelimiterGraph](https://www.mongodb.com/docs/search/indexes/analyzers/token-filters.md#std-label-wordDelimiterGraph-tf-ref token filter.

Attributes

It has the following attributes:

Example

The following index definition indexes the message field in the [minutes](https://www.mongodb.com/docs/search/indexes/analyzers/custom.md#std-label-custom-analyzers-eg-coll collection using a custom analyzer called wordDelimiterGraphFlatten. The custom analyzer specifies the following:

Apply the [whitespace](https://www.mongodb.com/docs/search/indexes/analyzers/tokenizers.md#std-label-whitespace-tokenizer-ref tokenizer to create tokens based on occurrences of whitespace between words.

Apply the following filters to the tokens:

• [wordDelimiterGraph](https://www.mongodb.com/docs/search/indexes/analyzers/token-filters.md#std-label-wordDelimiterGraph-tf-ref token filter to split tokens based on sub-words, generate tokens for the original words, and also protect the word SIGN_IN from delimination.

• [flattenGraph](https://www.mongodb.com/docs/search/indexes/analyzers/token-filters.md#std-label-flattenGraph-tf-ref token filter to flatten the tokens to a flat form.

Visual Editor

1. In the Custom Analyzers section, click Add Custom Analyzer.

2. Select the Create Your Own radio button and click Next.

3. Type wordDelimiterGraphFlatten in the Analyzer Name field.

4. Expand Tokenizer if it's collapsed.

5. Select whitespace from the dropdown.

6. Expand Token Filters and click Add token filter.

7. Select wordDelimiterGraph from the dropdown and configure the following fields for the token filter.

Select the following fields:

Type SIGN_IN in the protectedWords.words field.

Select protectedWords.ignoreCase.

1. Click Add token filter to add the token filter to your custom analyzer.

2. Click Add token filter to add another token filter.

3. Select flattenGraph from the dropdown.

4. Click Add token filter to add the token filter to your custom analyzer.

5. Click Add to create the custom analyzer.

6. In the Field Mappings section, click Add Field Mapping to apply the custom analyzer on the message field.

7. Select message from the Field Name dropdown and String from the Data Type dropdown.

8. In the properties section for the data type, select wordDelimiterGraphFlatten from the Index Analyzer and Search Analyzer dropdowns.

9. Click Add, then Save Changes.

db.minutes.createSearchIndex(
  "default",
  {
    "mappings": {
      "fields": {
        "message": {
          "type": "string",
          "analyzer": "wordDelimiterGraphFlatten"
        }
      }
    },
    "analyzers": [
      {
        "name": "wordDelimiterGraphFlatten",
        "charFilters": [],
        "tokenizer": {
          "type": "whitespace"
        },
        "tokenFilters": [
          {
            "type": "wordDelimiterGraph",
            "delimiterOptions": {
              "generateWordParts": true,
              "preserveOriginal": true
            },
            "protectedWords": {
              "words": [
                "SIGN_IN"
              ],
              "ignoreCase": false
            }
          },
          {
            "type": "flattenGraph"
          }
        ]
      }
    ]
  }
)

The following query searches the message field in the [minutes](https://www.mongodb.com/docs/search/indexes/analyzers/custom.md#std-label-custom-analyzers-eg-coll collection for the term sign.

1. Click the Query button for your index.

2. Click Edit Query to edit the query.

3. Click on the query bar and select the database and collection.

4. Replace the default query with the following and click Search:

{
  "$search": {
    "index": "default",
    "text": {
      "query": "sign",
      "path": "message"
    }
  }
}

Output:

SCORE: 0.6763891577720642 _id:  "3"
  message: "try to sign-in"
  page_updated_by: Object
  text: Object

db.minutes.aggregate([
  {
    "$search": {
      "index": "default",
      "text": {
        "query": "sign",
        "path": "message"
      }
    }
  },
  {
    "$project": {
      "_id": 1,
      "message": 1
    }
  }
])

Output:

[
  {
    _id: 3,
    message: 'try to sign-in'
  }
]

MongoDB Search returns the document with _id: 3 in the results for the query term sign even though the document contains the hyphenated term sign-in in the title field. The [wordDelimiterGraph](https://www.mongodb.com/docs/search/indexes/analyzers/token-filters.md#std-label-wordDelimiterGraph-tf-ref token filter creates a token filter graph and the [flattenGraph](https://www.mongodb.com/docs/search/indexes/analyzers/token-filters.md#std-label-flattenGraph-tf-ref token filter transforms the token filter graph into a flat form suitable for indexing. Specifically, MongoDB Search creates the following tokens (searchable terms) for the document in the results, which it then matches to the query term sign:

icuFolding

The icuFolding token filter applies character folding from Unicode Technical Report #30 such as accent removal, case folding, canonical duplicates folding, and many others detailed in the report.

Attributes

It has the following attribute:

Example

The following index definition indexes the text.sv_FI field in the [minutes](https://www.mongodb.com/docs/search/indexes/analyzers/custom.md#std-label-custom-analyzers-eg-coll collection using a custom analyzer named diacriticFolder. The custom analyzer specifies the following:

Apply the [keyword tokenizer](https://www.mongodb.com/docs/search/indexes/analyzers/tokenizers.md#std-label-keyword-tokenizer-ref to tokenize all the terms in the string field as a single term.

Use the icuFolding token filter to apply foldings such as accent removal, case folding, canonical duplicates folding, and so on.

Visual Editor

1. In the Custom Analyzers section, click Add Custom Analyzer.

2. Select the Create Your Own radio button and click Next.

3. Type diacriticFolder in the Analyzer Name field.

4. Expand Tokenizer if it's collapsed.

5. Select keyword from the dropdown.

6. Expand Token Filters and click Add token filter.

7. Select icuFolding from the dropdown.

8. Click Add token filter to add the token filter to your custom analyzer.

9. Click Add to create the custom analyzer.

10. In the Field Mappings section, click Add Field Mapping to apply the custom analyzer on the text.sv_FI nested field.

11. Select text.sv_FI nested from the Field Name dropdown and String from the Data Type dropdown.

12. In the properties section for the data type, select diacriticFolder from the Index Analyzer and Search Analyzer dropdowns.

13. Click Add, then Save Changes.

db.minutes.createSearchIndex(
  "default",
  {
    "analyzer": "diacriticFolder",
    "mappings": {
      "fields": {
        "text": {
          "type": "document",
          "fields": {
            "sv_FI": {
              "analyzer": "diacriticFolder",
              "type": "string"
            }
          }
        }
      }
    },
    "analyzers": [
      {
        "name": "diacriticFolder",
        "charFilters": [],
        "tokenizer": {
          "type": "keyword"
        },
        "tokenFilters": [
          {
            "type": "icuFolding"
          }
        ]
      }
    ]
  }
)

The following query uses the [wildcard](https://www.mongodb.com/docs/search/query/operators-collectors/wildcard.md#std-label-wildcard-ref operator to search the text.sv_FI field in the [minutes](https://www.mongodb.com/docs/search/indexes/analyzers/custom.md#std-label-custom-analyzers-eg-coll collection for all terms that contain the term avdelning, preceded and followed by any number of other characters.

1. Click the Query button for your index.

2. Click Edit Query to edit the query.

3. Click on the query bar and select the database and collection.

4. Replace the default query with the following and click Search:

```json { "$search": { "index": "default", "wildcard": { "query": "avdelning", "pat

（本文超出站点单文档体积上限，已截断）