Engine Library

Engine Library
- Hint
- Engine traits

Implementations of the framework for the SearXNG engines.

searx.enginelib.EngineCache
searx.enginelib.Engine
searx.enginelib.EngineAbout
searx.enginelib.traits

There is a command line for developer purposes and for deeper analysis. Here is an example in which the command line is called in the development environment:

$ ./manage dev.env
(dev.env)$ python -m searx.enginelib --help

Hint

The long term goal is to modularize all implementations of the engine framework here in this Python package. ToDo:

move implementations of the SearXNG’s engines loader to a new module in the searx.enginelib namespace.

class searx.enginelib.EngineCache(engine_name: str, expire: int | None = None)[source] : Persistent (SQLite) key/value cache that deletes its values again after expire seconds (default/max: MAXHOLD_TIME). This class is a wrapper around ENGINES_CACHE (ExpireCacheSQLite).

In the engine you can find an
exemplary implementation of such a cache other examples are implemented
in:

For introspection of the DB, jump into developer environment and run command to
show cache state:

```
$ ./manage dev.env
(dev.env)$ python -m searx.enginelib cache state

cache tables and key/values
===========================
[demo_offline        ] 2025-04-22 11:32:50 count        --> (int) 4
[startpage           ] 2025-04-22 12:32:30 SC_CODE      --> (str) fSOBnhEMlDfE20
[duckduckgo          ] 2025-04-22 12:32:31 4dff493e.... --> (str) 4-128634958369380006627592672385352473325
[duckduckgo          ] 2025-04-22 12:40:06 3e2583e2.... --> (str) 4-263126175288871260472289814259666848451
[radio_browser       ] 2025-04-23 11:33:08 servers      --> (list) ['https://de2.api.radio-browser.info',  ...]
[soundcloud          ] 2025-04-29 11:40:06 guest_client_id --> (str) EjkRJG0BLNEZquRiPZYdNtJdyGtTuHdp
[wolframalpha        ] 2025-04-22 12:40:06 code         --> (str) 5aa79f86205ad26188e0e26e28fb7ae7
number of tables: 6
number of key/value pairs: 7
```

In the “cache tables and key/values” section, the table name (engine name) is at
first position on the second there is the calculated expire date and on the
third and fourth position the key/value is shown.

About duckduckgo: The *vqd coode* of ddg depends on the query term and therefore
the key is a hash value of the query term (to not to store the raw query term).

In the “properties of ENGINES\_CACHE” section all properties of the SQLiteAppl /
ExpireCache and their last modification date are shown:

```
properties of ENGINES_CACHE
===========================
[last modified: 2025-04-22 11:32:27] DB_SCHEMA           : 1
[last modified: 2025-04-22 11:32:27] LAST_MAINTENANCE    :
[last modified: 2025-04-22 11:32:27] crypt_hash          : ca612e3566fdfd7cf7efe...
[last modified: 2025-04-22 11:32:30] CACHE-TABLE--demo_offline: demo_offline
[last modified: 2025-04-22 11:32:30] CACHE-TABLE--startpage: startpage
[last modified: 2025-04-22 11:32:31] CACHE-TABLE--duckduckgo: duckduckgo
[last modified: 2025-04-22 11:33:08] CACHE-TABLE--radio_browser: radio_browser
[last modified: 2025-04-22 11:40:06] CACHE-TABLE--soundcloud: soundcloud
[last modified: 2025-04-22 11:40:06] CACHE-TABLE--wolframalpha: wolframalpha
```

These properties provide information about the state of the ExpireCache and
control the behavior. For example, the maintenance intervals are controlled by
the last modification date of the LAST\_MAINTENANCE property and the hash value
of the password can be used to detect whether the password has been changed (in
this case the DB entries can no longer be decrypted and the entire cache must be
discarded).

class searx.enginelib.Engine[source] : Class of engine instances build from YAML settings.

Further documentation see [General Engine Configuration](https://docs.searxng.org/dev/engines/engine_overview.html#general-engine-configuration).

The defaults are taken from [`searx.engines.ENGINE_DEFAULT_ARGS`](https://docs.searxng.org/dev/engines/engines.html#searx.engines.ENGINE_DEFAULT_ARGS).

#### Hint

This class is currently never initialized and only used for type hinting.

engine\_type: ProcessorType = 'online'
:   Type of the engine ([Search processors](https://docs.searxng.org/src/searx.search.processors.html#searx-search-processors))

paging: [bool](https://docs.python.org/3/builtins/functions.html#bool) = False
:   Engine supports multiple pages.

max\_page: [int](https://docs.python.org/3/builtins/functions.html#int) = 0
:   If the engine supports paging, then this is the value for the last page
    that is still supported. `0` means unlimited numbers of pages.

time\_range\_support: [bool](https://docs.python.org/3/builtins/functions.html#bool) = False
:   Engine supports search time range.

safesearch: [bool](https://docs.python.org/3/builtins/functions.html#bool) = False
:   Engine supports SafeSearch

language\_support: [bool](https://docs.python.org/3/builtins/functions.html#bool) = False
:   Engine supports languages (locales) search.

fetch\_traits: Callable[[[EngineTraits](#searx.enginelib.traits.EngineTraits), [bool](https://docs.python.org/3/builtins/functions.html#bool)], [None](https://docs.python.org/3/builtins/constants.html#None)]
:   Function to to fetch engine’s traits from origin.

traits: [traits.EngineTraits](#searx.enginelib.traits.EngineTraits)
:   Traits of the engine.

name: [str](https://docs.python.org/3/builtins/stdtypes.html#str)
:   Name that will be used across SearXNG to define this engine. In settings, on
    the result page ..

engine: [str](https://docs.python.org/3/builtins/stdtypes.html#str)
:   Name of the python file used to handle requests and responses to and from
    this search engine (file name from without
    `.py`).

categories: [list](https://docs.python.org/3/builtins/stdtypes.html#list)[[str](https://docs.python.org/3/builtins/stdtypes.html#str)] = ['general']
:   Specifies to which [categories](https://docs.searxng.org/admin/settings/settings_engines.html#engine-categories) the engine should be added.

language: [str](https://docs.python.org/3/builtins/stdtypes.html#str) = ''
:   If the engine supports only one language, this language is specified here
    (`en`, `de`, `“no”` or ..); otherwise, the value remains empty. For
    the YAML configuration: think of the [YAML-Norway problem](https://ruuda.nl/2023/the-yaml-document-from-hell#the-norway-problem)

    ```
    - name: google norway
      engine: google
      language: "no"
    ```

    Depending on `language_support`, this value has similar but also slightly
    different meanings.

    - When `language_support` is **true**, the map of
      [`traits.EngineTraits.languages`](#searx.enginelib.traits.EngineTraits.languages) is reduced to the selected
      language
    - When `language_support` is **false**, then the implementation of the
      engine only supports this one `language`

region: [str](https://docs.python.org/3/builtins/stdtypes.html#str) = ''
:   For an engine, when there is `region: ...` in the YAML settings the engine
    does support only this one region:

    ```
    - name: google belgium
      engine: google
      region: fr-BE
    ```

enable\_http: [bool](https://docs.python.org/3/builtins/functions.html#bool)
:   Enable HTTP (by default only HTTPS is enabled).

enable\_http3: [bool](https://docs.python.org/3/builtins/functions.html#bool) = False
:   Enables the use of HTTP/3 if available

shortcut: [str](https://docs.python.org/3/builtins/stdtypes.html#str)
:   Code used to execute bang requests (`!foo`)

timeout: [float](https://docs.python.org/3/builtins/functions.html#float)
:   Specific timeout for search-engine.

display\_error\_messages: [bool](https://docs.python.org/3/builtins/functions.html#bool)
:   Display error messages on the web UI.

disabled: [bool](https://docs.python.org/3/builtins/functions.html#bool) = False
:   To disable by default the engine, but not deleting it. It will allow the
    user to manually activate it in the settings.

inactive: [bool](https://docs.python.org/3/builtins/functions.html#bool) = False
:   Remove the engine from the settings (*disabled & removed*).

about: [EngineAbout](#searx.enginelib.EngineAbout) = EngineAbout(website='', wikidata\_id='', official\_api\_documentation='', use\_official\_api=False, require\_api\_key=False, results='', description='', language='')
:   Additional fields describing the engine.

using\_tor\_proxy: [bool](https://docs.python.org/3/builtins/functions.html#bool) = False
:   Using tor proxy (`true`) or not (`false`) for this engine.

:   When this option is activated (default), the language (locale) that is
    selected by the user is used to build and send a `Accept-Language` header
    in the request to the origin search engine.

tokens: [list](https://docs.python.org/3/builtins/stdtypes.html#list)[[str](https://docs.python.org/3/builtins/stdtypes.html#str)] = []
:   A list of secret tokens to make this engine *private*, more details see
    [Private Engines (tokens)](https://docs.searxng.org/admin/settings/settings_engines.html#private-engines).

weight: [float](https://docs.python.org/3/builtins/functions.html#float) = 1.0
:   Weighting of the results of this engine ([weight](https://docs.searxng.org/admin/settings/settings_engines.html#settings-engines)).

proxies: [dict](https://docs.python.org/3/builtins/stdtypes.html#dict)[[str](https://docs.python.org/3/builtins/stdtypes.html#str), [dict](https://docs.python.org/3/builtins/stdtypes.html#dict)[[str](https://docs.python.org/3/builtins/stdtypes.html#str), [str](https://docs.python.org/3/builtins/stdtypes.html#str)]]
:   Set proxies for a specific engine (YAML):

    ```
    proxies :
      http:  socks5://proxy:port
      https: socks5://proxy:port
    ```

setup(*engine\_settings: [dict](https://docs.python.org/3/builtins/stdtypes.html#dict)[[str](https://docs.python.org/3/builtins/stdtypes.html#str), [Any](https://docs.python.org/3/library/typing.html#typing.Any)]*) → [bool](https://docs.python.org/3/builtins/functions.html#bool) | [None](https://docs.python.org/3/builtins/constants.html#None)[[source]](https://docs.searxng.org/_modules/searx/enginelib.html#Engine.setup)
:   Dynamic setup of the engine settings.

    With this method, the engine’s setup is carried out. For example, to
    check or dynamically adapt the values handed over in the parameter
    `engine_settings`.

    Whether the initialization was successful can be indicated by the return
    value `True` or even `False`.

    - If no return value (`None` ) is given from this method , this is
      equivalent to `True`.
    - If an exception is thrown as part of the initialization, this is
      equivalent to `False`.

    The method is optional and is called **synchronously** as part of the
    initialization of the service and is therefore only suitable for simple
    (local) exams/changes at the engine setting.

    The [`Engine.init`](#searx.enginelib.Engine.init) method must be used for longer tasks in which
    values of a remote must be determined, for example.

init(*engine\_settings: [dict](https://docs.python.org/3/builtins/stdtypes.html#dict)[[str](https://docs.python.org/3/builtins/stdtypes.html#str), [Any](https://docs.python.org/3/library/typing.html#typing.Any)]*) → [bool](https://docs.python.org/3/builtins/functions.html#bool) | [None](https://docs.python.org/3/builtins/constants.html#None)[[source]](https://docs.searxng.org/_modules/searx/enginelib.html#Engine.init)
:   Initialization of the engine.

    The method is optional and called **asynchronous** (in a thread). The
    method is comparable to [`Engine.setup`](#searx.enginelib.Engine.setup), it is suitable, for
    caching data that first needs to be requested from a remote.

    The method is optional and runs **asynchronously** (in a thread), it is
    comparable to [`Engine.setup`](#searx.enginelib.Engine.setup). For instance, it is suitable for
    caching data that first needs to be requested from a remote source.

    The evaluation of the return value is analogous to [`Engine.setup`](#searx.enginelib.Engine.setup).

abstractmethod search(*query: [str](https://docs.python.org/3/builtins/stdtypes.html#str)*, *params: OfflineParamTypes*) → [EngineResults](https://docs.searxng.org/dev/engines/index.html#searx.result_types.EngineResults)[[source]](https://docs.searxng.org/_modules/searx/enginelib.html#Engine.search)
:   Search method of the `offline` engines

abstractmethod request(*query: [str](https://docs.python.org/3/builtins/stdtypes.html#str)*, *params: OnlineParamTypes*) → [None](https://docs.python.org/3/builtins/constants.html#None)[[source]](https://docs.searxng.org/_modules/searx/enginelib.html#Engine.request)
:   Method to build the parameters for the request of an `online`
    engine.

abstractmethod response(*resp: [SXNG\_Response](https://docs.searxng.org/dev/extended_types.html#searx.extended_types.SXNG_Response)*) → [EngineResults](https://docs.searxng.org/dev/engines/index.html#searx.result_types.EngineResults)[[source]](https://docs.searxng.org/_modules/searx/enginelib.html#Engine.response)
:   Method to parse the response of an `online` engine.

class searx.enginelib.EngineAbout(*, website: str = '', wikidata_id: str = '', official_api_documentation: str = '', use_official_api: bool = False, require_api_key: bool = False, results: str = '', description: str = '', language: str = '')[source] : Additional fields describing the engine.

```
about:
   website: https://example.com
   wikidata_id: Q306656
   official_api_documentation: https://example.com/api-doc
   use_official_api: true
   require_api_key: true
   results: HTML
```

website: [str](https://docs.python.org/3/builtins/stdtypes.html#str)
:   Official web-site of the origin.

wikidata\_id: [str](https://docs.python.org/3/builtins/stdtypes.html#str)
:   [Wikidata ID](https://www.wikidata.org/wiki/Wikidata:Identifiers)

official\_api\_documentation: [str](https://docs.python.org/3/builtins/stdtypes.html#str)
:   URL of the official API (regardless of whether it is used)

use\_official\_api: [bool](https://docs.python.org/3/builtins/functions.html#bool)
:   SearXNG engine makes use of the official API or not

require\_api\_key: [bool](https://docs.python.org/3/builtins/functions.html#bool)
:   API requires a key or not.

results: [str](https://docs.python.org/3/builtins/stdtypes.html#str)
:   Data format of the source (online-engines: of the response).

description: [str](https://docs.python.org/3/builtins/stdtypes.html#str)
:   Brief description of the engine and where it gets its data from.

    This value should only be set as long as no description of the data source
    is available via a [`EngineAbout.wikidata_id`](#searx.enginelib.EngineAbout.wikidata_id).

language: [str](https://docs.python.org/3/builtins/stdtypes.html#str)
:   Deprecated! Migrate your setting from engine.about.language to
    engine.language

searx.enginelib.ENGINES_CACHE: ExpireCacheSQLite = <searx.cache.ExpireCacheSQLite object> : Global searx.cache.ExpireCacheSQLite instance where the cached values from all engines are stored. The MAXHOLD_TIME is 7 days and the MAINTENANCE_PERIOD is set to two hours.

Engine traits

Engine’s traits are fetched from the origin engines and stored in a JSON file in the data folder. Most often traits are languages and region codes and their mapping from SearXNG’s representation to the representation in the origin search engine. For new traits new properties can be added to the class EngineTraits.

To load traits from the persistence EngineTraitsMap.from_data can be used.

class searx.enginelib.traits.EngineTraitsEncoder(*, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False, indent=None, separators=None, default=None)[source] : Encodes EngineTraits to a serializable object, see json.JSONEncoder.

default(*o: [Any](https://docs.python.org/3/library/typing.html#typing.Any)*) → [Any](https://docs.python.org/3/library/typing.html#typing.Any)[[source]](https://docs.searxng.org/_modules/searx/enginelib/traits.html#EngineTraitsEncoder.default)
:   Return dictionary of a [`EngineTraits`](#searx.enginelib.traits.EngineTraits) object.

class searx.enginelib.traits.EngineTraits(regions: dict[str, str]=<factory>, languages: dict[str, str]=<factory>, all_locale: str | None = None, data_type: Literal['traits_v1'] = 'traits_v1', custom: dict[str, ~typing.Any]=<factory>)[source] : The class is intended to be instantiated for each engine.

regions: [dict](https://docs.python.org/3/builtins/stdtypes.html#dict)[[str](https://docs.python.org/3/builtins/stdtypes.html#str), [str](https://docs.python.org/3/builtins/stdtypes.html#str)]
:   Maps SearXNG’s internal representation of a region to the one of the engine.

    SearXNG’s internal representation can be parsed by babel and the value is
    send to the engine:

    ```
    regions ={
        'fr-BE' : <engine's region name>,
    }

    for key, egnine_region regions.items():
       searxng_region = babel.Locale.parse(key, sep='-')
       ...
    ```

languages: [dict](https://docs.python.org/3/builtins/stdtypes.html#dict)[[str](https://docs.python.org/3/builtins/stdtypes.html#str), [str](https://docs.python.org/3/builtins/stdtypes.html#str)]
:   Maps SearXNG’s internal representation of a language to the one of the engine.

    SearXNG’s internal representation can be parsed by babel and the value is
    send to the engine:

    ```
    languages = {
        'ca' : <engine's language name>,
    }

    for key, egnine_lang in languages.items():
       searxng_lang = babel.Locale.parse(key)
       ...
    ```

all\_locale: [str](https://docs.python.org/3/builtins/stdtypes.html#str) | [None](https://docs.python.org/3/builtins/constants.html#None) = None
:   To which locale value SearXNG’s `all` language is mapped (shown a “Default
    language”).

data\_type: [Literal](https://docs.python.org/3/library/typing.html#typing.Literal)['traits\_v1'] = 'traits\_v1'
:   Data type, default is ‘traits\_v1’.

custom: [dict](https://docs.python.org/3/builtins/stdtypes.html#dict)[[str](https://docs.python.org/3/builtins/stdtypes.html#str), [Any](https://docs.python.org/3/library/typing.html#typing.Any)]
:   A place to store engine’s custom traits, not related to the SearXNG core.

get\_language(*searxng\_locale: [str](https://docs.python.org/3/builtins/stdtypes.html#str)*, *default: [str](https://docs.python.org/3/builtins/stdtypes.html#str) | [None](https://docs.python.org/3/builtins/constants.html#None) = None*) → [str](https://docs.python.org/3/builtins/stdtypes.html#str) | [None](https://docs.python.org/3/builtins/constants.html#None)[[source]](https://docs.searxng.org/_modules/searx/enginelib/traits.html#EngineTraits.get_language)
:   Return engine’s language string that *best fits* to SearXNG’s locale.

    Parameters:
    :   - **searxng\_locale** – SearXNG’s internal representation of locale
          selected by the user.
        - **default** – engine’s default language

    The *best fits* rules are implemented in
    [`searx.locales.get_engine_locale`](https://docs.searxng.org/src/searx.locales.html#searx.locales.get_engine_locale). Except for the special value `all`
    which is determined from [`EngineTraits.all_locale`](#searx.enginelib.traits.EngineTraits.all_locale).

get\_region(*searxng\_locale: [str](https://docs.python.org/3/builtins/stdtypes.html#str)*, *default: [str](https://docs.python.org/3/builtins/stdtypes.html#str) | [None](https://docs.python.org/3/builtins/constants.html#None) = None*) → [str](https://docs.python.org/3/builtins/stdtypes.html#str) | [None](https://docs.python.org/3/builtins/constants.html#None)[[source]](https://docs.searxng.org/_modules/searx/enginelib/traits.html#EngineTraits.get_region)
:   Return engine’s region string that best fits to SearXNG’s locale.

    Parameters:
    :   - **searxng\_locale** – SearXNG’s internal representation of locale
          selected by the user.
        - **default** – engine’s default region

    The *best fits* rules are implemented in
    [`searx.locales.get_engine_locale`](https://docs.searxng.org/src/searx.locales.html#searx.locales.get_engine_locale). Except for the special value `all`
    which is determined from [`EngineTraits.all_locale`](#searx.enginelib.traits.EngineTraits.all_locale).

copy()[[source]](https://docs.searxng.org/_modules/searx/enginelib/traits.html#EngineTraits.copy)
:   Create a copy of the dataclass object.

classmethod fetch\_traits(*engine: [Engine](#searx.enginelib.Engine) | [types.ModuleType](https://docs.python.org/3/library/types.html#types.ModuleType)*) → [EngineTraits](#searx.enginelib.traits.EngineTraits) | [None](https://docs.python.org/3/builtins/constants.html#None)[[source]](https://docs.searxng.org/_modules/searx/enginelib/traits.html#EngineTraits.fetch_traits)
:   Call a function `fetch_traits(engine_traits)` from engines namespace to fetch
    and set properties from the origin engine in the object `engine_traits`. If
    function does not exists, `None` is returned.

set\_traits(*engine: [Engine](#searx.enginelib.Engine) | [types.ModuleType](https://docs.python.org/3/library/types.html#types.ModuleType)*) → [None](https://docs.python.org/3/builtins/constants.html#None)[[source]](https://docs.searxng.org/_modules/searx/enginelib/traits.html#EngineTraits.set_traits)
:   Set traits from self object in a [`Engine`](#searx.enginelib.Engine) namespace.

    Parameters:
    :   **engine** – engine instance build by [`searx.engines.load_engine()`](https://docs.searxng.org/dev/engines/engines.html#searx.engines.load_engine)

class searx.enginelib.traits.EngineTraitsMap[source] : A python dictionary to map EngineTraits by engine name.

ENGINE\_TRAITS\_FILE: [Path](https://docs.python.org/3/library/pathlib.html#pathlib.Path) = PosixPath('/home/runner/work/searxng/searxng/searx/data/engine\_traits.json')
:   File with persistence of the [`EngineTraitsMap`](#searx.enginelib.traits.EngineTraitsMap).

save\_data()[[source]](https://docs.searxng.org/_modules/searx/enginelib/traits.html#EngineTraitsMap.save_data)
:   Store EngineTraitsMap in in file `self.ENGINE_TRAITS_FILE`

classmethod from\_data() → [EngineTraitsMap](#searx.enginelib.traits.EngineTraitsMap)[[source]](https://docs.searxng.org/_modules/searx/enginelib/traits.html#EngineTraitsMap.from_data)
:   Instantiate [`EngineTraitsMap`](#searx.enginelib.traits.EngineTraitsMap) object from `ENGINE_TRAITS`

set\_traits(*engine: [Engine](#searx.enginelib.Engine) | [types.ModuleType](https://docs.python.org/3/library/types.html#types.ModuleType)*)[[source]](https://docs.searxng.org/_modules/searx/enginelib/traits.html#EngineTraitsMap.set_traits)
:   Set traits in a `Engine` namespace.

    Parameters:
    :   **engine** – engine instance build by [`searx.engines.load_engine()`](https://docs.searxng.org/dev/engines/engines.html#searx.engines.load_engine)

Made with Furo

💾 🌐