Caches

Back to top

Implementation of caching solutions.

• searx.cache.ExpireCache and its searx.cache.ExpireCacheCfg

class searx.cache.ExpireCacheCfg(name: str, db_url: str = '', MAX_VALUE_LEN: int = 10240, MAXHOLD_TIME: int = 604800, MAINTENANCE_PERIOD: int = 3600, MAINTENANCE_MODE: Literal['auto', 'off'] = 'auto', password: bytes = b'ultrasecretkey')[source] : Configuration of a ExpireCache cache.

name: [str](https://docs.python.org/3/builtins/stdtypes.html#str)
:   Name of the cache.

db\_url: [str](https://docs.python.org/3/builtins/stdtypes.html#str)
:   URL of the SQLite DB, the path to the database file. If unset a default
    DB will be created in /tmp/sxng\_cache\_{self.name}.db

MAX\_VALUE\_LEN: [int](https://docs.python.org/3/builtins/functions.html#int)
:   Max length of a *serialized* value.

MAXHOLD\_TIME: [int](https://docs.python.org/3/builtins/functions.html#int)
:   Hold time (default in sec.), after which a value is removed from the cache.

MAINTENANCE\_PERIOD: [int](https://docs.python.org/3/builtins/functions.html#int)
:   Maintenance period in seconds / when [`MAINTENANCE_MODE`](#searx.cache.ExpireCacheCfg.MAINTENANCE_MODE) is set to
    `auto`.

MAINTENANCE\_MODE: [Literal](https://docs.python.org/3/library/typing.html#typing.Literal)['auto', 'off']
:   Type of maintenance mode

    `auto`:
    :   Maintenance is carried out automatically as part of the maintenance
        intervals ([`MAINTENANCE_PERIOD`](#searx.cache.ExpireCacheCfg.MAINTENANCE_PERIOD)); no external process is required.

    `off`:
    :   Maintenance is switched off and must be carried out by an external process
        if required.

password: [bytes](https://docs.python.org/3/builtins/stdtypes.html#bytes)
:   Password used by [`ExpireCache.secret_hash`](#searx.cache.ExpireCache.secret_hash).

    The default password is taken from [secret\_key](https://docs.searxng.org/admin/settings/settings_server.html#server-secret-key).
    When the password is changed, the hashed keys in the cache can no longer be
    used, which is why all values in the cache are deleted when the password is
    changed.

class searx.cache.ExpireCacheStats(cached_items: dict[str, list[tuple[str, Any, int | None]]])[source] : Dataclass which provides information on the status of the cache.

cached\_items: [dict](https://docs.python.org/3/builtins/stdtypes.html#dict)[[str](https://docs.python.org/3/builtins/stdtypes.html#str), [list](https://docs.python.org/3/builtins/stdtypes.html#list)[[tuple](https://docs.python.org/3/builtins/stdtypes.html#tuple)[[str](https://docs.python.org/3/builtins/stdtypes.html#str), [Any](https://docs.python.org/3/library/typing.html#typing.Any), [int](https://docs.python.org/3/builtins/functions.html#int) | [None](https://docs.python.org/3/builtins/constants.html#None)]]]
:   Values in the cache mapped by context name.

class searx.cache.ExpireCache[source] : Abstract base class for the implementation of a key/value cache with expire date.

abstractmethod set(*key: [str](https://docs.python.org/3/builtins/stdtypes.html#str)*, *value: [Any](https://docs.python.org/3/library/typing.html#typing.Any)*, *expire: [int](https://docs.python.org/3/builtins/functions.html#int) | [None](https://docs.python.org/3/builtins/constants.html#None)*, *ctx: [str](https://docs.python.org/3/builtins/stdtypes.html#str) | [None](https://docs.python.org/3/builtins/constants.html#None) = None*) → [bool](https://docs.python.org/3/builtins/functions.html#bool)[[source]](https://docs.searxng.org/_modules/searx/cache.html#ExpireCache.set)
:   Set *key* to *value*. To set a timeout on key use argument
    `expire` (in sec.). If expire is unset the default is taken from
    [`ExpireCacheCfg.MAXHOLD_TIME`](#searx.cache.ExpireCacheCfg.MAXHOLD_TIME). After the timeout has expired,
    the key will automatically be deleted.

    The `ctx` argument specifies the context of the `key`. A key is
    only unique in its context.

    The concrete implementations of this abstraction determine how the
    context is mapped in the connected database. In SQL databases, for
    example, the context is a DB table or in a Key/Value DB it could be
    a prefix for the key.

    If the context is not specified (the default is `None`) then a
    default context should be used, e.g. a default table for SQL databases
    or a default prefix in a Key/Value DB.

abstractmethod get(*key: [str](https://docs.python.org/3/builtins/stdtypes.html#str)*, *default: [Any](https://docs.python.org/3/library/typing.html#typing.Any) = None*, *ctx: [str](https://docs.python.org/3/builtins/stdtypes.html#str) | [None](https://docs.python.org/3/builtins/constants.html#None) = None*) → [Any](https://docs.python.org/3/library/typing.html#typing.Any)[[source]](https://docs.searxng.org/_modules/searx/cache.html#ExpireCache.get)
:   Return *value* of *key*. If key is unset, `None` is returned.

abstractmethod maintenance(*force: [bool](https://docs.python.org/3/builtins/functions.html#bool) = False*, *truncate: [bool](https://docs.python.org/3/builtins/functions.html#bool) = False*) → [bool](https://docs.python.org/3/builtins/functions.html#bool)[[source]](https://docs.searxng.org/_modules/searx/cache.html#ExpireCache.maintenance)
:   Performs maintenance on the cache.

    `force`:
    :   Maintenance should be carried out even if the maintenance interval has
        not yet been reached.

    `truncate`:
    :   Truncate the entire cache, which is necessary, for example, if the
        password has changed.

abstractmethod state() → [ExpireCacheStats](#searx.cache.ExpireCacheStats)[[source]](https://docs.searxng.org/_modules/searx/cache.html#ExpireCache.state)
:   Returns a [`ExpireCacheStats`](#searx.cache.ExpireCacheStats), which provides information
    about the status of the cache.

static build\_cache(*cfg: [ExpireCacheCfg](#searx.cache.ExpireCacheCfg)*) → [ExpireCacheSQLite](#searx.cache.ExpireCacheSQLite)[[source]](https://docs.searxng.org/_modules/searx/cache.html#ExpireCache.build_cache)
:   Factory to build a caching instance.

    #### Note

    Currently, only the SQLite adapter is available, but other database
    types could be implemented in the future, e.g. a Valkey (Redis)
    adapter.

static normalize\_name(*name: [str](https://docs.python.org/3/builtins/stdtypes.html#str)*) → [str](https://docs.python.org/3/builtins/stdtypes.html#str)[[source]](https://docs.searxng.org/_modules/searx/cache.html#ExpireCache.normalize_name)
:   Returns a normalized name that can be used as a file name or as a SQL
    table name (is used, for example, to normalize the context name).

secret\_hash(*name: [str](https://docs.python.org/3/builtins/stdtypes.html#str) | [bytes](https://docs.python.org/3/builtins/stdtypes.html#bytes)*) → [str](https://docs.python.org/3/builtins/stdtypes.html#str)[[source]](https://docs.searxng.org/_modules/searx/cache.html#ExpireCache.secret_hash)
:   Creates a hash of the argument `name`. The hash value is formed
    from the `name` combined with the [`password`](#searx.cache.ExpireCacheCfg.password). Can be used, for example, to make the
    `key` stored in the DB unreadable for third parties.

class searx.cache.ExpireCacheSQLite(cfg: ExpireCacheCfg)[source] : Cache that manages key/value pairs in a SQLite DB. The DB model in the SQLite DB is implemented in abstract class SQLiteAppl.

The following configurations are required / supported:

- [`ExpireCacheCfg.db_url`](#searx.cache.ExpireCacheCfg.db_url)
- [`ExpireCacheCfg.MAXHOLD_TIME`](#searx.cache.ExpireCacheCfg.MAXHOLD_TIME)
- [`ExpireCacheCfg.MAINTENANCE_PERIOD`](#searx.cache.ExpireCacheCfg.MAINTENANCE_PERIOD)
- [`ExpireCacheCfg.MAINTENANCE_MODE`](#searx.cache.ExpireCacheCfg.MAINTENANCE_MODE)

DB\_SCHEMA: [int](https://docs.python.org/3/builtins/functions.html#int) = 1
:   As soon as changes are made to the DB schema, the version number must be
    increased. Changes to the version number require the DB to be recreated (or
    migrated / if an migration path exists and is implemented).

init(*conn: [Connection](https://docs.python.org/3/library/sqlite3.html#sqlite3.Connection)*) → [bool](https://docs.python.org/3/builtins/functions.html#bool)[[source]](https://docs.searxng.org/_modules/searx/cache.html#ExpireCacheSQLite.init)
:   Initializes the DB schema and properties, is only executed once even
    if called several times.

    If the initialization has not yet taken place, it is carried out and a
    True is returned to the caller at the end. If the initialization has
    already been carried out in the past, False is returned.

maintenance(*force: [bool](https://docs.python.org/3/builtins/functions.html#bool) = False*, *truncate: [bool](https://docs.python.org/3/builtins/functions.html#bool) = False*) → [bool](https://docs.python.org/3/builtins/functions.html#bool)[[source]](https://docs.searxng.org/_modules/searx/cache.html#ExpireCacheSQLite.maintenance)
:   Performs maintenance on the cache.

    `force`:
    :   Maintenance should be carried out even if the maintenance interval has
        not yet been reached.

    `truncate`:
    :   Truncate the entire cache, which is necessary, for example, if the
        password has changed.

create\_table(*table: [str](https://docs.python.org/3/builtins/stdtypes.html#str)*) → [bool](https://docs.python.org/3/builtins/functions.html#bool)[[source]](https://docs.searxng.org/_modules/searx/cache.html#ExpireCacheSQLite.create_table)
:   Create DB `table` if it has not yet been created, no recreates are
    initiated if the table already exists.

property table\_names: [list](https://docs.python.org/3/builtins/stdtypes.html#list)[[str](https://docs.python.org/3/builtins/stdtypes.html#str)]
:   List of key/value tables already created in the DB.

property next\_maintenance\_time: [int](https://docs.python.org/3/builtins/functions.html#int)
:   Returns (unix epoch) time of the next maintenance.

set(*key: [str](https://docs.python.org/3/builtins/stdtypes.html#str)*, *value: [Any](https://docs.python.org/3/library/typing.html#typing.Any)*, *expire: [int](https://docs.python.org/3/builtins/functions.html#int) | [None](https://docs.python.org/3/builtins/constants.html#None)*, *ctx: [str](https://docs.python.org/3/builtins/stdtypes.html#str) | [None](https://docs.python.org/3/builtins/constants.html#None) = None*) → [bool](https://docs.python.org/3/builtins/functions.html#bool)[[source]](https://docs.searxng.org/_modules/searx/cache.html#ExpireCacheSQLite.set)
:   Set key/value in DB table given by argument `ctx`. If expire is
    unset the default is taken from [`ExpireCacheCfg.MAXHOLD_TIME`](#searx.cache.ExpireCacheCfg.MAXHOLD_TIME).
    If `ctx` argument is `None` (the default), a table name is
    generated from the [`ExpireCacheCfg.name`](#searx.cache.ExpireCacheCfg.name). If DB table does not
    exists, it will be created (on demand) by [`self.create_table`](#searx.cache.ExpireCacheSQLite.create_table).

setmany(*opt\_list: [list](https://docs.python.org/3/builtins/stdtypes.html#list)[[tuple](https://docs.python.org/3/builtins/stdtypes.html#tuple)[[str](https://docs.python.org/3/builtins/stdtypes.html#str), [Any](https://docs.python.org/3/library/typing.html#typing.Any), [int](https://docs.python.org/3/builtins/functions.html#int) | [None](https://docs.python.org/3/builtins/constants.html#None)]]*, *ctx: [str](https://docs.python.org/3/builtins/stdtypes.html#str) | [None](https://docs.python.org/3/builtins/constants.html#None) = None*) → [int](https://docs.python.org/3/builtins/functions.html#int)[[source]](https://docs.searxng.org/_modules/searx/cache.html#ExpireCacheSQLite.setmany)
:   Efficient bootload of the cache from a list of options. The list
    contains tuples with the arguments described in
    [`ExpireCacheSQLite.set`](#searx.cache.ExpireCacheSQLite.set).

get(*key: [str](https://docs.python.org/3/builtins/stdtypes.html#str)*, *default: [Any](https://docs.python.org/3/library/typing.html#typing.Any) = None*, *ctx: [str](https://docs.python.org/3/builtins/stdtypes.html#str) | [None](https://docs.python.org/3/builtins/constants.html#None) = None*) → [Any](https://docs.python.org/3/library/typing.html#typing.Any)[[source]](https://docs.searxng.org/_modules/searx/cache.html#ExpireCacheSQLite.get)
:   Get value of `key` from table given by argument `ctx`. If
    `ctx` argument is `None` (the default), a table name is generated
    from the [`ExpireCacheCfg.name`](#searx.cache.ExpireCacheCfg.name). If `key` not exists in
    the table or the table not exists, the `default` value is returned.

pairs(*ctx: [str](https://docs.python.org/3/builtins/stdtypes.html#str)*) → [Iterator](https://docs.python.org/3/library/collections.abc.html#collections.abc.Iterator)[[tuple](https://docs.python.org/3/builtins/stdtypes.html#tuple)[[str](https://docs.python.org/3/builtins/stdtypes.html#str), [Any](https://docs.python.org/3/library/typing.html#typing.Any)]][[source]](https://docs.searxng.org/_modules/searx/cache.html#ExpireCacheSQLite.pairs)
:   Iterate over key/value pairs from table given by argument `ctx`.
    If `ctx` argument is `None` (the default), a table name is
    generated from the [`ExpireCacheCfg.name`](#searx.cache.ExpireCacheCfg.name).

state() → [ExpireCacheStats](#searx.cache.ExpireCacheStats)[[source]](https://docs.searxng.org/_modules/searx/cache.html#ExpireCacheSQLite.state)
:   Returns a [`ExpireCacheStats`](#searx.cache.ExpireCacheStats), which provides information
    about the status of the cache.

Copyright © SearXNG team

Made with Furo

💾 🌐