Google Engines

Google Engines

Google API

SearXNG’s implementation of the Google API is mainly done in get_google_info.

For detailed description of the REST-full API see: Query Parameter Definitions. The linked API documentation can sometimes be helpful during reverse engineering. However, we cannot use it in the freely accessible WEB services; not all parameters can be applied and some engines are more special than other (e.g. Google News).

Google WEB

This is the implementation of the Google WEB engine. Some of this implementations (manly the get_google_info) are shared by other engines:

Google Images
Google News
Google Videos
Google Scholar
Google Autocomplete

This implementation uses Nokia user agents to request an XML layout from Google. The normal web version requires executing JavaScript to load the results and therefore is currently not used here. See Google discussion for more information on that topic.

searx.engines.google.max_page = 50 : Google supports up to 50 pages of results, see the Google max_page discussion.

searx.engines.google.get_google_info(params: OnlineParams, eng_traits: EngineTraits) → dict[str, Any][source] : Composing various (language) properties for the google engines (Google API).

This function is called by the various google engines ([Google WEB](#google-web-engine), [Google Images](#google-images-engine), [Google News](#google-news-engine) and
[Google Videos](#google-videos-engine)).

Parameters:
:   - **param** ([*dict*](https://docs.python.org/3/builtins/stdtypes.html#dict)) – Request parameters of the engine. At least
      a `searxng_locale` key should be in the dictionary.
    - **eng\_traits** – Engine’s traits fetched from google preferences
      ([`searx.enginelib.traits.EngineTraits`](https://docs.searxng.org/dev/engines/enginelib.html#searx.enginelib.traits.EngineTraits))

Return type:
:   [dict](https://docs.python.org/3/builtins/stdtypes.html#dict)

Returns:
:   Py-Dictionary with the key/value pairs:

    language:
    :   The language code that is used by google (e.g. `lang_en` or
        `lang_zh-TW`)

    country:
    :   The country code that is used by google (e.g. `US` or `TW`)

    locale:
    :   A instance of [`babel.core.Locale`](https://babel.readthedocs.io/en/latest/api/core.html#babel.core.Locale) build from the
        `searxng_locale` value.

    params:
    :   Py-Dictionary with additional request arguments (can be passed to
        [`urllib.parse.urlencode()`](https://docs.python.org/3/library/urllib.parse.html#urllib.parse.urlencode)).

        - `hl` parameter: specifies the interface language of user interface.
        - `ie` parameter: sets the character encoding scheme that should
          be used to interpret the query string (‘utf8’).
        - `oe` parameter: sets the character encoding scheme that should
          be used to decode the XML result (‘utf8’).

    headers:
    :   Py-Dictionary with additional HTTP headers (can be passed to
        request’s headers)

        - `Accept: '*/*`

searx.engines.google.detect_google_sorry(resp: SXNG_Response)[source] : Detect Google’s bot-protection responses (CAPTCHA / sorry pages).

Google may block requests in several ways:

1. Redirect to sorry.google.com (standard CAPTCHA).
2. HTTP 302 redirect to `/sorry/index?...` on the same host – when the
   HTTP client doesn’t follow the redirect, the response body is a short
   HTML stub with a link to the sorry page.
3. Short HTML response (<2000 bytes) containing “/sorry/” – a meta-refresh
   or JS redirect variant.

searx.engines.google.fetch_traits(engine_traits: EngineTraits)[source] : Fetch languages from Google.

Google Autocomplete

searx.autocomplete.google_complete(query: str, sxng_locale: str) → list[str][source] : Autocomplete from Google. Supports Google’s languages (searx.engines.google.get_google_info) by using the async REST API:

```
https://www.google.com/complete/search?{args}
```

Google Images

Google Images: see searx.engines.google.

searx.engines.google_images.max_page = 50 : Google supports up to 50 pages of results, see the Google max_page discussion.

Google Videos

Google Videos: see searx.engines.google.

searx.engines.google_videos.max_page = 50 : Google supports up to 50 pages of results, see the Google max_page discussion.

Google News

Google News: see searx.engines.google.

searx.engines.google_news.max_page = 50 : Google supports up to 50 pages of results, see the Google max_page discussion.

Google Scholar

Google Scholar is a freely accessible web search engine that indexes the full text or metadata of scholarly literature across an array of publishing formats and disciplines.

Compared to other Google services the Scholar engine has a simple GET REST-API and there does not exists async API. Even though the API slightly vintage we can make use of the Google API to assemble the arguments of the GET request.

Configuration

- name: google scholar
  engine: google_scholar
  shortcut: gos

Implementations

searx.engines.google_scholar.max_page = 50 : Google max 50 pages

searx.engines.google_scholar.request(query: str, params: OnlineParams) → None [source] : Google-Scholar search request

searx.engines.google_scholar.response(resp: SXNG_Response) → EngineResults [source] : Parse response from Google Scholar

searx.engines.google_scholar.time_range_args(params: OnlineParams) → dict[str, int][source] : Returns a dictionary with a time range arguments based on params[“time_range”].

Google Scholar supports a detailed search by year. Searching by *last
month* or *last week* (as offered by SearXNG) is uncommon for scientific
publications and is not supported by Google Scholar.

To limit the result list when the users selects a range, all the SearXNG
ranges (*day*, *week*, *month*, *year*) are mapped to *year*. If no range
is set an empty dictionary of arguments is returned.

Example; when user selects a time range and we find ourselves in the year
2025 (current year minus one):

```
{ "as_ylo" : 2024 }
```

searx.engines.google_scholar.detect_google_captcha(dom: ElementBase | _Element)[source] : In case of CAPTCHA Google Scholar open its own not a Robot dialog and is not redirected to sorry.google.com.

searx.engines.google_scholar.parse_gs_a(text: str | None) → tuple[list[str], str, str, datetime | None][source] : Parse the text written in green.

Possible formats:
\* “{authors} - {journal}, {year} - {publisher}”
\* “{authors} - {year} - {publisher}”
\* “{authors} - {publisher}”

Made with Furo

💾 🌐