Scripts to update static data in git://searx/data/


This script saves Ahmia’s blacklist for onion sites.

Output file: git://searx/data/ahmia_blacklist.txt (CI Update data …).


Fetch currencies from git://searx/engines/ engine.

Output file: git://searx/data/currencies.json (CI Update data …).


Fetch website description from websites and from git://searx/engines/ engine.

Output file: git://searx/data/engine_descriptions.json.


From descriptions[engine][language] = [description, source] To

  • output[language][engine] = description_and_source

  • description_and_source can be:
    • [description, source]

    • description (if source = “wikipedia”)

    • [f”engine:lang”, “ref”] (reference to another existing description)


Update git://searx/data/external_bangs.json using the duckduckgo bangs from BANGS_URL.


Minimize the number of nodes

A -> B -> C

  • B is child of A

  • C is child of B

If there are no C equals to <LEAF_KEY>, then each C are merged into A. For example (5 nodes):

d -> d -> g -> <LEAF_KEY> (ddg)
  -> i -> g -> <LEAF_KEY> (dig)

becomes (3 nodes):

d -> dg -> <LEAF_KEY>
  -> ig -> <LEAF_KEY>
searxng_extra.update.update_external_bangs.BANGS_URL = ''

JSON file which contains the bangs.


Fetch firefox useragent signatures

Output file: git://searx/data/useragents.json (CI Update data …).


Update searx.enginelib.traits.EngineTraitsMap and git://searx/


Persistence of engines traits, fetched from the engines.


Is generated from intersecting each engine’s supported traits.

The script git://searxng_extra/update/ is called in the CI Update data …

class searxng_extra.update.update_engine_traits.UnicodeEscape[source]

Escape unicode string in pprint.pformat


Fetchs supported languages for each engine and writes json file with those.

searxng_extra.update.update_engine_traits.filter_locales(traits_map: EngineTraitsMap)[source]

Filter language & region tags by a threshold.

searxng_extra.update.update_engine_traits.get_unicode_flag(locale: Locale)[source]

Determine a unicode flag (emoji) that fits to the locale


Fetch OSM keys and tags.

To get the i18n names, the scripts uses Wikidata Query Service instead of for example OSM tags API (side note: the actual change log from might be useful to normalize OSM tags).

Output file: git://searx/data/osm_keys_tags (CI Update data …).


Wikidata SPARQL query that returns type-categories and types. The returned tag is Tag:{category}={type} (see get_tags()). Example:


Wikidata SPARQL query that returns keys. Example with “payment”:

rdfs:label get all the labels without language selection (as opposed to SERVICE wikibase:label).


Update pygments style

Call this script after each upgrade of pygments

class searxng_extra.update.update_pygments.Formatter(**options)[source]


Update locale names in git://searx/data/locales.json used by Locales

searxng_extra.update.update_locales.get_locale_descr(locale: Locale, tr_locale)[source]

Get locale name e.g. ‘Français - fr’ or ‘Português (Brasil) - pt-BR’

  • locale – instance of Locale

  • tr_locale – name e.g. ‘fr’ or ‘pt_BR’ (delimiter is underscore)


Fetch units from git://searx/engines/ engine.

Output file: git://searx/data/wikidata_units.json (CI Update data …).