Scripts to update static data in git://searx/data/


This script saves Ahmia’s blacklist for onion sites.

Output file: git://searx/data/ahmia_blacklist.txt (CI Update data …).


Fetch currencies from git://searx/engines/ engine.

Output file: git://searx/data/currencies.json (CI Update data …).


Fetch website description from websites and from git://searx/engines/ engine.

Output file: git://searx/data/engine_descriptions.json.


From descriptions[engine][language] = [description, source] To

  • output[language][engine] = description_and_source

  • description_and_source can be:
    • [description, source]

    • description (if source = “wikipedia”)

    • [f”engine:lang”, “ref”] (reference to another existing description)


Update git://searx/data/external_bangs.json using the duckduckgo bangs (CI Update data …). loads:

This script loads the javascript, then the bangs.

The javascript URL may change in the future ( for example ), but most probably it will requires to update RE_BANG_VERSION


Minimize the number of nodes

A -> B -> C

  • B is child of A

  • C is child of B

If there are no C equals to <LEAF_KEY>, then each C are merged into A. For example (5 nodes):

d -> d -> g -> <LEAF_KEY> (ddg)
  -> i -> g -> <LEAF_KEY> (dig)

becomes (3 noodes):

d -> dg -> <LEAF_KEY>
  -> ig -> <LEAF_KEY>


Fetch firefox useragent signatures

Output file: git://searx/data/useragents.json (CI Update data …).


This script generates from intersecting each engine’s supported languages.

Output files: git://searx/data/engines_languages.json and git://searx/ (CI Update data …).

class searxng_extra.update.update_languages.UnicodeEscape[source]

Escape unicode string in pprint.pformat


Determine a unicode flag (emoji) that fits to the lang_code


Fetch OSM keys and tags.

To get the i18n names, the scripts uses Wikidata Query Service instead of for example OSM tags API (sidenote: the actual change log from might be useful to normalize OSM tags).

Output file: git://searx/data/osm_keys_tags (CI Update data …).


Wikidata SPARQL query that returns type-categories and types. The returned tag is Tag:{category}={type} (see get_tags()). Example:


Wikidata SPARQL query that returns keys. Example with “payment”:

rdfs:label get all the labels without language selection (as opposed to SERVICE wikibase:label).


Update pygments style

Call this script after each upgrade of pygments


Fetch units from git://searx/engines/ engine.

Output file: git://searx/data/wikidata_units.json (CI Update data …).