Search · Metamapper

Search is an integral part of Metamapper. Once your datastore schema is crawled and saved in the metastore, it should also be indexed via some sort of search engine. Once indexed, your users will be able to discover data assets with simple search queries.

Configuring the Backend

You can configure the search backend by referencing the full Python class path in the METAMAPPER_SEARCH_BACKEND environment variable.

By default, Metamapper uses Elasticsearch for powering search of data assets.

Right now Metapper doesn't require any special features of Elasticsearch so your own container instance or managed service will work just fine.

You can set the Elasticsearch instance URL using the METAMAPPER_ELASTIC_URL environment variable. Connection specific arguments are provided by overriding ELASTIC_CLIENT_KWARGS in settings.py. For Metamapper and Elasticsearch containers running with docker-compose it would look something like this:

METAMAPPER_ELASTIC_URL='http://elastic:9200'

View the source for the default ElasticBackend that ships with Metamapper.

Rolling Your Own Search Backend

You can roll out your own search backend as long as the module and class can be imported by Metamapper via string. It must conform to the BaseSearchBackend abstract base class. The search method must return a Django queryset or an iterable of items that support __getitem__ (such as a list of dictionaires) with the following fields:

Field	Description
`pk`	The primary key of the object to return.
`model_name`	The string representation of the Django model associated with this object. Can be `Table`, `Column`, or `Comment`.
`score`	Search relevancy score. Used for sorting the results.
`datastore_id`	The primary key of the datastore that the object belongs to.

For example, if you installed Metamapper using the suggested Docker setup, you could place a new search backend class in the contrib folder:

# filepath: ./metamapper-setup/metamapper/contrib/custom_search_backend.py

from app.omnisearch.backends.base_search_backend import BaseSearchBackend


class MyCustomSearchBackend(BaseSearchBackend):
    def search(self, search_query_string, **extra_filters):
        return my_super_custom_search_implementation(search_query_string, **extra_filters)

And import the custom search backend using the METAMAPPER_SEARCH_BACKEND environment variable:

METAMAPPER_SEARCH_BACKEND='metamapper.contrib.custom_search_backend.MyCustomSearchBackend'

And, remember, if necessary, you can also add additional tasks and other management commands to keep your new search backend in sync with Metamapper.