Metamapper
  • Documentation
  • Discussion
  • Blog
  • Github

›Installation (Self-Hosted)

Metadata Management

  • Welcome
  • Getting Started
  • Schema Inspection
  • SSH Tunnels
  • Custom Properties
  • Annotations
  • Search

Datastores

  • Overview
  • AWS Athena
  • AWS Glue Data Catalog
  • Azure Synapse
  • Google BigQuery
  • Hive Metastore
  • MySQL
  • Oracle
  • PostgreSQL
  • Redshift
  • Snowflake
  • Microsoft SQL Server

Workspace Management

  • Introduction
  • Access Management
  • Single Sign-On (SSO)
  • SSO Setup: Google
  • SSO Setup: Github
  • SSO Setup: SAML2

Installation (Self-Hosted)

  • Getting Started
  • Configuring Metamapper
  • Extensions
  • Asynchronous Workers
  • Email Configuration
  • File Storage
  • Security
  • Search
  • Healthchecks

Configuring Metamapper

We understand the need to customize certain aspects of Metamapper for your own purposes. This section covers how to handle overriding the default settings.

Environment Variables

Many aspects of Metamapper are configured via environment variables. These are read into the settings module, which flows through to the rest of the application.

The following is a list of settings and what they control:

Environment VariableDescriptionDefault Value
ENVIRONMENTdevelopment
METAMAPPER_SECRET_KEYUsed for cryptographic signing, such as JWT secret.None
METAMAPPER_FERNET_KEYUsed for encrypting sensitive information. Can be a comma-delimited list.None
METAMAPPER_WEBSERVER_ORIGINBase address of your Metamapper instance.http://localhost:5000
METAMAPPER_GRAPHQL_ORIGINhttp://localhost:5000
METAMAPPER_AUTHENTICATION_MIDDLEWAREdjango.contrib.auth.middleware.AuthenticationMiddleware
METAMAPPER_BEACONSend anonymous, aggregated usage data to the Metamapper teamTrue
METAMAPPER_CACHEOPS_REDIS_URLNone
METAMAPPER_CELERY_BROKER_URLSee Asynchronous Workers section.redis://redis:6379/0
METAMAPPER_CELERY_RESULT_BACKENDNone
METAMAPPER_DB_HOSTdatabase
METAMAPPER_DB_NAMEmetamapper
METAMAPPER_DB_PASSWORDpostgres
METAMAPPER_DB_PORT5432
METAMAPPER_DB_USERpostgres
METAMAPPER_EMAIL_BACKENDSee Email Notifications section.django.core.mail.backends.console.EmailBackend
METAMAPPER_EMAIL_HOSTlocalhost
METAMAPPER_EMAIL_PASSWORDNone
METAMAPPER_EMAIL_PORT25
METAMAPPER_EMAIL_USE_SSLFalse
METAMAPPER_EMAIL_USE_TLSFalse
METAMAPPER_EMAIL_USERNone
METAMAPPER_GITHUB_CLIENT_IDNone
METAMAPPER_GITHUB_CLIENT_SECRETNone
METAMAPPER_GOOGLE_CLIENT_IDNone
METAMAPPER_GOOGLE_CLIENT_SECRETNone
METAMAPPER_INCLUDE_EXAMPLE_DATASTORESLoad sample data on bootupFalse
METAMAPPER_SEARCH_BACKENDSee Search section.app.omnisearch.backends.elastic_backend.ElasticBackend
METAMAPPER_FILE_STORAGE_BACKENDSee File Storage section.django.core.files.storage.FileSystemStorage
METAMAPPER_FILE_STORAGE_BUCKET_ACLprivate
METAMAPPER_MEDIA_ROOTuploads/
AWS_ACCESS_KEY_IDNone
AWS_SECRET_ACCESS_KEYNone

Usage Statistics

Metamapper is a open source project. Our team depends heavily on community feedback to drive our roadmap.

Additionally, Metamapper collects anonymous usage statistics for installations that have set ENVIRONMENT to production or staging. This can be disabled in the Workspace Settings panel on a per-workspace basis.

If you want to disable usage statistics collection for your entire Metamapper installation, you can set the following environment variable:

METAMAPPER_BEACON=False

Advanced Configuration

If you are using the recommended bootstrap for Metamapper, you have granular controls over your configuration via Python modules. We accomplish this by overwriting the default configuration files with ones that you provide.

For example, Metamapper defaults to a very simple gunicorn configuration to manage web requests. The Docker setup exposes the gunicorn configuration as a Python module, which let's you customize settings such as timeouts, worker class type, and much more.

Configuration overrides are placed in the conf directory.

Celery

Environment Variable: METAMAPPER_CELERY_CONFIG_MODULE

Default: metamapper.conf.celery

The bootstrap provides a default Celery configuration to get you started. You can manually override this module by setting the METAMAPPER_CELERY_CONFIG_MODULE to any importable module.

For example, if the following file (that you created) is accessible via the PYTHONPATH:

# metamapper/conf/celery_overrides.py

broker_url = 'amqp://guest:guest@rabbitmq:5672'

broker_connection_timeout = 10.0

accept_content = ['application/json']

result_serializer = 'json'

task_serializer = 'json'

task_default_queue = 'priority'

task_default_rate_limit = '1000/s'

You could update your environment with the following variable:

METAMAPPER_CELERY_CONFIG_MODULE='metamapper.conf.celery_overrides'

Celery will now use RabbitMQ as a broker and only consume from the priority task queue. This can be especially powerful when you need to scale and customize your workers.

IMPORTANT: Using this setting will completely overwrite your Celery configuration. No default values will be retained.

Gunicorn

Environment Variable: METAMAPPER_GUNICORN_CONFIG_PATH

Default: ./metamapper/conf/gunicorn.py

You can override the default gunicorn configuration in a similar way. The only difference is that you just need to provide a path to a Python file. It does not need to be an importable module.

METAMAPPER_GUNICORN_CONFIG_PATH='/opt/conf/custom_gunicorn.py'

You can read more about configuring Gunicorn via it's documentation.

IMPORTANT: Using this setting will completely overwrite your Gunicorn configuration. No default values will be retained.

Django

Environment Variable: METAMAPPER_SETTINGS_OVERRIDE_MODULE

Default: None

Metamapper handles a serious amount of configuration using the django.conf.settings module. You can override virtually any setting this file by referencing an importable module using the METAMAPPER_SETTINGS_OVERRIDE_MODULE environment variable.

Unlike the other overrides mentioned above, this strategy will only override settings that you provide.

For example, if the following file (that you created) is accessible via the PYTHONPATH:

# metamapper/conf/settings_overrides.py

TIME_ZONE = 'America/Los_Angeles'

You could update your environment with the following variable:

METAMAPPER_SETTINGS_OVERRIDE_MODULE='metamapper.conf.settings_overrides'

This would override the default timezone for your Metamapper instance:

>>> from django.conf import settings
>>> settings.TIME_ZONE
'America/Los_Angeles'

Please note that overriding many of the default settings of Metamapper is not advised. Please make sure you know what you are changing before shipping to production, as it could have unintended consequences.

Last updated on 9/22/2020
← Getting StartedExtensions →
  • Environment Variables
  • Usage Statistics
  • Advanced Configuration
    • Celery
    • Gunicorn
    • Django
Metamapper
Documentation
User GuideInstallation Guide
Community
DiscussionGitHub
Copyright © 2020 Scott Cruwys