Google BigQuery
Metamapper supports Google BigQuery as a connection source. BigQuery is a serverless, highly scalable, and cost-effective multi-cloud data warehouse designed for business agility.
This guide details how to setup and troubleshoot BigQuery in Metamapper.
Prequisites
An existing Google Cloud Platform (GCP) project. It must have billing enabled and an attached credit card.
An existing BigQuery instance in the GCP project. Metamapper supports a single BigQuery instance per connection.
Permissions in the GCP project that allow you to create Identity Access Management (IAM) service accounts. Metamapper uses an IAM service account to authenticate, which we will create later in this guide.
Create a GCP IAM service account
Navigate to the IAM Service Accounts page in the GCP console. Select the project that you want to use to access your BigQuery instance.
Next, click on Create Service Account. This is how we will generate the credentials you will input during the Metamapper datastore setup process.
When you create te service account, you will need to grant the account BigQuery Metadata Viewer permissions. Metamapper uses this to hit the BigQuery metadata API during the schema introspection process.
The last step is to create and download a JSON project key. The project key file contains information about the project, which Metamapper uses to authenticate during introspection.
Save the JSON project key file to your computer. You will need to copy and paste the raw JSON into the Credentials field when you set up your datastore connection in Metamapper.
Questions? Feedback?
Did this article help? If you have questions or feedback, feel free to submit a pull request with your recommendations, open a Github issue, or join the discussion on Discord.