Skip to main content
waffle.svg
Domo Knowledge Base

Google BigQuery Unload Connector

Version 3

 

Intro

BigQuery is a data platform to create, manage, share and query data. Domo's BigQuery Unload connector lets you bring in larger results from BigQuery through the assistance of the Google Cloud Storage service. To learn more about the BigQuery API, go to https://cloud.google.com/bigquery/docs/reference/v2/.

The Google BigQuery Unload connector is a "Cloud App" connector, meaning it retrieves data stored in the cloud. In the Data Center, you can access the connector page for this and other Cloud App connectors by clicking Cloud App in the toolbar at the top of the window.

This topic discusses the fields and menus that are specific to the Google BigQuery Unload connector user interface. For general information about adding DataSets, setting update schedules, and editing DataSet information, see Adding a DataSet Using a Data Connector.

Prerequisites

To connect to BigQuery and create a DataSet, you must have the following:

  • A JSON BigQuery service account key

  • A JSON Google Cloud Storage service account key

  • A Domo client ID

  • A Domo client secret

To generate the necessary service account keys, do the following:
  1. In the GCP Console, open the IAM & Admin page.

  2. Click Service accounts in the left-hand navigation pane.

  3. Select your project and click Open.

  4. Click Create Service Account.

  5. Enter a name and description for the service account.

  6. Click Create.

  7. Select Project Owner.

  8. Click Continue.

  9. Click Create key.

  10. Select JSON as the key type.

  11. Click Create.

A private key will be saved to your computer.

To create a Domo client ID and client secret, do the following:
  1. Navigate to https://developer.domo.com and log in.

  2. Click New Client (found under "My Account" in the top right corner).

  3. Create a new client with Application Scopes of Data and User.

  4. When you click Create, you will be redirected to the client ID and client secret.

  5. Copy the client ID and client secret, as you will need them soon.

Connecting to BigQuery

This section enumerates the options in the Credentials and Details panes in the Google BigQuery Unload Connector page. The components of the other panes in this page, Scheduling and Name & Describe Your DataSet, are universal across most connector types and are discussed in greater length in Adding a DataSet Using a Data Connector.

Credentials Pane

This pane contains fields for entering credentials to connect to your (third-party tool) account. The following table describes what is needed for each field:  

Field

Description

Service Account Key JSON BigQuery

Enter your Google BigQuery JSON service account key.

Service Account Key JSON Google Cloud Storage

Enter your Google Cloud Storage JSON service account key.

Domo Client ID

Enter your Domo client ID.

Domo Client Secret

Enter your Domo client secret.

For information about obtaining these credentials, see "Prerequsites," above.

Once you have entered valid credentials, you can use the same account any time you go to create a new Google BigQuery Unload DataSet. You can manage connector accounts in the Accounts tab in the Data Center. For more information about this tab, see Managing User Accounts for Connectors.

Details Pane

This pane contains a primary Reports menu, along with various other menus which may or may not appear depending on the report type you select.

Menu

Description

Report

Select the report you want to run. The following reports are available:

Enter Query Manually Runs a BigQuery SQL query and returns results if the query completes.
Generate Query Retrieves table data.
DataSet ID Select the BigQuery dataset id for your data.
For more info about BigQuery datasets, visit https://cloud.google.com/bigquery/docs/datasets-intro
Table ID Select the BigQuery table id for your data.
For more info about BigQuery tables, visit https://cloud.google.com/bigquery/docs/tables-intro
Query Enter the BigQuery SQL query you want to execute. Only standard SQL query is supported.
Google Cloud Storage bucket name Select the Google Cloud Storage bucket name that will be used for temporary storage as we transfer your data into Domo.
Force to string Enter a comma separated list of fields that need to be treated as STRING in Domo.
Force to number Enter a comma separated list of fields that need to be treated as NUMBER in Domo.
Upsert key columns Enter a comma separated list of upsert key column names.

Other Panes

For information about the remaining sections of the connector interface, including how to configure scheduling, retry, and update options, see Adding a DataSet Using a Data Connector.