Skip to main content
waffle.svg
Domo Knowledge Base

Snowflake Internal Unload Advanced Partition Connector

Version 3

 

Intro

Snowflake is a cloud-based analytic data warehouse system. Use Domo's Snowflake Internal Unload Advanced Partition connector to unload data from your Snowflake database into internal Amazon S3. The files are then downloaded from internal S3 into Domo. This connector should be preferred over the other Snowflake connectors if you are executing very large queries. It allows you to chose whether or not you want to partition your data. Unlike other partition connectors, it allows you to use partitioning on non date columns. To learn more about the Snowflake API, visit their page (https://docs.snowflake.net/manuals).

You connect to your Snowflake account in the Data Center. This topic discusses the fields and menus that are specific to the Snowflake Internal Unload Advanced Partition connector user interface. General information for adding DataSets, setting update schedules, and editing DataSet information is discussed in Adding a DataSet Using a Data Connector.

Prerequisites

To connect to your Snowflake account and create a DataSet, you must have the following:

  • The full name of your account (provided by Snowflake). For US West Region account_name, and All other regions account_name.region_id. This is the portion of your Snowflake URL immediately following https://.

    Note that your full account name may include additional segments that identify the region and cloud platform where your account is hosted. The following table provides account name examples by cloud platform/region. For each row, it assumes your account name is "xy12345."

    snowflake_account_names.png 

  • The username and password used to connect to your Snowflake host.

Connecting to Your Snowflake Account

This section enumerates the options in the Credentials and Details panes in the Snowflake Internal Unload Advanced Partition Connector page. The components of the other panes in this page, Scheduling and Name & Describe Your DataSet, are universal across most connector types and are discussed in greater length in Adding a DataSet Using a Data Connector.

Credentials Pane

This pane contains fields for entering credentials to connect to your Snowflake account. The following table describes what is needed for each field:  

Field

Description

Account Name Enter your Snowflake account name. For more information, see Prerequisites
Username Enter the username you use to connect to your Snowflake host.
Password Enter the password you use to connect to your Snowflake host.
Role Enter your role.

Once you have entered valid Snowflake credentials, you can use the same account any time you go to create a new Snowflake Internal Unload Advanced Partition DataSet. You can manage connector accounts in the Accounts tab in the Data Center. For more information about this tab, see Managing User Accounts for Connectors.

Details Pane

This pane contains a primary Reports menu, along with various other menus which may or may not appear depending on the report type you select.

Menu

Description

Report

Select the Query Type. The following query types are available:

Custom Query Enter the SQL query to execute.
Query Builder Select a table and fields to auto generate your query.
Warehouses Select the warehouse containing the data you want to retrieve. The list populates with all of the warehouses you have access to based on your credentials.  
Databases Select the database containing the data you want to retrieve. The list populates with all of the databases you have access to based on your credentials.
Schemas Select the database schema containing the data you want to retrieve. The list populates with all of the schemas you have access to based on your credentials.
Database Objects Select the database object.
Tables Select the database table you want to retrieve. The list populates with all of the tables you have access to based on your credentials. If you do not select a database table, the connector retrieves your entire database schema.
Table Columns Select the table columns you want to retrieve. The list populates with all of the columns you have access to based on your credentials. If you do not select a table column, the connector retrieves your entire selected database table.
Table Query Helper This field shows the query that is generated based on the selected fields.
Views Select the view.
View Columns Select the view columns that you would like to include in your report.
View Query Helper This field shows the query that is generated based on the selected fields.
Query

Enter the SQL query to use to retrieve your data. You can use the Query Helper parameter to help you write a usable SQL query. To use the Query Helper, do the following:

  1. Select your desired warehouse, database, database schema, database table, and table columns in each menu.

  2. Copy the SQL statement that appears in the Query Helper field.

  3. Paste the copied SQL statement into the Query field.

Partition Support Choose whether or not to partition your data.
Partition Criteria Select whether you want to partition the data using the date keys, non date keys, or meta query.
Partition Query to determine partition tags

Enter the partition meta query to determine the distinct partition tags (or keys). The column containing the Date data is your partition column.

Example: SELECT DISTINCT(DATE_COLUMN_NAME) FROM TABLENAME

Partition Support Format Select the partition support format.
Partition Column Name Select the partition column name.
Past Year Enter the number of past years that you want to get data for. Value can be X, where X is a positive integer. For example: 5.
Future Years Enter the number of future years that you want to get data for. Value can be X, where X is a positive integer. For example:5.
Past Months Enter the number of past months that you want to get data for. Value can be X, where X is a positive integer. For example:5
Future Months Enter the number of future months that you want to get data for. Value can be X, where X is a positive integer. For example:5.
Past Days Enter the number of past days that you want to get data for. Value can be X, where X is a positive integer. For example: 30.
Date Format Select the required date format. By default yyyy-MM-dd will be used.
Custom Date Format Enter the custom date format.
Partition Non Date Key Column Name Select the partition column name.

Other Panes

For information about the remaining sections of the connector interface, including how to configure scheduling, retry, and update options, see Adding a DataSet Using a Data Connector.

FAQs

What kind of credentials do I need to power up this connector?

You need the full name of your Snowflake account, and the username and password associated with your Snowflake host. You may also provide your role.

Are there any API limits that I need to be aware of?

Limits depend on your server configuration.

How frequently will my data update?

Datasets can run as often as every 15 minutes. However, depending on the runtime of the query, datasets may need to run less frequently.

What do I need to be aware of while writing a query?

Make sure that all the words, table names and field names are correctly spelled. Refer to the Query Helper field for query help. 

Why can't I connect to my Snowflake database? Do I need to whitelist any IP addresses?

Before you can connect to a Snowflake database, you must also whitelist a number of IP addresses on your database server on the port you want to connect to. For the full list of IP addresses, see Whitelisting IP Addresses.

What's the difference between the Snowflake Internal Unload Advanced Partition and Snowflake Managed Unload connectors?

Both connectors support partitioning, but the Snowflake Internal Unload Advanced Partition connector allows to select whether or not you want to use partitioning for your data. Also, unlike the other partition connectors, the Snowflake Internal Unload Advanced Partition connector allows you to use partitioning on non date columns.