Skip to main content
waffle.svg
Domo Knowledge Base

Workbench 5 Beta

Version 5

 

Intro

Domo Workbench is a secure, client-side solution for uploading your on-premise data to Domo.

Use Workbench 5 to routinely upload data from external DataSets (ODBC, CSV, Microsoft Excel, Jira, QuickBooks, XML, and JSON) to Domo. You can also secure network communication using a proxy server, manage scheduled jobs, add columns based on calculations, transform columns using regular expressions. Domo Workbench 5 enables you to securely manage schema changes, take over a Connector DataSet so it can be maintained further by the Workbench, and perform bulk actions on your jobs.

This tutorial teaches you how to create and run a job for your DataSet using Domo Workbench 5.

What You Need to Get Started

  • Domo Workbench
  • To have the Workbench 5 Beta enabled in your Tool Downloads, send an email to beta.admin@domo.com.
Note: Beta features should not be used in mission-critical production processes as issues may still exist that might not be fixed immediately.

What can Domo Workbench 5 do?

  • Securely upload DataSets to Domo using helpful interface components.
  • Upload data manually or on a schedule.
  • Create groups of DataSet jobs that run on the same schedule.
  • Import and export job files.
  • Preview and validate jobs before you run them.
  • Add, create, save, and delete accounts.
  • Add, create, save, delete, open, preview, and cancel jobs.
  • Revert to a previous version of a job you saved.
  • Add/delete transforms to/from jobs, which includes the following:
    • Add new columns.
    • Change date format.
    • Encrypt sensitive data in specific columns.
    • Filter columns in the DataSet.
    • Set the time zone for the DataSet job.
    • Replace column values in the DataSet.
    • Combine the DataSet with another DataSet.
  • Export/import Excel, CSV, QuickBooks, OLAP, ODBC, JSON, and XML file types.
  • View logs.
  • Send logs as a zip file.
  • Run logs from the command line.
  • Add and manage plugins for using Workbench.
  • Set up notifications for DataSet jobs that can email you if DataSets are uploaded successfully or if it experiences an error.

Creating a Job for Uploading Data Using Workbench 5

You can upload data from external data sources to Domo after specifying connection and data information as a job, which you can save and use again.
Creating a DataSet job in Workbench 5:

  1. In the Jobs pane, click the + icon to create a new job.

    1.png
     
  2. Specify the basic settings for the job, such as the DataSet job name, the method of data transport (such as Local File Provider, ODBC Connection Provider, etc.), the data reader (data source) type (such as JIRA, Excel, etc.), the DataSet name and type, and an optional DataSet description.
     
    Note: The DataSet name is the only setting you can change after creating the job. If you want to change any other settings, you will need to create a new job.
  3. After providing the details, click Create. A newly added job entry will appear in your jobs listing. However, at this stage, the DataSet Job is still not ready for execution.

Configuring and Scheduling your job

  1. In the Configure subtab for the job:
    1. Provide the source of the data in the Source menu. For example, if the data source was a local file, you would select the file path here.

      2.png
       
    2. You can choose whether you want newly uploaded data to REPLACE the existing data in the DataSet or be APPENDED (added) to the existing data. You do this in the Update method menu.
    3. You can specify the error handling method, and the job run duration and conditions in the Advanced Settings. This section provides the advanced options for how your job uploads data in Domo.
  2. You can schedule a DataSet job to routinely upload data from external DataSets to Domo. You can select a Basic Schedule (update manually, update when file is changed, or update on a schedule.) Or, you can opt for Advanced Scheduling (once a day, specific days of the week/month, etc.)
  3. Click the Save button to save your changes. 

Known Beta Bugs and Improvements:

  • Taking over a Connector DataSet: There will be a message in the UI indicating you need to be an Editor or Owner of the DataSet in order to take over the DataSet using Workbench.
  • Taking over a Connector DataSet: If you are not the owner/editor of a DataSet, after selecting a DataSet there will be a message indicating that you do not have the correct permissions to take over the DataSet.
  • Schema Protection: Small window sizes sometimes prevent the bottom row of the schema from appearing in the Schema tab. Making the screen bigger will fix this problem. This will be fixed in a future iteration.
  • ODBC: With this release the user will only be able to use the x86 version of the MSSQL driver. In a future release the x64 version will be available as well.

Schema Protections for your job

To further improve the robustness of Data Pipelines, Domo has update schema protections in Workbench. With these improvements you can choose columns you want to protect and unprotect from schema changes, and choose between three different options about how you want the job to behave.

Video - Introduction to Schema Improvements in Workbench 5

Schema Protection Types

Domo Workbench 5 offers three different schema protection types. 

  • Allow schema changes safely.
  • Permit unprotected schema changes.
  • Completely block all schema changes.

Video - Deep Dive on the Three Schema Options in Workbench 5

Allow safe schema changes

If you select this option, the unprotected columns will accept the changes during the next execution, and will automatically be re-protected afterwards.

Note: They are only re-protected if schema changes. If the schema does not change, the columns will remain unprotected until it changes.

New columns will be added, and will automatically be protected in future runs. The “Allow safe schema changes” option is the default setting for new jobs.

Allow unprotected schema changes

If you choose to allow unprotected schema changes, columns will remain unprotected after the job is executed and any new columns added will automatically be added as unprotected columns.

Block all schema changes

If you choose to block all schema changes, you will not see the Protect option for your columns. If you attempt to make changes to any column and run the job, you will receive an error stating about your forceful change.

3.png

Protect column

The Protect column indicates whether your column (data field) is protected or not when schema changes happen to your job.

  • If the Protect checkbox is selected for a column, it means that column is protected, and no schema changes can be made. If you try to remove the protection for any column (by unchecking it) the Domo Workbench will give you a warning. Unchecking the Protect column checkbox will allow schema changes to be made for that specific column.
Important: Unprotected columns can be changed and deleted, which may break content in Domo.

4.png

  • The Protect column provides you a control to protect or unprotect all columns with a single click. Select the checkbox next to the column header (Protect) to protect all your columns at once. Similarly, you can unprotect your columns at the same time by deselecting this checkbox.

    5.png 6.png
     
  • The column header also provides the control to get a filtered view of protected, unprotected, or both type of entries with a single click.
    • When you click to select the regular select view 7.png, the Protect column will display all protected entries.
    • When you click to select the unchecked view 8.png, the Protect column will display the unprotected entries.
    • When you click to select the mixed view 9.png, the Protect column displays both protected and unprotected entries together.
  • You can change the names or properties for the Destination Column and the Data Type column, but in order for these changes to take effect, the column will need to be unprotected.

    10.png
     
  • Similarly, you cannot delete a protected column.
  • If you want to allow the schema change, uncheck the column and re-run the job.

Viewing the Error messages for your job run

When a job errors due to a schema change, Workbench offers error message and color-coding in the schema tab to help you understand what changed in your schema. These error messages will help you to find the changes so you can decide whether to allow the schema change to happen by unprotecting the column, or to find the error in your data and fix it before re-running the job in Workbench.

Video - Troubleshooting Changing a Data Type in Workbench 5

Video - Troubleshooting Column Name Changes in Workbench 5

Once you make changes to your schemas and run the job, the Workbench will show a notification in your Schema tab, and the changed columns will appear in the designated color codes.

11.png

If any errors occurred during the job run, you can view these errors in the Overview Tab, Preview window, and Logs.

Overview Tab

Click on the Overview Tab to view the errors. Here, you can see execution status and the error messages received for your job run.

12.png

Preview Window

Click on the Preview 13.png button in your job view.

14.png

The output window will appear showing the preview for errors for your job.

15.png

Log viewer

  • You can view a detailed log for your job run in the History Tab. This log displays the errors in detail as well.
  • Click on the History Tab.
  • Here, you will see options for viewing the job’s history from the DataSet and local logs for this job.
  • Click the Open Log Viewer button.

    16.png

The Domo Workbench Log Viewer window will appear displaying the detailed log with error messages occurred.

17.png

Protecting Data Types while changing Column Names

Changing the name of a column in the Data Source

If the name of the column is changed in the data source, the column is unprotected and the Data Type is locked, Workbench will remove the column with a changed name and add it as a new column with the new name. At this point, the column will be reindexed with the data type present in the column at the time it is reindexed.

Changing the name of a column in the Destination Column of Workbench

If you want to change the name of a column in Domo while protecting the data type from changing, it is possible with the new schema protection changes in Workbench.

Video - Protecting Data Types while Changing column Names in Workbench 5

In this example, you want to update the Warranty claims column name in the Destination Column, and the column has been unprotected. The lock data type column has been selected.

18.png

The column has been updated with a new Destination column name “2020 Warranty claims”.

19.png

In addition to Lock Data Type being selected, the error handling options you have chosen in the configuration tab will play a role in how Workbench responds to this configuration.

20.png

If Abort job on invalid data error is the selected option for this job and invalid data is introduced, the job will fail because the data type is locked, even though the column is unprotected.

If Skip entire row if it contains invalid data is selected for this job and invalid data is introduced, the job will succeed because the row of data will be skipped and not included in the data sent to Domo. The messaging included in the execution logs will indicate a row has been skipped, and the Data Type will remain the same:

21.png

If Replace invalid cells with NULL value is selected the job will execute successfully and invalid values will be replaced with NULL, resulting is this messaging in the execution logs. The Data Type will remain unchanged:

22.png

Taking over a Connector DataSet

Domo Workbench allows you to import the Workbench DataSet as well as the Connector DataSet. Once you select your DataSet, it will be further maintained by the Workbench.

Note: You need to have Editor or Owner permissions on the DataSet to successfully take it over using the Domo Workbench.

Video - Taking Over a Connector DataSet in Workbench 5

To select your DataSet:

  1. Visit the Overview tab of your job, and click the Browse button next to your DataSet name. Note that, you can also change your DataSet name here if you wish.

    23.png
     
  2. The DataSet picker window appears showing the available Workbench DataSets and Connector DataSets.
  3. Select the Connector DataSet tab.
  4. Here, select your connector DataSet, and click Select.

    24.png
     
  5. As you chose to update the Connector DataSet through Workbench, you will receive a warning as below saying that the Workbench will take over the connector schedule and stop further updates from the Connector for this DataSet.

    25.png
     
  6. Click Continue. You can click Cancel if you do not wish the Workbench to take over the connector schedule for the selected DataSet.
  7. Now, in the Overview tab, you can see your DataSet Name updated to the selected connector DataSet name.
  8. For this change to take effect, save the job.

    26.png

Bulk Actions on your DataSet jobs

As you create multiple DataSets in Workbench, you can perform certain actions in group on your jobs like execute, preview, export, bulk edit, save, revert, and delete.

27.png

In order to perform these bulk operations on your jobs, you can select all jobs, or selective jobs that you intend to make changes to.

Video - Bulk Actions in Workbench 5

Execute

  1. Select the desired jobs from your job list and perform a mouse right-click operation.
  2. Click on Execute. The Bulk Edit wizard will appear showing your bulk change request details.
  3. Confirm your request and click Continue.

    28.png
     
  4. The progress bar will appear showing the job execution progress, and the output will be displayed in the output window.

    29.png
     
  5. Click Finish.

Preview

  1. You can preview your multiple jobs together using the Preview option.
  2. Select the desired jobs from your job list and perform a mouse right-click operation.
  3. Click on Preview.
  4. Select continue to execute the Preview.

Export

  1. You can export your multiple jobs together to a local backup using this Export option.
  2. Select the desired jobs from your job list and perform a mouse right-click operation.
  3. Click on Export.
  4. Export jobs to a local backup window will appear.
  5. Click the … (3dots) and select the local path to save your jobs at and click Export Jobs.

    30.png
     
  6. A success message will appear as your jobs are exported to the selected location.

Bulk Edit

  1. You can make changes to your job settings, job notifications, provider settings, and reader settings for multiple jobs at a time.
  2. Select the desired jobs from your job list and perform a mouse right-click operation.
  3. Click on Bulk edit.
  4. Bulk Edit wizard will appear displaying the choices of changes you can make on jobs together.
  5. Select the desired option, and click Continue.

    31.png
     
  6. Make the desired changes accordingly, and click Continue.
  7. Click Back to choose another option. Click Cancel to cancel the operation.
  8. You can choose to save the jobs automatically after making the bulk changes.
  9. The progress bar will appear showing the job execution progress.
  10. Click Finish.
  11. Your job changes will reflect in the job list.

Clearing a field using the Bulk Editor

You can use the bulk editor to clear a field from multiple jobs at the same time. You need to select the Edit checkbox 36.png and leave the field blank next to it, that you want to clear.

Let’s take an example where we try to clear the Source File Path provided for our data.

  1. Select the jobs from the job list that you want to make bulk changes to, and perform the mouse right-click operation.
  2. Click on Bulk edit.
  3. The Bulk Edit wizard will appear displaying the choices of changes you can make on jobs together. Here, select Edit provider settings and click Continue.

    32.png
     
  4. Now, the Bulk Edit wizard will display the fields that you can clear for the selected jobs. Select the Edit checkbox 36.png and leave the File Path field blank next to it (as we want to clear the file path that was selected while configuring the job.) and click Continue.

    33.png
     
  5. The Bulk Edit wizard will ask you for the change confirmation. Click Continue.

    34.png
     
  6. The progress bar will appear showing the progress. Click Finish.
  7. Now, visit the Configure Tab of the jobs that you made changes to. You will see that the Source file path that you had selected earlier while configuring the job, doesn’t show up there anymore.

    35.png

Thus, you can clear different fields for multiple jobs using the Bulk Edit feature.

Save

  1. You can save the changes made to your multiple jobs together using the Save option.
  2. Select the desired jobs from your job list and perform a mouse right-click operation.
  3. Click on Save.
  4. Select Continue to Save.

Revert

  1. You can revert the changes made to your multiple jobs together using the Revert option.
  2. Select the desired jobs from your job list and perform a mouse right-click operation.
  3. Click on Revert.
  4. Select Continue to Revert.

Delete

  1. You can delete your multiple jobs together using the Delete option.
  2. Select the desired jobs from your job list and perform a mouse right-click operation.
  3. Click on Delete.
  4. Select Continue to Delete.

FAQ’s

If my job was set to allow schema changes in the old UI, what are the settings with the new schema improvements?