Create a new Data Pipeline recipe

AVAILABILITY

Data Pipeline recipes are available to workspaces in the private preview program only.

Contact your Workato representative to add Data Pipeline to your workspace.

Configure the Workato Data Pipeline source

The steps in this section use Salesforce as the example source.

1

Go to the project you plan to use for your Data Pipeline.

2

Select Create > Data pipeline.

3

Provide a name for your pipeline in the Name field and select the project in which you plan to store the pipeline from the Location drop-down menu.

4

Select Start building.

5

Select Extract new/updated records from source app to add a source application. Select Salesforce from Your connected source apps.

Choose a source app shows applications that are already connected.

6

Choose the Salesforce connection you plan to use for this pipeline:

Choose a Salesforce connection

7

Select the Sync objects you plan to use in your pipeline after the connection setup screen loads the objects available in the connection.

Select objects to sync

Note that each object has an associated schema. When you select the object to sync, your Workato Data Pipeline recipe fetches both the data and the associated schema of the object implicitly.

8

Select Autosync new fields if you plan to automatically sync new fields as they're created in the source. Otherwise, select Block new fields.

Pipelines can automatically handle schema changes, also called schema drift. Visit schema drift on the Concepts page for more information about how Workato treats schema drift in a pipeline.

9

Select Sync frequency to configure how often data should be synced to the destination.

Configure sync frequency

• Time unit

• Choose between minutes, hours, or days.

• Trigger every

• Enter the interval with at least 30 minutes between syncs. For example, if you choose hours as a time unit and enter 12 here, the sync occurs every twelve hours.

10

Use the calendar modal to select the date from which your pipeline should begin to sync records in the When first started, this pipeline should pick up records from field.

This value cannot be changed after you run the pipeline.

Learn more about this input field.

11

Select Save to save your progress before you continue to the next section.

Configure how data is loaded in the Workato Data Pipeline

The steps in this section use Snowflake as the example destination.

1

Select Load data to target table in destination app to add the destination. Select Snowflake from Choose a destination app.

2

Choose the Snowflake connection you plan to use for this pipeline:

Choose a Snowflake connection

3

If the database schema defined in the connection is the same as the destination schema, continue to the next step.

1

Toggle Use a different schema than connection to select a new schema from the destination or to create a new schema.

2

Choose an existing database schema in the Database schema drop-down, or select Create schema and name the new schema.

Use a different schema than connection

4

Select Save to save the pipeline.

5

The status section shows Inactive pipeline and provides options to Move the location of your pipeline or Edit your pipeline.

Select Start pipeline to start the data pipeline. After you start the pipeline, the page shows tabs for Object syncs, Connections, and Settings. Note that the pipeline loads historical data on its first run.

6

Select Object syncs to monitor the status of each object sync.

Pipeline page shows new tabs