Triggers

SUMMARY

• Triggers initiate recipe actions in Workato.
• Triggers follow in-sequence delivery, maintain processed jobs records, prevent duplicate processing, and ensure job completion in order.
• Trigger mechanisms come in three types: polling (checking regularly), real-time (immediate notifications, often through webhooks), and scheduled (execute at specific times/intervals).
• Trigger dispatches can be single (real-time data sync) or batch (improved throughput with large data volumes).

Video guide: How to set up triggers correctly

Triggers determine what event to listen to execute the actions described in a recipe.

Trigger events can be set off in apps (such as Salesforce or Jira) when a certain event happens (new contact is created or existing ticket is updated), when a new line is added in a file, or according to a schedule (fires at a certain time or interval).

Depending on the available API, Workato can receive trigger events in real-time, or check for the occurrence of an event periodically by polling the app.

Triggers can be classified into different types based on when they check for new events (trigger mechanism) and how they group new events (trigger dispatch).

Trigger mechanisms and dispatches

Trigger behavior

Workato recipes pick up and queue trigger events in-sequence to be processed as recipe jobs. The recipe maintains a cursor and progresses through the trigger event queue synchronously, with an adjustable throughput. No duplication of jobs occur as Workato maintains a record of the trigger events it has seen.

Trigger events are queued and processed by the recipe as jobs

Workato triggers have the following behavior.

• In-sequence delivery

Triggers will be delivered in chronological order, for example, oldest records will be processed first, or in the sequence they were delivered to Workato.

• Durable cursor position

Recipes remember the jobs it has processed even across stopped and running states. Whenever a recipe is started, it will pick up where it stopped and begin processing all trigger events since it was stopped.

When the recipe is stopped at 10/21/2017, 11.30 am and started again days or weeks later, it will pick up trigger events from when it stopped at 10/21/2017, 11.30 am

• No duplication

Each recipe maintains a record of the trigger events it has seen, and will not process duplicate events.

• Flow control

Recipes process trigger events synchronously, for example, only process a second job when the first job has been completed. For moving large volumes of data, you can maximize data throughput with batch processing and running multiple concurrent jobs. Learn more about Workato's recipe concurrency.

• Guaranteed delivery

For Workato polling triggers, Workato guarantees trigger event delivery. Even if servers experience temporary downtime, or if the network is unstable, Workato ensures that triggers are picked up and processed in-sequence.

Webhook events, which powers most real-time Workato triggers, inherently have the possibility of being lost. To mitigate this, most Workato-built real-time triggers (a notable exception is the HTTP webhook trigger) have a backup polling mechanism that ensures missed webhook trigger events will be picked up.

Trigger mechanisms

Trigger mechanisms determine when a trigger will fire. In this section, we cover polling triggers, real-time triggers and scheduled triggers.

Polling triggers

Polling triggers periodically check for new events by querying the app at intervals based on your Workato plan, which can be as frequent as every five minutes. Each poll may retrieve multiple events, potentially creating several jobs from a single poll.

When you first start a recipe, the polling trigger collects events from the date specified in the When first started, this recipe should pick up events from field. For instance, a recipe might fetch all NetSuite customers created or updated since January 1, 2017, at 10:00 AM PST. After the initial fetch, the recipe continues to poll at regular intervals according to your plan. For example, every five minutes, it would retrieve new NetSuite customers created or updated in the last five minutes.

When you stop the recipe, polling triggers cease to fetch new events. If you restart the recipe, polling triggers retrieve all events that occurred while the recipe was stopped.

LONG ACTIONS JOB SEQUENCE

If your recipe has a long action step and job, note that when the action is put on hold, Workato automatically disregards the job sequence. The next job begins to process even if the previous job is not finished and the recipe concurrency is set to 1.

Example: Polling trigger

A Workato account on the Business plan has a five-minute polling interval, as indicated in their polling triggers.

Trigger hint regarding the specific polling interval

For example, if a recipe is set to poll every five minutes for new Salesforce accounts, it fetches any new accounts created during each poll. Multiple newly created accounts can result in multiple jobs created from a single poll.

If you stop the recipe on February 1, 2017, at 12:00 AM PST, it no longer fetches new trigger events. When you restart the recipe on March 10, 2017, Workato retrieves all Salesforce accounts created since February 1, 2017.

Real-time triggers

Real-time triggers are usually built on top of an asynchronous notification mechanism. Real-time triggers typically require registration in your connected app (either through the API or manually through the app interface), to let the app know that you are interested in a specific event. When that event occurs, the app will send a notification to Workato and generate a trigger event.

Webhooks are one such mechanism. Most real-time triggers supported on Workato are built on webhooks. The advantage of webhooks is that there is no delay, and it is more efficient as we only receive notifications from apps when events occur, instead of Workato having to check at regular intervals for new events.

Real-time triggers supported by Workato (this excludes HTTP real-time triggers) are generally webhooks supported by regular polling. The polling intervals for real-time triggers are generally longer than the normal polling triggers, however, instead of polling once every 5 minutes or so, the trigger can now poll once every hour. The polling mechanism in real-time triggers is also what allows you to select a From date when you start your recipe.

Scheduled triggers

Scheduled triggers are executed at specified days and times, hourly, daily, monthly, and so on.

Various schedule options for Salesforce scheduled search trigger

At the scheduled time or interval, this trigger will fetch all events matching the specified criteria. Also, unlike other triggers, scheduled triggers will return events that have already been picked up previously.

Scheduled triggers will return events in batches, similar to how batch triggers work. Users can specify the maximum batch size, for example, if the batch size is set to 100 and 420 new events are identified, 5 new jobs will be created. The first four jobs will have 100 events each and the fifth will have 20 jobs.

Scheduler/clock/timer

Scheduler triggers allows you to schedule when exactly your recipe will run. There are two triggers:

Scheduler triggers

• New scheduled event

This trigger allows you to specify the time the recipe should first trigger, and subsequently, the time intervals to continue triggering on.

Basic scheduler trigger that triggers the first time when recipe is started, then subsequently every hour after that

• New scheduled event (advanced)

This trigger allows you to specify the days of the week the recipe should trigger on, as well as the times it should trigger on. If you specify only the minutes, for example, 30, the recipe will trigger 24 times in a day, every 30 minutes past the hour. If both hour and minute input fields are filled, the recipe will trigger once a day.

Advanced scheduler trigger that triggers every Sunday at midnight 0000

Trigger dispatches

Trigger dispatches determine whether a trigger returns a single event or a list of events. In this section, we cover single triggers and batch triggers.

Single triggers

Single triggers are useful for continuous real-time synchronization of data, for example, moving opportunities from Salesforce into NetSuite as sales orders the moment these opportunities close. Most Workato triggers are single triggers.

Batch triggers

Batch triggers are generally used for increasing throughput, as trigger events are retrieved in lists instead of single events, for example, moving high volume of user activity data from Marketo into data warehouses like Redshift.

Batch triggers are similar to polling triggers in terms of how they fetch new events. You can specify group size/batch size during trigger configuration.

Batch triggers process trigger events in batches of user-specified sizes

For further details about batch triggers, refer to the Batch processing article here.

Bulk triggers

Bulk triggers are used for moving large amounts of data. They rely on sending over the records as a CSV stream. Within Workato, CSV streaming allows easy shift of data from a source to destination, as a CSV object. Individual access to each record isn't possible but instead large number of records can be efficiently sent over. This is useful when conducting exports, or daily syncs of high throughput workflows like Jira tasks, updates to tables in ServiceNow. Bulk triggers are an integral part of the ETL/ELT paradigm, and form the primary way to consume data from these sources in Workato. It is imperative to note that there's no upper limit on the number of records that can be exported as a single job with bulk triggers. This allows for easy monitoring and tracking when considering high volume use cases.

Bulk triggers can be utilized for New as well as New/Updated Records. They run on a schedule with a control similar to the scheduler trigger. They also contain the schema of the associated object to allow for deterministic parsing of the CSV object in the recipe at runtime.

CANNOT RUN BULK TRIGGERS IN TEST MODE

Bulk triggers require dedicated users to periodically check if the action is complete. This is not available in test mode.

When first started, this recipe should pick up events from

The When first started, this recipe should pick up events from input field allows your recipes to retrieve past trigger events from a specified date and time. This feature enables your recipe to capture events that have already occurred, rather than only new events created after the recipe starts.

In the following example, the Salesforce New object trigger has a When first started, this recipe should pick up events from date set to July 14, 2020, 12:00 AM PST, with the Accounts object selected.

Setting the When first started, this recipe should pick up events from field. The trigger picks up new accounts created since midnight of July 14, 2020.

With this setting, when the recipe starts, it only picks up Salesforce accounts created after July 14, 2020, 12:00 AM PST.

Not all triggers have the When first started, this recipe should pick up events from input field. For triggers without this field, the date and time from which trigger events are fetched is predetermined by default, often set as an offset from the recipe's start time. Common default values include:

• When the recipe is first started
• An hour before the recipe is first started
• A day before the recipe is first started. This offset is typically mentioned in the trigger hint for the connector.

Trigger hint indicating the default offset of 1 hour ago for Google Calendar

The When first started, this recipe should pick up events from value can only be set once and is locked from further changes after the recipe has been started for the first time.

TIMEZONE

For triggers without a dedicated Timezone input field, the timezone of the user who creates the recipe is used. When another user views the recipe, the time adjusts to their timezone, but the underlying value remains based on the creator's timezone. This means:

• If User A (with timezone -0700 Pacific Time) creates a recipe, the input time is stored with their timezone.
• If User B (with timezone +0530 Asia/Kolkata) views or deploys the recipe, the time is displayed in their timezone but retains the original value set by User A.
• The workspace timezone does not affect the When first started, this recipe should pick up events from input field unless explicitly set in the connector.

Trigger conditions

Learn how to use trigger conditions in three minutes

TRIGGER CONDITION USAGE

Triggers conditions aren't compatible with batch triggers and bulk triggers. Recipes using them may behave unexpectedly. It is recommended to use application specific query based filtering when using batch and bulk triggers.

Trigger conditions are additional rules that define what kind of trigger events should be selected for processing, for example, you can specify that only Salesforce accounts from California must be processed.

Trigger conditions are evaluated by Workato after the trigger events have been fetched. This means that Workato retrieves all new Salesforce accounts created in the last 5 minutes, and then filters out accounts not from California. This also means that, with a New Salesforce account trigger, accounts which are subsequently updated to be from California are not picked up. If your recipe seems to be missing events that you expected it to register, this might be the reason why.

Note: trigger conditions generally do not monitor a field change, but simply check if the trigger condition is fulfilled. For example, if you wish to sync only closed won opportunities from Salesforce into your ERP system, setting the following trigger condition will cause every update made to the opportunity when its in Closed Won state be picked up by the recipe. This is as opposed to having the recipe pick up the opportunity only when its status is marked closed won for the first time.

To add a trigger condition, select the Set trigger condition toggle. The trigger datatree will appear, displaying the variables that can be used to formulate the trigger condition.

Select the Set trigger condition toggle to configure trigger condition fields

Define the trigger condition. For more information on the available conditions you can use, refer to the IF condition article. The following ensures that only Salesforce accounts with a Warm rating value will be picked up by the trigger. Values are case sensitive and should be exact.

Define the trigger condition

To add an additional trigger condition, select + and choose from the OR or AND in the picklist. The selected operator will define how all additional trigger conditions will be added.

Adding another trigger condition with the OR operator

Define the additional trigger condition. Values are case sensitive and should be exact. The following ensures that accounts with a Hot rating value will also be picked up by the trigger. Notice that from the third trigger condition onwards, trigger conditions will be combined with the previously selected operator (OR or AND).

Define the additional trigger condition. Subsequent trigger conditions has to use the same AND operator

Change data capture

Change Data Capture (CDC) is a process used to capture and track changes made to data in a database. This process enables real-time or near-real-time monitoring and synchronization of data changes. This allows applications to stay up-to-date with the latest changes in the database without using continuous polling.

The primary goal of CDC is to identify and capture changes such as inserts, updates, and deletions made to database tables. These changes are then propagated to downstream systems, data warehouses, or analytics platforms, ensuring that all systems have access to the most current data.

How CDC works in Workato

Workato uses triggers to monitor changes in an app or system you specify. Workato triggers handle CDC by monitoring changes in real-time and providing notifications for those changes, which facilitates data replication and synchronization across different systems.

Triggers follow in-sequence delivery, maintain processed jobs records, prevent duplicate processing, and ensure job completion in order. Trigger dispatches can be single (real-time data sync) or bulk/batch. Bulk/batch provides improved throughput if you are working with large data volumes.

Supported data sources for CDC

Workato supports CDC for the following data sources:

• Software as a Service (SaaS) platforms
• On-premise systems
• Various databases, such as MySQL, PostgreSQL, and Snowflake
• Workato FileStorage
• Cloud storage services like Amazon S3
• Enterprise Resource Planning (ERP) systems

Advanced strategies

Explore advanced CDC strategies, including filtering and conditional triggers, handling large volumes of changes, and optimizing performance:

• Filtering and conditional triggers

• Advanced CDC strategies include the use of filtering and conditional triggers to manage the flow of data changes. This allows for more granular control over which changes to capture and propagate to downstream systems.

• Handling large volumes of changes

• Batch processing and micro-batching can be used to efficiently process and transfer if you are working with large volumes of data changes. This helps manage the change load and ensure timely data synchronization.

• Optimizing performance

• Performance optimization can be achieved through techniques such as built-in cursor management for tracking high watermarks, auto-deduplication, and in-order processing.

• Variable speed data pipelines

• Workato supports the setup of variable speed data pipelines, including near real-time/continuous data streaming, micro-batches for frequent polling, and batching for periodic schedules. This flexibility allows for tailored data orchestration strategies that fit specific business needs.