On-prem file trigger - New CSV file in folder

1. How it works

This trigger monitors an on-premises folder. Based on a schedule you define, this trigger will check if there is a new CSV file inside that folder. It will read that CSV file and output the file metadata and CSV rows.

Use this trigger when you have an on-premises folder to which you frequently add CSV files. Examples of these files could be weekly sales reports or daily data dumps from other systems. Using this trigger, you can build a recipe to read those files and import the records into other system (for example, Salesforce, databases).

Important Notes:

• This trigger supports batching, which allows you to process huge amount of data in shorter time.

• Every time this trigger processes a file, it will rename the file and append the text processing at the end of filename. This is to prevent the trigger from picking up the file again in the next run, and make sure it only pickup new file in folder.

Trigger renames files when processing them

2. Input fields

For this trigger to work, you need to configure 4 sections:

1. Schedule settings
2. Folder settings
3. CSV settings
4. Batch size

2.1 Schedule settings

Configure this to tell the trigger how often it should check for new CSV file in on-premises folder. There are 2 ways to schedule: interval and date/time.

Using interval, you are telling the trigger to check after a certain amount of time.

Using date/time, you are telling the trigger to check at a specific time every day.

Optionally, you can choose which days of the week the trigger should run. Use Add/remove optional fields at the bottom of the form to add this.

2.2 Folder settings

Configure this to specify which on-premises folder the trigger should monitor for new CSV file.

Optionally, you can define a naming pattern so the trigger only picks up files with certain names. Enable this input field using Add/remove optional fields at the bottom of the form.

2.3 CSV settings

Configure this so the trigger will understand the structure of your CSV file. It is recommended that you upload a sample CSV file for fast and accurate setup.

2.4 Batch size

Lastly, set a batch size. Batch size determines how many CSV rows trigger will process at the same time. Learn more about batching here.

This is useful when you want to import the data from your CSV files into other apps like Salesforce or databases. Larger batch size will speed up this data import.

However, apps will often have an API limit of how much data they can receive at once. Therefore, try to experiment with different batch sizes or check the apps' API limits to see what works best for you.

3. Output

The following is how this trigger's output looks like:

Output pill	Description
CSV Rows	This CSV Rows output pill represents the list of rows in the CSV file. You can use this pill with repeat step to loop through all rows in the CSV file. Expand the pill, you will see all columns in your CSV file, which can be used for mapping.
File path (original)	File path before trigger processed this file. for example, C:/Program Files/sales_report.csv
File path (processed)	When trigger processes a file, it will rename the file and mark as processing. This is the file path after trigger processed the file. for example, C:/Program Files/sales_report.csv.1526632883663.processing
File name (processed)	When trigger processes a file, it will rename the file and mark as processing. This is the filename after trigger processed the file. for example, sales_report.csv.1526632883663.processing
File size (bytes)	File size in bytes.
Created time	The time the file was created.
Last modified time	The time the file was last modified.
Batch size	The batch size that you have set in the input.
Batch number	Trigger divides the CSV rows into multiple batches, then assign each batch to a job for processing. Batch number shows ID of the batch that a job is processing. Useful when you need to debug in job report. Learn more about batching here.
First batch	True if this is the first batch. Otherwise False. Useful when you need to debug in job report.
Last batch	True if this is the last batch. Otherwise False. Useful when you need to debug in job report.