# File Streaming
File streaming is the concept of reading and writing a file in smaller parts (chunks) in sequence. This allows us to transfer large files between apps or file systems without worrying about hitting the size limit of the sending and receiving systems. A common example is transferring records from a shared file system (SFTP) to a file hosting platform for analysis (Amazon S3).
# How it works
When transferring a file from a source app to a destination app, Workato splits the file into smaller chunks and downloads them. These chunks are then uploaded to the destination app in separate requests. This allows us to stay within the size limit of both apps, regardless of the total size of the original file.
The following diagram illustrates how this works:
As a result, Workato is able to transfer files of any size between apps that support streaming.
# How to use it
File streaming is supported only if both the source download and destination upload actions support this feature. It is automatically initiated when you map a File contents datapill (from the source app) to a File contents input field (of the destination app).
If either the source datapill or the destination input field does not support streaming, then standard data mapping is performed, i.e. the entire file is downloaded and then uploaded without any chunking. In this case, the file transfer is limited by the size limit imposed by the respective apps. The smaller of the 2 will be the resultant limit.
In either case, no additional configuration is required in the recipe.
| Connector | Triggers/Actions that support file streaming | |
|---|---|---|
| Download | Upload | |
| Amazon S3 | ||
| Anaplan | ||
| BigQuery | ||
| Box | ||
| Bynder | ||
| Microsoft Sharepoint | ||
| On-prem files | ||
| Quickbase | ||
| Salesforce | ||
| SFTP | ||
| Zoom | ||
Table of all current triggers/actions that support file streaming.
This sample recipe (opens new window) transfers files from an On-prem file system to Amazon S3 with file streaming.