The Read CSV task can use the File Set element to read a collection of CSV files from a directory and then store the data into a single RowSet variable. The CSV files must be uniform in data structure, and each CSV file must contain the same number of columns and the columns must be of the same data type (such as VARCHAR or INT). When the Read CSV task has been processed, the output RowSet variable can be used as the input RowSet variable for a write task, such as
Write to Excel, or
SQL.
Follow the steps below to read a set of CSV files:
From within the Project Designer page, expand the Data Translation folder in the Component Library, and then drag the Read CSV task to the Project Outline.
On the Basic tab of the Read CSV task, specify the value for Output RowSet Variable. The output RowSet variable will contain the combined data read from each file in the File Set.
Click the
Add
button, and then Add a File Set.
On the Basic tab, specify a Base Directory that contains CSV files.
Click the
Add
button, and then Add a Wildcard Filter element to the Read CSV task.
Wildcard Filter elements do not contain any attributes to change. Click Next to add an Include Files element.
Enter a wildcard Pattern on the
Basic
tab. In this example, "*.csv" is used to only read files that end with the CSV file extension.
Click Save. When the Read CSV task is processed, the data from each CSV file in the File Set is stored into a single RowSet variable.