You can read objects stored in subdirectories in Amazon S3 V2 mappings in advanced mode. You can use recursive read for flat, Avro, JSON, ORC, and Parquet files. The files that you read using recursive read must have the same metadata.

To enable recursive read, select the source type as

Directory

in the advanced source properties. Enable the

Recursive Directory Read

advanced source property to read objects stored in subdirectories.

You can also use recursive read when you

specify wildcard characters

in a folder path or file name. For example, you can use a wildcard character to recursively read files in the following ways:

Folder path is
/abc*/
. Returns all files within any folder or subfolder that has a pattern
abc
at the starting of the folder name.
Folder path is
/abc*/
and file name is
myfile*
. Returns all files that have a pattern
myfile
at the starting of the file name within a folder or subfolder and has a pattern
abc
at the starting of the folder name.