Microsoft Azure Data Lake Storage Gen2 Connector

Back Next

Rules and guidelines for reading from subdirectories

Consider the following rules and guidelines when you read objects stored in subdirectories:

Mappings: When you read from or write to a flat file in Microsoft Azure Data Lake Storage Gen2, ensure that the directory or subdirectory name does not contain the percentage (%) character. Else, the mapping fails.
You cannot use recursive read when you read from partition columns.
When you read a complex file in a mapping, do not use a tilde (
~
) in the subdirectory name or file name.
When the FileName field for the source and target is mapped, the file is created in the following format:
target_filename=source_directory_subdirectory1_sudirectory2

When you read a flat file with only headers and no data and map the FileName field, the expected directory structure is not created with the FileName field.
Mappings in advanced mode: When you read a complex file and enable recursive read, ensure that the path specified in the directory override or file name override matches the file path in the source.
When you read a flat file, do not use the following special characters in the directory name or sub-directory name in the directory override:

[] {} " ' + ^ % * ? space