When you run a mapping to read data from an Avro, flat, JSON, ORC, or Parquet file, you can use wildcard characters to specify the source file name.
To use wildcard characters for the source file name, select the source type as
Directory
and enable the
Allow Wildcard Characters
option in the advanced read properties of the Microsoft Azure Data Lake Storage Gen2 data object.
When you run a mapping in the native environment or on the Spark and Databricks Spark engine to read an Avro, JSON, ORC, Parquet, or flat file, you can use the
?
and
*
wildcard characters for the source file name.
You can use the following wildcard characters:
? (Question mark)
The question mark character (?) allows one occurrence of any character. For example, if you enter the source file name as
a?b.txt
, the Data Integration Service reads data from files with the following names:
a1b.txt
a2b.txt
aab.txt
acb.txt
* (Asterisk)
The asterisk mark character (*) allows zero or more than one occurrence of any character. If you enter the source file name as
a*b.txt
, the Data Integration Service reads data from files with the following names:
aab.txt
a1b.txt
ab.txt
abc11b.txt
When you read data from the Avro, JSON, ORC, Parquet, or flat file that contains a colon (:) character in the file name, the mapping fails.