Sets the amount of detail that appears in the log file. You can choose terse, normal, verbose initialization, or verbose data. Default is normal.

Number of concurrent connections to read data from

Microsoft Azure Data Lake Storage Gen2

. When reading a large file or object, you can spawn multiple threads to process data. Configure

Block Size

to partition a large file into smaller parts. Default is 10.

Overrides the file system name.

Select the type of source from which you want to read data. You can select the following source types:

File
Directory

Indicates whether you want to use wildcard characters for the source file name when you run a mapping to read data from an Avro, flat, JSON, ORC, or Parquet file.

Overrides the default directory path. You can specify an absolute or a relative directory path.

Absolute path. The Data Integration Service searches this directory path in the specified file system. Example of absolute path:
Dir1/Dir2
Relative path: The Data Integration Service searches this directory path in the native directory path of the object. Example of relative path:
/Dir1/Dir2

Overrides the file name.

Specify the number of sub-directory levels you want to read in the directory in the native environment or on the Spark engine.

Partitions a large file or object into smaller parts each of specified block size. When reading a large file, consider partitioning a large file into smaller parts and configure

Concurrent Threads

to spawn required number of threads to process data in parallel.

The number of seconds to wait when attempting to connect to the server. A timeout will occur if the connection cannot be established in the specified amount of time. Default is 0.

Compresses data when you read data

Microsoft Azure Data Lake Storage Gen2

. You can read the following compressed files:

None: no compression format.
Gzip: applicable to flat files. The source file extension must be
.GZ
.
Snappy: applicable to complex files.

Default is None.