Table of Contents

Search

  1. Preface
  2. Introduction to PowerExchange for Microsoft Azure Data Lake Storage Gen2
  3. PowerExchange for Microsoft Azure Data Lake Storage Gen2 Configuration
  4. Microsoft Azure Data Lake Storage Gen2 Connections
  5. PowerExchange for Microsoft Azure Data Lake Storage Gen2 Data Objects
  6. Microsoft Azure Data Lake Storage Gen2 Mappings
  7. Appendix A: Microsoft Azure Data Lake Storage Gen2 Datatype Reference

PowerExchange for Microsoft Azure Data Lake Storage Gen2 User Guide

PowerExchange for Microsoft Azure Data Lake Storage Gen2 User Guide

Advanced Properties

Advanced Properties

You can use the advanced properties to specify data object read operation properties to read data from a
Microsoft Azure Data Lake Storage Gen2
server.
The following table describes the Advanced source column properties of the data object read operation:
Property
Description
Concurrent Threads
Number of concurrent connections to read data from
Microsoft Azure Data Lake Storage Gen2
. When reading a large file or object, you can spawn multiple threads to process data. Configure
Block Size
to partition a large file into smaller parts. Default is 10.
Filesystem Name Override
Overrides the file system name.
Directory Path Override
Overrides the default directory path. You can specify an absolute or a relative directory path.
  • Absolute path. The Data Integration Service searches this directory path in the specified file system. Example of absolute path:
    Dir1/Dir2
  • Relative path: The Data Integration Service searches this directory path in the native directory path of the object. Example of relative path:
    /Dir1/Dir2
File Name Override
Overrides the file name.
Source Type
Select the type of source from which you want to read data. You can select the following source types:
  • File
  • Directory
Default is File. The directory read is not applicable to binary and complex files.
Block Size
Partitions a large file or object into smaller parts each of specified block size. When reading a large file, consider partitioning a large file into smaller parts and configure
Concurrent Threads
to spawn required number of threads to process data in parallel.
Timeout Interval
The number of seconds to wait when attempting to connect to the server. A timeout will occur if the connection cannot be established in the specified amount of time. Default is 0.
Compression Formats
Compresses data when you read data
Microsoft Azure Data Lake Storage Gen2
. You can read the following compressed files:
  • None: no compression format.
  • Gzip: applicable to flat files. The source file extension must be
    .GZ
    .
  • Snappy: applicable to complex files.
Default is None.
If you select a
.GZ
source object and select None in the compression format, though the mapping runs successfully, data is not written to the target object.
Tracing Level
Sets the amount of detail that appears in the log file. You can choose terse, normal, verbose initialization, or verbose data. Default is normal.

0 COMMENTS

We’d like to hear from you!