Table of Contents

Search

  1. Preface
  2. Introduction to Data Engineering Streaming
  3. Data Engineering Streaming Administration
  4. Sources in a Streaming Mapping
  5. Targets in a Streaming Mapping
  6. Streaming Mappings
  7. Transformation in Streaming Mappings
  8. Window Transformation
  9. Appendix A: Connections
  10. Appendix B: Monitoring REST API Reference
  11. Appendix C: Sample Files

Advanced Properties

Advanced Properties

You can use the advanced properties to specify data object write operation properties to write data to a Microsoft Azure Data Lake Storage Gen2 data object.
The following table describes the advanced properties that you configure for a Microsoft Azure Data Lake Storage Gen2 write operation:
Property
Description
Operation Type
The type of data object operation.
This is a read-only property.
Concurrent Threads
The number of threads to use to move data to the staging area in Microsoft Azure Data Lake Storage Gen2. Default is 10.
Filesystem Name Override
Overrides the default file name.
Directory Override
Overrides the default directory path.
The Microsoft Azure Data Lake Store Gen2 directory that you use to write data. Default is root directory. The directory path specified at run time overrides the path specified while creating a connection.
File Name Override
Name of the file to which you want to write data. The file specified at run time overrides the file specified while creating a connection.
Write Strategy
Not applicable for streaming mappings.
Block Size
Not applicable for streaming mappings.
Compression Format
Not applicable for streaming mappings.
Timeout Interval
The number of seconds to wait when attempting to connect to the server. A timeout will occur if the connection cannot be established in the specified amount of time.
Stream Rollover Size in GB
Optional. Target file size, in gigabytes (GB), at which to trigger rollover. A value of zero (0) means that the target file does not roll over based on size. Default is 1 GB.
Stream Rollover Time in Hours
Optional. Length of time, in hours, for a target file to roll over. After the time period has elapsed, the target file rolls over. A value of zero (0) means that the target file does not roll over based on time. Default is 1 Hour.
Interim Directory
The active directory location of the complex file target. This directory stages all the files currently in open state. When the stream rollover condition is met, the files are moved from the interim directory to the target directory.

0 COMMENTS

We’d like to hear from you!