When you configure a mapping, ensure that you use only the special characters that are allowed in Microsoft Azure Data Lake Storage Gen2 in the directory name, file name, or object path.

You cannot read and write primitive data types in nested and multi-line indented JSON files in mappings.

When a column name in the source starts with a number and you create a target at runtime, the corresponding target column is prefixed with an underscore character (_).

When you create a Microsoft Azure Data Lake Storage Gen2 target at runtime to write an Avro, ORC, or Parquet file, you cannot write null values with primitive data types.

When you use the JVM options to configure the HTTPS proxy server and read and write Avro, Parquet, and ORC files, the mapping fails with the following error:

When you append data to a Microsoft Azure Data Lake Storage Gen2 target, you must map all the incoming fields to target fields.

When you create a Microsoft Azure Data Lake Storage Gen2 target at runtime and specify the path in the object name field, ensure that you specify the complete path including the file system name.

When the data has more than one escape character and you do not select the

Disable escape char when a qualifier is set

option, all the escape characters are not written to the flat file target in Microsoft Azure Data Lake Storage Gen2.

You cannot read hierarchical data types in a mapping. You can use the Hierarchy Parser transformation to convert the hierarchical input into relational output.

When you run a mapping to write a JSON file to a Microsoft Azure Data Lake Storage Gen2 target, the mapping writes the values of double data type in exponential format in the target.

When you run a mapping configured with a fixed partition of 8 to read from a parquet file of size 20 GB or more and write to a Microsoft Azure Data Lake Storage Gen2 target, the mapping fails with the following error:

If a mapping includes parameterized source and target,

Allow parameter to be overridden at run time

checkbox is selected, and the source object selected resides in a folder during the mapping task creation, the mapping fails with the following error:

When you use the Snappy compression format to write data to Microsoft Azure Data Lake Storage Gen2, the mapping retains a

snappy-1.1.8****-libsnappyjava.so

file in the temp directory on the agent machine after it runs successfully.

When you create a new target and select

Handle Special Characters

to append time stamp to the file name, the file name override is not honored in a mapping.

When you run a mapping to create a Microsoft Azure Data lake storage Gen2 target, use the append strategy, and enable

Handle Special Characters

, the file is either created or appended based on the timestamp you include in the file name.

When you parameterize the target connection and object and create a new target at runtime, the target is created in the root directory specified in the Connection Directory path even if you specify the complete path in the create target object name.

When you read

and write

complex files, set the JVM options for type DTM to increase the -Xms and -Xmx values in the system configuration details of the Secure Agent to avoid java heap space error. The recommended -Xms and -Xmx values are 512 MB and 1024 MB respectively.

When you read

and write

complex files, ensure that the file name does not contain percentage (%) or hash (#). Otherwise, the data preview and mapping fails at runtime.

When you run a mapping in advanced mode to read data from a Microsoft Azure Data lake storage Gen2 source, use a parameter file to parameterize the source connection and object, and specify the directory and file override in the advanced properties, the mapping considers the values specified in the parameter file.

When you read data from and write data to Microsoft Azure Data Lake Storage Gen2, use the same storage account for both the source and target connections.

When you read data from and write data to Microsoft Azure Data Lake Storage Gen2 and use the shared key authentication, ensure that you use the same access key for both the source and target connections.

When you use the managed identity authentication, you cannot use system assigned identity.

When you use managed identity authentication, ensure that the storage account specified in the connection is not the same as the storage account specified in the staging location and log location for the Azure cluster.

You can read and write hierarchical data types for Avro, JSON, and Parquet files. You can also read hierarchical data types for ORC files.

When you set the qualifier mode to Minimal and use an escape character, the escape characters are not escaped and quoted in the target. To resolve this issue, set the qualifier mode to All.

When you set the qualifier mode to All and do not specify a value for the qualifier, \00 (Null) is considered as the qualifier.

You cannot add multiple pipelines in a mapping.

When you read from a complex file source of size 128 MB or more, the Secure Agent writes incorrect data and creates multiple target files without overriding the existing target.

You cannot read zero-byte files when you run mappings in advanced mode.

When you upload a schema file for the source and create a Microsoft Azure Data Lake Storage Gen2 target at runtime, ensure that source file is not empty.

When you append data to an existing target, you must configure any overrides in the advanced target properties, else the mapping fails.

When you append data to a target created at runtime and if a file with the same name exists in the target directory, the mapping fails.

When a JSON file has a field with empty struct data, the Secure Agent ignores the field and reads the remaining fields during metadata read.

When you run a mapping in advanced mode and map the source fields of double or float data type to the target fields of string data type, the format of the values changes in the target.

Value in the source	Value in the target
1.7976931348623157e+308	1.79769313486232e+308
-9999999999999.99	-10000000000000
4.94065645841247e-324	4.9e-324
7956318123.99392483	7956318123.99392