When you import target objects, the Secure Agent appends a FileName field to the imported target object. When you map the FileName field in the target object to an incoming field, the Secure Agent creates the folder structure and the target files based on the FileName field. For example:
The FileName field is applicable to the following file formats:
Avro
ORC
Parquet
Consider the following guidelines when using the target FileName field in mappings:
Do not map the source object FileName field to the target object FileName field. If you map the FileName field in the target object to an incoming field, the Secure Agent does not create directory structure as expected.
When you use the FileName field in a target object, the Secure Agent creates folders with the following different names for null values:
For mappings:
_EMPTY_
For mappings in advanced mode:
_HIVE_DEFAULT_PARTITION_
When you map a date type incoming field to the FileName field in the target object, the Secure Agent creates a nested folder structure based on the incoming date value for target objects.
When you map an incoming field to the FileName field in the target object, the mapping runs successfully for the first time. At subsequent runs, the mapping fails with the following error:
Operation failed: Index: 0, Size: 0.
To successfully rerun the
mapping
, use a dummy target file at design time and override the dummy target file in advanced target properties.