Rules and Guidelines for Reading from or Writing to Partition Columns
Rules and Guidelines for Reading from or Writing to Partition Columns
Consider the following rules and guidelines when you read from and write to a partition folder:
If the source contains htypes and the partitioned column includes String data types, a validation error is encountered.
When you import a directory that has a partition folder, the data type for the partition column is imported as a String.
For
Create Target
, you can add partition fields and arrange the partition columns in an order.
When you import a data object, the data and FileName port always shows 0 as the partition order.
You can read data from or write data to partition folders with Avro, Parquet, and Orc files.
You can use only primitive data types such as String, Integer, Long, Date, and Timestamp in a partitioned column. If you specify other data types, a validation error is encountered.
The partitioned directory that you select cannot have a partitioned column named FileName. The name is case insensitive.
For
Create Target
, you need to remove FileName port from the target side manually, otherwise the selected files in the directory are not partitioned.