Table of Contents

Search

  1. Preface
  2. Introduction to PowerExchange for HDFS
  3. PowerExchange for HDFS Configuration
  4. HDFS Connections
  5. HDFS Data Objects
  6. HDFS Data Extraction
  7. HDFS Data Load
  8. HDFS Mappings
  9. Appendix A: Data Type Reference

PowerExchange for HDFS User Guide

PowerExchange for HDFS User Guide

Rules and Guidelines for Reading from or Writing to Partition Columns

Rules and Guidelines for Reading from or Writing to Partition Columns

Consider the following rules and guidelines when you read from and write to a partition folder:
  • If the source contains htypes and the partitioned column includes String data types, a validation error is encountered.
  • When you import a directory that has a partition folder, the data type for the partition column is imported as a String.
  • For
    Create Target
    , you can add partition fields and arrange the partition columns in an order.
  • When you import a data object, the data and FileName port always shows 0 as the partition order.
  • You can read data from or write data to partition folders with Avro, Parquet, and Orc files.
  • You can use only primitive data types such as String, Integer, Long, Date, and Timestamp in a partitioned column. If you specify other data types, a validation error is encountered.
  • The partitioned directory that you select cannot have a partitioned column named FileName. The name is case insensitive.
  • For
    Create Target
    , you need to remove FileName port from the target side manually, otherwise the selected files in the directory are not partitioned.

0 COMMENTS

We’d like to hear from you!