Table of Contents

Search

  1. Preface
  2. Introduction to Data Transformation
  3. Data Processor Transformation
  4. Wizard Input and Output Formats
  5. Relational Input and Output
  6. XMap
  7. Libraries
  8. Schema Object
  9. Command Line Interface
  10. Scripts
  11. Parsers
  12. Script Ports
  13. Document Processors
  14. Formats
  15. Data Holders
  16. Anchors
  17. Transformers
  18. Actions
  19. Serializers
  20. Mappers
  21. Locators, Keys, and Indexing
  22. Streamers
  23. Validators, Notifications, and Failure Handling
  24. Validation Rules
  25. Custom Script Components

Data Transformation User Guide

Data Transformation User Guide

Configure the Complex File Reader For Parquet Input

Configure the Complex File Reader For Parquet Input

After you create a Data Processor transformation that converts Parquet input, add the transformation to a mapping with a complex file reader. Configure the complex file reader to process Parquet input.
  1. In the Mapping editor, create a complex file reader object.
  2. To configure the complex file reader, perform the following steps:
    1. In the
      Advanced
      tab of the
      Properties
      view, select the
      File Format
      property, and then choose
      Input Format
      .
    2. In the
      Advanced
      tab, select the
      Input Format
      property, and then type
      com.informatica.parquet.ParquetToXML
      .
    3. To optimize performance, use the
      Input Format Parameters
      property to tune the
      MaxOutputAccumulation
      parameter. By default, the
      MaxOutputAccumulation
      parameter, which defines the expected number of output records, is set to 50,000. To change the setting to 250,000 for example, enter
      "MaxOutputAccumulation"="250000"
      .
    4. By default, the complex file reader adds the schema to the complex file reader output within a single element directly after the root element. If you do not want to add the schema to the output, select the
      Input Format Parameters
      property, then type
      "InjectSchema"="false"
      .
      Use a semi-colon to separate multiple parameters, for example
      "MaxOutputAccumulation"="250000";"InjectSchema"="false"
      .
  3. Add the Data Processor transformation to the mapping. The transformation input port should remain set to binary input, the default setting for Parquet input.
  4. Link the complex file reader output port to the Data Processor transformation input port. The complex file reader output port should remain set to binary output.


Updated September 26, 2018