Table of Contents

Search

  1. Preface
  2. Introduction to Data Transformation
  3. Data Processor Transformation
  4. Wizard Input and Output Formats
  5. Relational Input and Output
  6. Using the IntelliScript Editor
  7. XMap
  8. Libraries
  9. Schema Object
  10. Command Line Interface
  11. Scripts
  12. Parsers
  13. Script Ports
  14. Document Processors
  15. Formats
  16. Data Holders
  17. Anchors
  18. Transformers
  19. Actions
  20. Serializers
  21. Mappers
  22. Locators, Keys, and Indexing
  23. Streamers
  24. Validators, Notifications, and Failure Handling
  25. Validation Rules
  26. Custom Script Components

Configure the Complex File Reader For Parquet Input

Configure the Complex File Reader For Parquet Input

After you create a Data Processor transformation that converts Parquet input, add the transformation to a mapping with a complex file reader. Configure the complex file reader to process Parquet input.
  1. In the Mapping editor, create a complex file reader object.
  2. To configure the complex file reader, perform the following steps:
    1. In the
      Advanced
      tab of the
      Properties
      view, select the
      File Format
      property, and then choose
      Input Format
      .
    2. In the
      Advanced
      tab, select the
      Input Format
      property, and then type
      com.informatica.parquet.ParquetToXML
      .
    3. To optimize performance, use the
      Input Format Parameters
      property to tune the
      MaxOutputAccumulation
      parameter. By default, the
      MaxOutputAccumulation
      parameter, which defines the expected number of output records, is set to 50,000. To change the setting to 250,000 for example, enter
      "MaxOutputAccumulation"="250000"
      .
    4. By default, the complex file reader adds the schema to the complex file reader output within a single element directly after the root element. If you do not want to add the schema to the output, select the
      Input Format Parameters
      property, then type
      "InjectSchema"="false"
      .
      Use a semi-colon to separate multiple parameters, for example
      "MaxOutputAccumulation"="250000";"InjectSchema"="false"
      .
  3. Add the Data Processor transformation to the mapping. The transformation input port should remain set to binary input, the default setting for Parquet input.
  4. Link the complex file reader output port to the Data Processor transformation input port. The complex file reader output port should remain set to binary output.