Table of Contents

Search

  1. Preface
  2. Informatica Developer
  3. The Model Repository
  4. Searches in Informatica Developer
  5. Connections
  6. Physical Data Objects
  7. Flat File Data Objects
  8. Logical View of Data
  9. Viewing Data
  10. Application Deployment
  11. Application Patch Deployment
  12. Application Patch Deployment Example
  13. Continuous Integration and Continuous Delivery (CI/CD)
  14. Object Import and Export
  15. Appendix A: Data Type Reference
  16. Appendix B: Keyboard Shortcuts
  17. Appendix C: Connection Properties

Developer Tool Guide

Developer Tool Guide

Hadoop Connection Properties

Hadoop Connection Properties

Use the Hadoop connection to configure mappings to run on a Hadoop cluster. A Hadoop connection is a cluster type connection. You can create and manage a Hadoop connection in the Administrator tool or the Developer tool. You can use infacmd to create a Hadoop connection. Hadoop connection properties are case sensitive unless otherwise noted.
You can configure run-time properties for the Hadoop environment in the Data Integration Service, the Hadoop connection, and in the mapping. You can override a property configured at a high level by setting the value at a lower level. For example, if you configure a property in the Data Integration Service custom properties, you can override it in the Hadoop connection or in the mapping. The Data Integration Service processes property overrides based on the following priorities:
  1. Mapping custom properties set using
    infacmd ms runMapping
    with the
    -cp
    option
  2. Mapping run-time properties for the Hadoop environment
  3. Hadoop connection advanced properties for run-time engines
  4. Hadoop connection advanced general properties, environment variables, and classpaths
  5. Data Integration Service custom properties
When a mapping uses Hive Server 2 to run a job or parts of a job, you cannot override properties that are configured on the cluster level in preSQL or post-SQL queries or SQL override statements.
Workaround: Instead of attempting to use the cluster configuration on the domain to override cluster properties, pass the override settings to the JDBC URL. For example:
beeline -u "jdbc:hive2://<domain host>:<port_number>/tpch_text_100" --hiveconf hive.execution.engine=tez

0 COMMENTS

We’d like to hear from you!