Implementing Data Engineering Integration with Google Dataproc

Implementing Data Engineering Integration with Google Dataproc

Configure Mappings to Run on Dataproc

Configure Mappings to Run on Dataproc

To run mappings on the Dataproc cluster, configure mappings with the following properties:
  1. In the
    Parameters
    section, create a parameter with the values shown in the following table:
    Name
    Type
    Precision
    Scale
    Default Value
    pushdownConnection
    Connection
    (blank)
    (blank)
    Use the dropdown control to select the Hadoop connection that the cluster configuration created.
  2. In the
    Run-Time
    section, choose the following values:
    • Under Validation Environments, select Spark.
    • For Execution Environment, select Hadoop.
    • Under the Hadoop section, for the Connection property, click the right side of the Value cell to see the dropdown menu, and choose the pushdownConnection parameter.

0 COMMENTS

We’d like to hear from you!