Table of Contents

Search

  1. Preface
  2. Introduction to Informatica Data Engineering Integration
  3. Mappings
  4. Mapping Optimization
  5. Sources
  6. Targets
  7. Transformations
  8. Python Transformation
  9. Data Preview
  10. Cluster Workflows
  11. Profiles
  12. Monitoring
  13. Hierarchical Data Processing
  14. Hierarchical Data Processing Configuration
  15. Hierarchical Data Processing with Schema Changes
  16. Intelligent Structure Models
  17. Blockchain
  18. Stateful Computing
  19. Appendix A: Connections Reference
  20. Appendix B: Data Type Reference
  21. Appendix C: Function Reference

Cluster Workflows Process

Cluster Workflows Process

Creation of a cluster workflow requires administrator and developer tasks.
The following image shows the process to create, configure, and run a cluster workflow:
The image shows a flowchart divided into administrator tasks and developer tasks. Beginning with the administrator tasks the tasks are verify prerequisites, create the cluster provisioning configuration, and create the Hadoop connection. On Azure only you must enter data Lake service principal certificate contents before creating the Hadoop connection. Then the flow goes to developer tasks: create the workflow and the create cluster task, create mapping tasks, optionally add a delete cluster task, and finally, deploy and run the workflow.
The cluster workflow development process requires tasks from the following users:
Administrator
  1. Verify installation
    . The domain can be on-premises or reside on the cloud platform.
  2. Create a cluster provisioning configuration
    . Create a cloud provisioning configuration in the Administrator tool. The cloud provisioning configuration contains all of the information that the Data Integration Service requires to contact and create resources on the cloud platform.
  3. Create a cluster connection
    . Create a dedicated cluster connection to associate with the cluster provisioning configuration.
For information about administrator tasks, see the
Data Engineering Administrator Guide
.
Developer
A developer completes the following tasks:
  1. Create the workflow and the Create Cluster task.
  2. Create Mapping tasks and other tasks as required.
  3. Create a Delete Cluster task if you want to delete the cluster and resources when processing is complete.
  4. Deploy and run the workflow.


Updated September 28, 2020