Working With Data Flows Overview
Use the Administrator tool to design the flow of data from the data source to the data target and to deploy the data flow. In a data flow, you can manage entities, data connections, and mappings between entities and nodes. You can deploy and undeploy data flows.
Before you create a data flow, perform the following tasks:
- Perform the postinstallation tasks.
- Determine the transport layer that you want to use in the data connection of the data flow.
- Get information about the data sources and data targets.
- Get information about the file rollover scheme that the application administrator has implemented for the data source. Determine a rollover scheme for the files that the target services create.
- Determine what transformations you want to apply to the data.
To create and deploy data flows, perform the following tasks:
- Design a data flow that reflects how you want EDS to move the data from the source to the target.
- Add source services and target services to the data flow.
- Connect source services to target services and add any data transformations that you want.
- Map the source services and target services to the nodes on which you want them to run. The source services use a data connection to send data to the target services.
- Deploy the data flow. You can deploy a data flow and also deploy the individual entities in the data flow. At deployment time,
EDS starts the source services and target services. The source services collect data, the transformations transform the data, and the target services write the data to the targets. The process of deployment of a data flow does not interrupt other data flows.
To change the design after you deploy a data flow, or to edit the properties of a running entity, undeploy the data flow. Make the design changes that you want and then redeploy the dataflow. In the following scenarios, you might lose data that the source generates during the period that the data flow is undeployed:
- The source application transmits the data as a stream.
- The source application performs an active file rollover when the data flow is in the undeployed state.