English
  • Data Engineering Streaming 10.4.0
  • All Products

Table of Contents

Search

  1. Preface
  2. Introduction to Data Engineering Streaming
  3. Data Engineering Streaming Administration
  4. Sources in a Streaming Mapping
  5. Targets in a Streaming Mapping
  6. Streaming Mappings
  7. Window Transformation
  8. Appendix A: Connections
  9. Appendix B: Monitoring REST API Reference
  10. Appendix C: Sample Files

User Guide

User Guide

Back Next

Data Engineering Streaming Engines

Data Engineering Streaming Engines

When you run a streaming mapping, you can choose to run the mapping in a Hadoop environment or in a Databricks environment. When you validate a mapping, you can validate it against one or all of the engines. The Developer tool returns validation messages for each engine.
When you run a streaming mapping, the Data Integration Service uses a proprietary rule-based methodology to determine the best engine to run the mapping. The rule-based methodology evaluates the mapping sources and the mapping logic to determine the engine. The Data Integration Service translates the mapping logic into code that the engine can process, and it transfers the code to the engine.
For more information about the run-time process on the different engines, see
Data Engineering Integration User Guide
.