Table of Contents

Search

  1. Preface
  2. Introduction to Data Engineering Streaming
  3. Data Engineering Streaming Administration
  4. Sources in a Streaming Mapping
  5. Targets in a Streaming Mapping
  6. Streaming Mappings
  7. Transformation in Streaming Mappings
  8. Window Transformation
  9. Appendix A: Connections
  10. Appendix B: Monitoring REST API Reference
  11. Appendix C: Sample Files

Transformations in a Streaming Mapping Overview

Transformations in a Streaming Mapping Overview

When you create a streaming mapping, certain transformations are valid or are valid with restrictions.
The transformation types that you can add to a streaming mapping correlate to the types that you can add to a batch mapping with Data Engineering Integration. Most of the restrictions related to batch mappings also apply to streaming mappings. The transformation restrictions mentioned in this guide apply specifically to streaming mappings.
The following table lists the supported transformations in a streaming mapping in the Hadoop and Databricks environment:
Transformations
Environment
Address Validator
Hadoop, Databricks
Aggregator
Hadoop, Databricks
Classifier
Hadoop, Databricks
Data Masking
Hadoop
Expression
Hadoop, Databricks
Filter
Hadoop, Databricks
Java
Hadoop
Joiner
Hadoop, Databricks
Lookup
Hadoop
Macro
Hadoop, Databricks
Normalizer
Hadoop, Databricks
Parser
Hadoop, Databricks
Python
Hadoop, Databricks
Rank
Hadoop, Databricks
Router
Hadoop, Databricks
Sorter
Hadoop
Standardizer
Hadoop, Databricks
Union
Hadoop, Databricks
Window
Hadoop, Databricks
The Window transformation is not supported for batch mappings.
For information about restrictions for batch mappings, see the
Data Engineering Integration User Guide
.