Table of Contents

Search

  1. Preface
  2. Introduction to Data Engineering Streaming
  3. Data Engineering Streaming Administration
  4. Sources in a Streaming Mapping
  5. Targets in a Streaming Mapping
  6. Streaming Mappings
  7. Window Transformation
  8. Appendix A: Connections
  9. Appendix B: Monitoring REST API Reference
  10. Appendix C: Sample Files

Transformations in a Streaming Mapping

Transformations in a Streaming Mapping

When you create a streaming mapping, certain transformations are valid or are valid with restrictions.
The transformation types that you can add to a streaming mapping correlate to the types that you can add to a batch mapping with Data Engineering Integration. Most of the restrictions related to batch mappings also apply to streaming mappings. The transformation restrictions mentioned in this guide apply specifically to streaming mappings.
The following table lists the supported transformations in a streaming mapping in the Hadoop and Databricks environment:
Transformations
Hadoop Environment
Databricks Environment
Address Validator transformation
Yes
Yes
Aggregator transformation
Yes
Yes
Classifier transformation
Yes
Yes
Data Masking transformation
Yes
-
Expression transformation
Yes
Yes
Filter transformation
Yes
Yes
Java transformation
Yes
-
Joiner transformation
Yes
Yes
Lookup transformation
Yes
-
Macro transformation
Yes
Yes
Normalizer transformation
Yes
Yes
Parser transformation
Yes
Yes
Python transformation
Yes
Yes
Rank transformation
Yes
Yes
Router transformation
Yes
Yes
Sorter transformation
Yes
-
Standardizer transformation
Yes
Yes
Union transformation
Yes
Yes
Window transformation
Yes
Yes
The Window transformation is not supported for batch mappings.
For information about restrictions for batch mappings, see the
Data Engineering Integration User Guide
.

0 COMMENTS

We’d like to hear from you!