Table of Contents

Search

  1. Preface
  2. Part 1: Version 10.4.1
  3. Part 2: Version 10.4.0
  4. Part 3: Version 10.2.2
  5. Part 4: Version 10.2.1
  6. Part 5: Version 10.2
  7. Part 6: Version 10.1.1
  8. Part 7: Version 10.1

Streaming Mappings in AWS Databricks

Streaming Mappings in AWS Databricks

Effective in version 10.4.1, you can run streaming mappings in AWS Databricks service in AWS cloud ecosystems.
You can use AWS Databricks to run mappings with the following functionality:

Sources and Targets

You can run streaming mappings against the following sources and targets within the Databricks environment:
  • Amazon S3
  • Kinesis Streams
  • Kinesis Firehose

Transformations

You can add the following transformations to a Databricks streaming mapping in AWS:

    Aggregator

    Expression

    Filter

    Joiner

    Normalizer

    Rank

    Router

    Union

    Window

Data Types

AWS Databricks supports the same data types as Azure Databricks.
The following data types are supported:

    Array

    Bigint

    Date/time

    Decimal

    Double

    Integer

    Map

    Struct

    Text

    String

For more information, see the following
Data Engineering Streaming 10.4.1 User Guide
.


Updated August 28, 2020