User Guide

10.4.0
- 10.5.7
- 10.5.4
- 10.5.2
- 10.5.1
- 10.5
- 10.4.1
- 10.2.2 Service Pack 1
- 10.2.2
- 10.2.1

Back Next

Example

You run the IT department of a major bank that has millions of customers. You want to monitor network activity in real time. You need to collect network activity data from various sources such as firewalls or network devices to improve security and prevent attacks. The network activity data includes Denial of Service (DoS) attacks and failed login attempts made by customers. The network activity data is written to Kafka queues.

You perform the following tasks:

Create a streaming mapping to read data from the Kafka queues that stream data in JSON, XML, CSV, or Avro formats.

Add a Lookup transformation to get data from a particular customer ID.

Add a Window transformation to accumulate the streamed data into data groups before processing the data.

Process the data. Add an Aggregator transformation to perform aggregations on the data from the customer ID.

Monitor jobs. Monitor statistics for the mapping job on the Monitoring tab of the Administrator tool.

The following image shows the mapping:

The mapping example shows a Kafka input, a Lookup transformation, a Window transformation, an Aggregator transformation, and a Kafka output.

Rename Saved Search

Table of Contents

User Guide

User Guide

Example

Example