Table of Contents

Search

  1. Preface
  2. Components
  3. Business services
  4. File listeners
  5. Fixed-width file formats
  6. Hierarchical schemas
  7. Intelligent structure models
  8. Mapplets
  9. Saved queries
  10. Shared sequences
  11. User-defined functions

Components

Components

Intelligent Structure Discovery process

Intelligent Structure Discovery
process

You can create an
intelligent structure
using
Intelligent Structure Discovery
.
After you provide a sample file,
Intelligent Structure Discovery
determines the underlying and repeating patterns of the data and creates a structure that represents the fields of data and their relationships. You can quickly model data for files whose structure is very hard, time consuming, and costly to find, such as log files, clickstreams, customer web access, error text files, or other internet, sensor, or device data that does not follow industry standards.
The following image shows the process by which
Intelligent Structure Discovery
deciphers the underlying patterns of data and creates a model of the data patterns:
This image shows unstuctured or structured data being deciphered by Intelligent Structure Discovery, which creates a model of the data.
When the model is based on input from a Hadoop Files source,
Intelligent Structure Discovery
determines the data types to use in the model based on the types and structures in the Hadoop schema. In the model,
Intelligent Structure Discovery
creates an array of structures of key-value pairs for data of type map. Nodes of type enum are represented as strings.


Updated October 15, 2020