Table of Contents


  1. Preface
  2. Components
  3. Business services
  4. File listeners
  5. Fixed-width file formats
  6. Hierarchical schemas
  7. Intelligent structure models
  8. Mapplets
  9. Saved queries
  10. Shared sequences
  11. User-defined functions



Intelligent Structure Discovery process

Intelligent Structure Discovery

You can create an
intelligent structure
Intelligent Structure Discovery
After you provide a sample file,
Intelligent Structure Discovery
determines the underlying and repeating patterns of the data and creates a structure that represents the fields of data and their relationships. You can quickly model data for files whose structure is very hard, time consuming, and costly to find, such as log files, clickstreams, customer web access, error text files, or other internet, sensor, or device data that does not follow industry standards.
The following image shows the process by which
Intelligent Structure Discovery
deciphers the underlying patterns of data and creates a model of the data patterns:
This image shows unstuctured or structured data being deciphered by Intelligent Structure Discovery, which creates a model of the data.
When the model is based on input from a Hadoop Files source,
Intelligent Structure Discovery
determines the data types to use in the model based on the types and structures in the Hadoop schema. In the model,
Intelligent Structure Discovery
creates an array of structures of key-value pairs for data of type map. Nodes of type enum are represented as strings.

Updated October 15, 2020