Components

Back Next

Using intelligent structure models in mappings in advanced mode

Using
intelligent structure model
s in mappings in advanced mode

To use an

intelligent structure model

in a mapping in advanced mode, configure the Source transformation to discover the data structure at the source or add a Structure Parser transformation downstream in the mapping.

The model generates hierarchical output. Models that you use in advanced mode can contain up to 8,000 fields.

To use an

intelligent structure model

when you configure the Source transformation, select an existing model or create a new model when you configure the Source transformation. You can use an

intelligent structure model

with an Amazon S3 V2 or a Microsoft Azure Data Lake Gen2 connection. The following table describes the actions that you perform when you configure the source properties to use an

intelligent structure model

Property	Action
Connection	Select an Amazon S3 V2 or a Microsoft Azure Data Lake Gen2 connection.
Source Type	Select Single Object .
Object	Select a file or folder as the source object: To select a file, select File as the source type in the Advanced area, and then select a file of a type that is supported by Intelligent Structure Discovery . To select a folder, select Directory as the source type in the Advanced area, and then select a folder that contains one or more Avro, Parquet, or ORC files.
Format	Select Discover Structure .
Intelligent Structure Model	Select one of the following options to associate a model with the transformation: Select. Select an existing model. New. Create a new model. Select Design New to create the model. Select Auto-generate from sample file for Intelligent Structure Discovery to generate a model based on sample input that you select.