Table of Contents

Search

  1. Preface
  2. Introduction to Informatica Data Engineering Integration
  3. Mappings
  4. Mapping Optimization
  5. Sources
  6. Targets
  7. Transformations
  8. Python Transformation
  9. Data Preview
  10. Cluster Workflows
  11. Profiles
  12. Monitoring
  13. Hierarchical Data Processing
  14. Hierarchical Data Processing Configuration
  15. Hierarchical Data Processing with Schema Changes
  16. Intelligent Structure Models
  17. Blockchain
  18. Stateful Computing
  19. Appendix A: Connections Reference
  20. Appendix B: Data Type Reference
  21. Appendix C: Function Reference

Overview of Intelligent Structure Models

Overview of Intelligent Structure Models

You can use CLAIRE®
Intelligent Structure Discovery
to parse semi-structured or structured data in mappings that run on the Spark engine.
Long, complex files with little or no structure can be difficult to understand much less parse. CLAIRE
Intelligent Structure Discovery
can automatically discover the structure in unstructured data.
CLAIRE uses machine learning algorithms to decipher data in semi-structured or unstructured data files and create a model of the underlying structure of the data. You can generate an
intelligent structure model
, a model of the pattern, repetitions, relationships, and types of fields of data discovered in a file, in
Informatica Intelligent Cloud Services
.
To use the model, you associate it with a data object that represents a complex file source in a data engineering mapping. You can run the mapping on the Spark engine to process the data. The mapping uses the
intelligent structure model
to extract and parse data from input files based on the structure expressed in the model.
To associate a model with a data object, you can perform one of the following tasks:
  • Create and export an
    intelligent structure model
    in Cloud
    Data Integration
    and then select the model when you create the data object.
  • Select an XML, JSON, ORC, Avro, or Parquet sample file when you create the data object.
    Intelligent Structure Discovery
    creates an
    intelligent structure model
    based on the sample file that you select.
    You cannot edit or refine a model that
    Intelligent Structure Discovery
    creates automatically. To edit and refine a model, create, edit, and export it in Cloud
    Data Integration
    .


Updated September 28, 2020