Table of Contents

Search

  1. Preface
  2. Transformations
  3. Source transformation
  4. Target transformation
  5. Access Policy transformation
  6. B2B transformation
  7. Aggregator transformation
  8. Cleanse transformation
  9. Data Masking transformation
  10. Data Services transformation
  11. Deduplicate transformation
  12. Expression transformation
  13. Filter transformation
  14. Hierarchy Builder transformation
  15. Hierarchy Parser transformation
  16. Hierarchy Processor transformation
  17. Input transformation
  18. Java transformation
  19. Java transformation API reference
  20. Joiner transformation
  21. Labeler transformation
  22. Lookup transformation
  23. Machine Learning transformation
  24. Mapplet transformation
  25. Normalizer transformation
  26. Output transformation
  27. Parse transformation
  28. Python transformation
  29. Rank transformation
  30. Router transformation
  31. Rule Specification transformation
  32. Sequence Generator transformation
  33. Sorter transformation
  34. SQL transformation
  35. Structure Parser transformation
  36. Transaction Control transformation
  37. Union transformation
  38. Velocity transformation
  39. Verifier transformation
  40. Web Services transformation

Transformations

Transformations

Output fields

Output fields

When you select an intelligent structure to use in a Structure Parser transformation, the intelligent structure model output fields appear on the
Output Fields
tab of the Properties panel.
In a mapping, the output fields are grouped into the following groups:
  • Unidentified group. This group contains the data that was not identified by the intelligent structure. You might want to pass this data to a target file for further analysis.
  • Output groups. One or more groups that contain the data that the intelligent structure model identifies.
You can select a group or groups to transfer pass-through fields to. Pass-through fields are fields that you don't map in the transformation field mapping and that the transformation transfers to the selected group or groups as is. Use this option if you want to use the pass-through fields later in the mapping, for example, to pass a timestamp field to the next downstream transformation. The Structure Parser transformation passes the pass-through fields to each selected group.
In a mapping in advanced mode, the output fields are put in one group. This group includes pass-through fields if you configure the transformation to enable pass-through fields.
The
Output Fields
tab displays the first five fields in each output group, including the name, type, precision, scale, and origin of each field. You can click the link at the bottom of the group to display all the fields in the group. The origin of a field shows the path of the respective node in the intelligent structure model. If a name of a node in the intelligent structure model contains special characters, the Secure Agent replaces them with an underscore (_) character, and the
Output Fields
tab displays the revised name as the output field name. The
Output Fields
tab doesn't display the origin of fields with revised names.
To edit the precision of an output field, click the precision value and enter the precision you require.
Intelligent Structure Discovery
doesn't enforce precision and scale on decimal fields.
You can't edit the transformation output fields. If you want to exclude output fields from the data flow or rename output fields before you pass them to a downstream transformation or target, configure the field rules for the downstream transformation.

Output groups for relational output

If you configure the transformation to generate relational output, the transformation can contain multiple output groups. Each output group contains one or more fields that you can write to a relational target or pass to a downstream transformation for further processing.
In advanced mode, the Structure Parser transformation can't generate relational output.

Output groups for JSON, JSON lines, and XML output

If you configure the transformation to generate JSON, JSON lines, or XML output, the transformation contains one output group. The output group contains one string field which contains the output data. The default precision is 1,000,000 characters. You can pass the data in the string field to a downstream transformation or to a target of the appropriate type. For example, if you configure the transformation to generate XML output, you can connect the output field to a flat file target to save the output to an XML file.

Output groups for Avro, Parquet, and ORC output

If you configure the transformation to generate Avro, Parquet, or ORC output, the transformation contains one output group. The output group contains one binary field which contains the output data. The default precision is 1,000,000 characters. You can pass the data in the binary field to a downstream transformation or to a target of the appropriate type. For example, if you configure the transformation to generate Parquet output, you can connect the output field to a target such as Amazon S3 or Hadoop Files.
In advanced mode, the Structure Parser transformation can't generate Avro, Parquet, or ORC output.

0 COMMENTS

We’d like to hear from you!