The DX_Aggregate active transformation groups segments of data to process.
You can use the transformation to aggregate row sets that you previously split back into a single document
You set the DXParentEventId and the DXGroupByKey ports to sort the data that you want to aggregate.
The DX_Aggregate transformation groups rows in the following manner:
When the transformation reads the first row in the group, it creates a temporary file and writes the content of the DXData port to the file.
On each subsequent row, the transformation appends the DXDelimiter and the content of the DXData port to the file. It then increments the number of rows in the DXRowCount port.
When the transformation reads the last row in the group, it closes the temporary file and increments the number of groups in the DXGroupSequence port. It then passes the output to the pipeline.
If you use a pass-through port, the transformation sends the value from the last row of the group to the pipeline.
If a row contains an error or a null value in the DXData port, the transformation drops the row from the file and does not pass the output to the pipeline.
The DX_Aggregate transformation determines the end of the group in the following cases:
There is no more data to aggregate.
The value of the DXParentEventID port differs from the value in the previous row.
The value of the DXGroupByKey port differs from the value in the previous row.
The row count for the group reached the value that you define in the DXMaxRowsInGroup port.