Rules and guidelines for custom query, SQL override, and batch size
Rules and guidelines for custom query, SQL override, and batch size
Consider the following rules and gudelines when you configure a custom query or SQL override:
If you use a custom query in the Source transformation and configure create target at runtime in the Target transformation, you must specify the
ExtendedColumnMetadata=true
property in the
Additional Connection Properties
field in the PostgreSQL connection. The property helps retain the constraints from the source in the target object.
Ensure that the list of selected columns, the data types, and the order of the columns that appear in the query matches the columns, data types, and order in which they appear in the source object.
When you configure an SQL override for a PostgreSQL source, partitioning is not applicable.
To configure the batch size consider using the following formula:
BatchSize = (<Threshold Memory in Bytes>)/(3* <Approximate Row Size in Bytes>)
.
In the formula, the threshold memory in bytes represents the maximum memory consumption that you want the operation to consume. The approximate row size is the size of a row, which you can calculate from the target table field definition.