Hive Connector

Back Next

Adding columns as partitions to the target

In this example, you want to create a table to write sales data from retail stores spread across Asia Pacific to a Hive table. You want to write data to partitions categorized based on country, state, and outlet names so that you can run queries easily across data split in partitions.

Configure a target transformation with the

Create New at Runtime

option.

To include a bucket, in the

Create New at Runtime

properties, specify the number of buckets you want in the target.

Include a bucket if you want to split incoming data into a bucket.

On the

Target Fields

tab, edit the metadata, and then select the column that you want to include as a bucket field in the Hive target table.

You cannot edit the metadata for a column that you select for partitioning. If you need to include buckets, you must select the column that you want as a bucket on the

Target Fields

tab, and then select those columns for partitioning on the

Partitions

tab.

Click the

icon in the

Partitions

tab to add the partition columns for a target.

The following image shows the

Partitions

tab where you can add the partition columns: The image shows the Add button in the Partitions tab to add the partition columns.

On the

Partitions

tab, select the required partitioning fields from the list of incoming fields from the source:

Click

Select

The

Partitions

tab shows the partition columns that you selected:

If required, change the partition order using the up and down arrows as shown in the following image:

Do not change the partitioning order in the Target Fields
tab in the Target transformation. You can change the partitioning order only from the Partitions
tab.

The columns that you select for partitioning are set by default to string while writing the data to the target.

Rename Saved Search

Table of Contents

Hive Connector

Hive Connector

Adding columns as partitions to the target

Adding columns as partitions to the target