Amazon Redshift Connector Best Practices

Amazon Redshift Connector Best Practices

Data Compression

Data Compression

Amazon Redshift is a columnar data warehouse in which each columns are stored in a separate file. As the data types of the data are the same in a column, you can apply the column encoding compression strategies.
Applying compression to Amazon Redshift columns saves storage space, reduces disk I/O, and improves query performance. Amazon Redshift provides compression strategies and by default applies no compression to the columns.
You can specify the following options for when you apply column compression encoding in the Amazon Redshift columns:
Automatic compression through COPY command in design time
You can specify the COPY command option to load data from Amazon S3 to Amazon Redshift cluster. The COPY command automatically determines the best compression encoding for the column when there is an empty table with no compression encoding specified to load the data to Amazon S3.
When you apply the compression encoding, the first 100,000 rows are read and the best compression encoding for the columns are determined. Then, the compression encoding is applied to the entire table.
You can use the following configuration parameters for managing the automatic compression through COPY command:
  • Set
    COMPUPDATE=ON
    to override the compression if you have defined the compression encoding.
  • Set
    COMPROWS
    to override the default 100,0000 rows.
    The following image shows the
    Copy Options
    properties:
Set the compression encoding while creating a target in the design time
You can specify the compression encoding manually for the Amazon Redshift columns when you create a target and specify the target schema definition at design time.
Run the ANALYZE command manually
You can run the ANALYZE command manually when the Amazon Redshift table is idle. When you run the ANALYZE command, the compression analysis is performed and a report is generated with the specified compression encoding for the tables.

0 COMMENTS

We’d like to hear from you!