Amazon Redshift Connector Best Practices

Amazon Redshift Connector Best Practices

Configure Amazon Redshift Workload Management

Configure Amazon Redshift Workload Management

In the Online Transaction Processing systems (OLTP), most queries are of uniform size and have similar execution cost for all jobs. However, in the case of data warehouse, the queries vary greatly in the execution cost, time, and size of the result set.
Some are interactive queries that have strict SLAs while others are adhoc queries analyzing large volumes of data that takes longer time to complete the task or session. The ETL jobs are differentiated according to the job execution, such as the incremental updates that takes shorter time while initial loads or massive updates takes longer time.
There is a need to manage the priority, memory allocated, and concurrency of the different warehouse jobs. To ensure that the slow running queries are not blocking fast running queries, Amazon Redshift Workload Management divides the cluster availability resource into slots. Then, combines and arranges those slots into queues and assigns the incoming queries to the queues. You can configure priority, memory, and concurrency for queues through the Amazon Redshift Workload Management interface.
The following image shows the Amazon Redshift Workload Management Configuration page:

0 COMMENTS

We’d like to hear from you!