Tasks

Back Next

Guidelines to get an accurate recommendation

Use the following guidelines to get an accurate recommendation during the tuning job:

Use sample data that closely matches the actual volume of the data that the

mapping

task will process.

Make sure that the mapping logic handles duplicate data in the target. The tuning job will write data to the target multiple times.

Set resource limits on your cloud environment by configuring the appropriate Spark properties before you tune the

mapping

task. Your cloud service provider charges you for the resources that each run uses.

For example, if you know that you can allocate only 4 GB to the Spark driver, you can configure

spark.driver.memory=4G

in the

mapping

task. CLAIRE will honor the pre-defined Spark property to create a tuning recommendation for other Spark properties.

Watch

Comments

0 COMMENTS

We’d like to hear from you! Log in to comment.