Table of Contents

Search

  1. Preface
  2. Part 1: Getting Started with Snowflake Data Cloud Connector
  3. Part 2: Data Integration with Snowflake Data Cloud Connector
  4. Part 3: SQL ELT with Snowflake Data Cloud Connector
  5. Appendix A: Data type reference
  6. Appendix B: Additional runtime configurations
  7. Appendix C: Upgrading to Snowflake Data Cloud Connector

Snowflake Data Cloud Connector

Snowflake Data Cloud Connector

Configure the private key on an advanced cluster

Configure the private key on an advanced cluster

After you generate the public and private key pair using OpenSSL, you need to additionally perform certain tasks for the connection to work in a mapping in advanced mode.
Before you run mappings with the configured connection on an
advanced cluster
, set the properties for the cluster application in the mapping task.
The following list describes the properties that you need to set in the advanced session properties in a mapping task:
Spark.NeedUserCredentialFileForAdapter=true
Copies the contents of the private key from the location you specify in
Spark.UserCredentialDirOnDIS
from the Secure Agent machine to the Spark driver and executers. The folder that contains the credential file does not have the 1 MB limit. You need to ensure that the credential file of the secret key content that you copy to the cluster application does not exceed 1 MB. You need to set the value to true. Default is false.
If you do not set this flag or you set this flag to false, the private key file is not copied to the cluster application and the mapping fails.
Spark.UserCredentialDirOnDIS=<private key file directory>
Overrides the default Secure Agent directory that contains the private key with the directory that you specify for copying the private key contents to the cluster application. The default directory is
/infa/user/credentials
. Ensure that the directory does not include the private key file name.
If you do not set this flag, the default location is used. To use the default location, create the
/infa/user/credentials
directory on the Secure Agent machine and the copy the private key file here.
If you set the flag to override the location specified in the advanced session properties of the mapping task, make sure that the override location that you specify in
Spark.UserCredentialDirOnDIS
contains the private key file. Ensure that the override location and the private key file have the write permissions.
The following image shows the configured advanced custom property in the mapping task:
The configured advanced custom property in the mapping task.

0 COMMENTS

We’d like to hear from you!