English
  • Data Engineering Integration 10.5.2
  • All Products

Table of Contents

Search

  1. Preface
  2. Part 1: Hadoop Integration
  3. Part 2: Databricks Integration
  4. Appendix A: Connections Reference

Integration Guide

Integration Guide

Back Next

Step 3. Add IAM Roles to the EC2 Policy and Databricks

Step 3. Add IAM Roles to the EC2 Policy and Databricks

Add IAM roles to EC2 policy and the Databricks account.
The IAM role that you add in steps 1 and 2 below is different from the S3 IAM that you created in "Step 1. Create an IAM Role and Policy to Access an S3 Bucket."
  1. Add the IAM role that corresponds to the Databricks account to the EC2 instance policy.
    1. Log into the Databricks account and select the
      AWS Account
       tab.
    2. Copy the role name at the end of the Role ARN.
      The following image shows an example of the Role ARN pane. In this example, the role name to copy is
      testco-role
      :
  2. Add the Role ARN to the EC2 policy.
    Modify the EC2 instance policy to allow Databricks to pass the IAM role you copied in step 1 above to the EC2 instances for the Databricks Spark clusters:
    1. In the AWS console, go to the IAM service and select the
      Policies
       tab.
    2. Search for AwsDatabricksUserCreationPolicy.
    3. Click
      Edit Policy
      .
      The policy opens in a JSON statement editing pane.
    4. Find the section of the statement that contains
      "Action": "iam:PassRole"
      .
      The following text provides an example of the section to search for: 
       {
      "Effect": "Allow",
      "Action": "iam:PassRole",
      "Resource": "arn:aws:iam::<aws-account-id-databricks>:role/<iam-role-for-s3-access>"
    },
    5. Paste the IAM role in the place of the string
      <iam-role-for-s3-access>
       shown in the example.
    6. Click
      Review Policy
      , then
      Save changes
      .
  3. Add the Instance Profile ARN to the Databricks account.
    1. In the Databricks Admin Console, click the
      IAM Roles
       tab.
    2. Click
      Add IAM Role
      .
    3. Paste the Instance Profile ARN string that you created in "Step 1. Create an IAM Role and Policy to Access an S3 Bucket."
      The following image shows where to paste the Instance Profile ARN:
    4. Click
      Add
      .