Communities
A collaborative platform to connect and grow with like-minded Informaticans across the globe
Product Communities
Connect and collaborate with Informatica experts and champions
Discussions
Have a question? Start a Discussion and get immediate answers you are looking for
User Groups
Customer-organized groups that meet online and in-person. Join today to network, share ideas, and get tips on how to get the most out of Informatica
Get Started
Community Guidelines
Knowledge Center
Troubleshooting documents, product guides, how to videos, best practices, and more
Knowledge Base
One-stop self-service portal for solutions, FAQs, Whitepapers, How Tos, Videos, and more
Support TV
Video channel for step-by-step instructions to use our products, best practices, troubleshooting tips, and much more
Documentation
Information library of the latest product documents
Velocity (Best Practices)
Best practices and use cases from the Implementation team
Learn
Rich resources to help you leverage full capabilities of our products
Trainings
Role-based training programs for the best ROI
Certifications
Get certified on Informatica products. Free, Foundation, or Professional
Product Learning Paths
Free and unlimited modules based on your expertise level and journey
Resources
Library of content to help you leverage the best of Informatica products
Tech Tuesdays Webinars
Most popular webinars on product architecture, best practices, and more
Product Availability Matrix
Product Availability Matrix statements of Informatica products
SupportFlash
Monthly support newsletter
Support Documents
Informatica Support Guide and Statements, Quick Start Guides, and Cloud Product Description Schedule
Product Lifecycle
End of Life statements of Informatica products
Ideas
Events
Change Request Tracking
Marketplace
English
  • Test Data Management 10.4.1
  • All Products

Table of Contents

Search

  1. Preface
  2. Support Changes
  3. Verify System Requirements
  4. Installation and Upgrade
  5. Post-installation Steps
  6. Emergency Bug Fixes Merged into 10.4.1.1
  7. 10.4.1.1 Fixed Limitations and Closed Enhancements
  8. 10.4.1.1 Known Limitations

Release Notes (10.4.1.1)

Release Notes (10.4.1.1)

Back Next

Configure Amazon Glue as the Hive Metastore

Configure Amazon Glue as the Hive Metastore

You can configure Amazon Glue as the Hive metastore with an Amazon EMR 5.29 cluster.
To enable integration with an EMR cluster with Glue, copy .jar files from the cluster to the domain, and then enable the Hive metastore setting in the hive-site.xml configuration before you refresh the cluster configuration and the Hadoop connection.
Consider the following rules and guidelines:
  • Glue does not support Hive transactions.
  • Kerberos is not supported for the Glue metastore service. You cannot use a Kerberos-enabled EMR cluster with Glue as the Hive metastore.

Copy .jar Files

Copy .jar files from the Amazon EMR cluster to the Informatica domain:
  1. Copy the hive-exec-1.2.1-spark2-amzn-1.jar file from the
    /usr/lib/spark/jars/
     directory of the Glue-enabled EMR 5.29 cluster and paste it in the following location:
    <Informatica installation directory>/services/shared/spark/lib_spark_2.4.3_hadoop_2.7.0
    Overwrite the existing hive-exec-1.2.1-spark2-amzn-1.jar file in the directory.
  2. Copy the aws-glue-datacatalog-spark-client-1.11.0.jar file from the
    /usr/share/aws/hmclient/lib
     directory of the Glue-enabled EMR 5.29 cluster to the following location:
    <Informatica installation directory>/services/shared/spark/lib_spark_2.4.3_hadoop_2.7.0

Configure Glue as the Hive Metastore

Enable support for Glue as the Hive metastore by setting the hive.metastore.uris property in the hive-site.xml cluster configuration file:
  1. Choose one of the following methods to edit the hive-site.xml settings:
    • Retrieve an archived version of the cluster configuration .zip archive and edit the hive-site.xml file within it. You can choose this method if the settings in the .zip archive are still the same as the configuration settings on the cluster.
    • Copy the cluster *-site.xml configuration files to your local machine, edit the hive-site.xml file, and then create the .zip archive. Choose this method if you do not have an existing .zip archive or if the cluster configuration settings have changed since the .zip archive was created.
    In either case, do not add the hive.metastore.uris property to hive-site.xml on the cluster.
  2. Edit the hive-site.xml file to add the hive.metastore.uris property:
    Property
    Value
    hive.metastore.uris 
    thrift://<Hive host name>:<port>
  3. Create a .zip archive that contains the set of *-site.xml files, or save the edited .zip archive.
  4. Refresh the cluster configuration on the domain.
    The hive.metastore.uris property is added to the cluster configuration and the associated Hive connection.
For more information about the .zip archive and using it to refresh the cluster configuration, see the cluster configuration chapter in the
Data Engineering Administrator Guide.

More Information

For more information about Amazon Glue, see the following Amazon documentation:
For information about Informatica support for Amazon Glue, see the Product Availability Matrix at https://network.informatica.com/community/informatica-network/product-availability-matrices.
0 COMMENTS
We’d like to hear from you!