Table of Contents

Search

  1. Preface
  2. Introduction to Data Engineering Streaming
  3. Data Engineering Streaming Administration
  4. Sources in a Streaming Mapping
  5. Targets in a Streaming Mapping
  6. Streaming Mappings
  7. Window Transformation
  8. Appendix A: Connections
  9. Appendix B: Monitoring REST API Reference
  10. Appendix C: Sample Files

Authentication Systems on Hadoop

Authentication Systems on Hadoop

You can run streaming mappings on a Hadoop cluster that uses a supported security management system.
Hadoop clusters use a variety of security management systems for user authentication. The following table shows the Spark engine support for the security management system installed on the Hadoop platform:
Hadoop Distribution
HDFS TE
Kerberos
LDAP
SSL/TLS
Amazon EMR
Not supported
Supported
Not supported
Not supported
Azure HDInsight
Not supported
Not supported
Not supported
Not supported
Cloudera CDH
Supported
Supported
Supported
Supported
Hortonworks HDP
Supported
Supported
Supported
Supported
MapR
Not supported
Supported
Not supported
Not supported

Additional Information for Authentication

Consider the following additional security support statements:
  • Apache Knox is not available in any of the mentioned Hadoop distributions.
  • Sqoop cannot access Kerberos or SSL enabled databases.
For more information about authentication systems, see
Data Engineering Administrator Guide
.