Table of Contents

Search

  1. Preface
  2. Introduction to Data Engineering Streaming
  3. Data Engineering Streaming Administration
  4. Sources in a Streaming Mapping
  5. Targets in a Streaming Mapping
  6. Streaming Mappings
  7. Window Transformation
  8. Appendix A: Connections
  9. Appendix B: Monitoring REST API Reference
  10. Appendix C: Sample Files

Configuration for Amazon Kinesis Streams Sources

Configuration for Amazon Kinesis Streams Sources

To use Amazon Kinesis Streams as sources, grant required permissions to the user. To use cross-account IAM role, create an IAM role and grant access to the role.
To configure access for Amazon Kinesis Streams as a source, perform the following tasks :
  • Grant consumer permissions that are part of the IAM policy to the AWS credentials that the IAM user specifies in the access key id.
    For the list of permissions, see the AWS documentation at https://docs.aws.amazon.com/streams/latest/dev/learning-kinesis-module-one-iam.html
  • Grant the following permissions to the user to fetch metadata:
    • kinesis:DescribeStream
    • kinesis:GetShardIterator
    • kinesis:GetRecords
  • To use cross-account IAM role, create an IAM role in an AWS account. IAM roles are used to provide secure access to AWS resources. The role is used to establish a trusted relationship between various AWS accounts. Additional restrictions are enforced using an external ID and the IAM role can only be assumed by using the external ID if the external ID is specified for a cross-account IAM role.
    Multi-factor authentication is not supported.