Amazon S3 Connector Guide

Back Next

S3 Connector Implementation

The S3 Connector provides configurable start events (Event Sources) and event services (Event Targets) that can be defined in a Connection object using Process Designer.

The Event Sources (start events) and Event Targets (event service definitions) share common attributes, so a single S3 Connection can monitor several different buckets for new objects, read delimited content from other buckets, and write to another bucket, provided that all the buckets are related to a single access/secret key pair (the S3 account).

With this flexible implementation, you can combine a set of related tasks in one connection or split your work into several separate connections.

Processing S3 Objects with Event Sources

After the S3 Connector processes the S3 objects in a bucket, those objects are automatically deleted to ensure they are not processed again. As a result, S3 buckets used with the connector must be defined either as temporary storage or as a data exchange buffer between an isolated application and a Secure Agent, from which the objects can be processed and moved to another location.

The connector handles each object as follows:

The start event (Event Source) you define reads the S3 object from the bucket and sends it to the processes handling the content.

After the content has been processed, the S3 object is removed from the bucket and the event source reads the next object.

If a problem is encountered during object processing, the processing terminates and an error message is printed to the log file. An error might be occur if:

An S3 object causes an error during processing and cannot be deleted because its content was not processed.

An S3 object cannot be moved to the local folder and deleted it from the bucket because it might cause an unexpected result. For example, if there is an error in the configuration of the connection where the process is running.

The Event Source settings have an incorrect encryption algorithm or password.

The connection is attempting to read encrypted S3 objects without first decrypting them. This is not permitted because the encryption keys would be removed from the bucket along with the original objects.

Encryption Method

Amazon S3 Connector supports client-side encryption using a master key that you store within your application.

When you upload an object to Amazon S3, you must provide a client-side master key. Amazon S3 Connector uses the master key to encrypt the encryption data key. Then, Amazon S3 Connector uses the encryption data key to encrypt the object.

When you download an object from Amazon S3, Amazon S3 Connector downloads the encrypted object from Amazon S3. Amazon S3 Connector uses the master key to decrypt the encryption data key. Then, Amazon S3 Connector uses the encryption data key to decrypt the object.

For more information about using a client-side master key, see the Amazon S3 user documentation.

If you try to consume encrypted objects from Amazon S3 without enabling data encryption in the Amazon S3 connection, the connection fails.

Encryption Algorithms

The implementation is based on the S3 client-side encryption. The S3 Connector supports several different encryption algorithms. The encryption settings are used for both encryption of Event Sources and decryption of Event Targets. Before you can read an encrypted S3 object, you must
decrypt the object.

For keys generated by the connection and based on the user password string, the encryption key is based on the PBKDF2WithHmacSHA1 key derivation algorithm with 4096 iterations, using the fixed salt, "Informatica".

For a user-provided encryption key, any key is acceptable provided the key length is correct.

Also see the Client-Side Encryption Settings below.

Java Cryptography Extension (JCE)

To use the encryption algorithms, you might need to install the Java Cryptography Extension (JCE). You can download the JCE from the Oracle website. To install it:

Extract the files from the JCE zip file you downloaded.

Copy the following jar files to $JAVA_HOME/jre/lib/security:

local_policy.jar

US_export_policy.jar

If these jar files already reside in the specified directory, make a backup copy and then
overwrite
them.

Restart your Secure Agent to begin using the new jar files.

After you perform these steps, you should be able to use all the encryption algorithms supported by the S3 Connector.

Introduction to Amazon S3 Connector

Download Guide

Watch

Comments

Cloud Application Integration Homepage