Amazon S3 Connector Guide

Back Next

Amazon S3 Event Source Properties

After you define an Amazon S3 connection and deploy it in a process, you can monitor a bucket for new objects. When the S3 Connector detects a new object, it reads the S3 object and sends it to the process that subscribed to the Event Source, which acts as a start event.

For each connection, you can add multiple event sources.

A connection begins processing objects in the bucket immediately after it is published, even if you have not yet defined any processes to consume or subscribe to the events. You can use the initial delay setting to postpone processing the first bucket until you have published a process to handle events from the Event Source.

The following table describes the file location settings applicable to all S3 event source types:

Property	Description
S3 Bucket Name	Required. Amazon S3 bucket name that contains the objects to be processed by the event source. The question mark (?) character is prohibited in this field.
S3 Prefix	The S3 prefix that determines which objects are processed. Event source can use this attribute to consume only subset of S3 objects that share common prefix. If no prefix is specified, the event source reads all objects in the bucket.

The following table describes the

Object Operations Settings

applicable to all S3 event source types:

Property	Description
Delete S3 Objects	Required. Use this property to set whether you want to delete none, some, or all processed S3 objects from the source bucket. Choose one of the following options: No: Do not delete any processed S3 object. Consumed: Delete all successfully processed S3 objects. All: Delete all, faulted or successfully processed, S3 objects. If you select No or Consumed , the connector uses the move settings to prevent an object from being consumed twice. If you have not set a Move to Bucket name, the connector uses the default bucket. For example, if you select No , the connector uses the Move To Bucket and Move Prefix properties to move processed S3 objects. Default: No.
Move To Bucket	Optional. Specify the bucket to which you want to move successfully processed S3 objects. If you do not specify a Move Prefix , it is a good practice to specify a move to bucket different from the source bucket. Default: Source bucket.
Move Prefix	Optional. Specify the prefix you want to add to the keys of successfully processed S3 objects. If you want to move processed objects to the source bucket, it is a good practice to specify a move prefix. Default: .ae-done/
Move Failed to Bucket	Optional. Specify the bucket to which you want to move failed S3 objects. If you do not specify a Move Failed Prefix , it is a good practice to specify a move failed to bucket that is different from the source bucket and the move to bucket. Default: Source bucket.
Move Failed Prefix	Optional. Specify the prefix you want to add to the keys of failed S3 objects. If you want to move failed objects to the source bucket, it is a good practice to specify a move prefix. Default: .ae-error/
Backup In Folder	Optional. Specify a local directory in which you want to store a copy of successfully processed S3 objects.
Backup Failed In Folder	Optional. Specify a local directory in which you want to store a copy of failed S3 objects.

The following table describes the

Failure Handling Settings

applicable to all S3 event source types:

Property	Description
Max Retry Interval	In case of a fault, specify the maximum time (in seconds) that you want an event source to wait before making a recovery attempt. The delay interval between retries increases exponentially until the connector reaches the maximum retry interval. Default maximum: 3600 seconds or 1 hour. Minimum allowed: 1 second Maximum allowed: 86400 seconds or 24 hours.

The following table describes the polling and other settings applicable to all S3 event source types:

Property	Description
Initial Delay	Seconds to wait before polling the S3 bucket. Default: 1 second. Set the initial delay that you need, for example, to publish the related process or take other required steps before polling can begin.
Delay	Seconds to wait before the next poll to check whether new files have arrived. Default is 1 second. Set a longer delay between polls to reduce the number of S3 API calls.
Max Messages Per Poll	The maximum number of objects to retrieve in each poll. Default: 10. Set a value that enables you to segment the load and manage performance if you anticipate that the monitor will find a large number of objects to process during each poll.