Overview
Used to output data from DDE to a file in an AWS S3 Bucket
Can be used as an integration point with third party systems e.g. Data warehouse etc
Plugin Configuration
General
Within the General tab:
The following settings can be configured:
Setting | Description |
|---|---|
Access key | The AWS Access Key used to authenticate the requests sent to Amazon S3 |
Secret key | The AWS Secret Key used to authenticate the requests sent to Amazon S3 |
Bucket | The S3 buckets that the data should be stored in |
Bucket Region | The region the bucket is stored in e.g. eu-west-1 |
File prefix | The prefix of the filename output to S3 |
File prefix (Field/Param) | Allows you to add a File prefix via a Field or Parameter |
Include Date/Time | Allows the selection of various Date / Time values to add to the Filename* |
StepNr | Adds the start step in case more than one step could start the KTR file. This provides another element of uniqueness. |
Node Name | Adds the Node Name (IP) to the File name. This provides another element of uniqueness if files are being output quickly from different nodes. |
Folder (optional) | The sub-folder within the S3 bucket where the data should be output |
Extension | The extension of the file that should be output e.g. csv, txt |
Buffer Size | During rule development it is useful to have a fast buffer retention period so that data written out by rules appears frequently in the S3 bucket. However when the rule is in production a short buffer period can cause a large number of files to accumulate. This Buffer Size setting allows you to adjust the Buffer and the frequency the data is passed to S3 |
Time Period (Mins) | Allows a time period to be set to control when the data is written to S3 |
This was made configurable in August 2020. Prior to this the mandated Date / Time value made duplicates filenames extremely unlikely and therefore files could not be overwritten. Being able to set this and other values and choose to overwrite a file was advantageous so these changes were made. Customers should check the filename selections to ensure they are unique enough in a CX environment to not overwrite if this is not required.
Content
Within the Content tab:
The following settings can be configured:
Setting | Description |
|---|---|
Separator | The delimiter that is used to separate the values within the file.
 To use a TAB as a delimiter select the “Insert TAB” button |
Enclosure | The data qualifier enclosing the data
|
Header | If “Header” is selected the first row of data within the output file created will contain the field names output. |
Fields
The Fields tab is required to be populated with the fields that are required to be output to the file:
Example
The following table illustrates an example of how data will be exported to S3 with certain setting applied:
Content | Fields | Example Data Output |
|---|---|---|
 |  | Field1,Field2,Field3 1,"A","A" 2,"B","B" 3,"C","C" |
Understanding how files are formatted
The rule files will be formatted as you specify in the step configuration, however the file names will be different when executed within . Example:
Description | File name |
|---|---|
File prefix | MyFile_clientname_s3output_3_172.31.11.190_20180202_114856_0_6.txt |
DDE Client name | MyFile_clientname_s3output_3_172.31.11.190_20180202_114856_0_6.txt |
Rule name | MyFile_clientname_s3output_3_172.31.11.190_20180202_114856_0_6.txt |
Rule version | MyFile_clientname_s3output_3_172.31.11.190_20180202_114856_0_6.txt |
Node Name (internal) IP address | MyFile_clientname_s3output_3_172.31.11.190_20180202_114856_0_6.txt |
Inlcude Date / Time stamp | MyFile_clientname_s3output_3_172.31.11.190_20180202_114856_0_6.txt |
File count (Split every X rows) | MyFile_clientname_s3output_3_172.31.11.190_20180202_114856_0_6.txt |