Scenario Two

Follow these steps to enable S3 events if your needs fit under scenario two:

  1. Retrieve relevant SQS ARNs
  2. Retrieve your account ID
  3. Open the S3 event management pane
  4. Create a SNS Topic
  5. Update the SQS access policy
  6. Create event notification
  7. Create a SNS subscription

Retrieve relevant SQS ARNs

Follow these steps to get the relevant SQS ARNs. If the data collector is managed by Monte Carlo, please reach out to your representative for these values instead.

  1. Open the Cloudformation console and search for the Monte Carlo data collector. Select the stack
3220
  1. Select the β€œOutputs” tab
2090
  1. Save the Metadata Queue ARN for later
    Key: MetadataEventQueue

Retrieve your account ID

Follow these steps to retrieve your account ID. If the data collector is managed by Monte Carlo, please reach out to your representative for these values instead.

Be sure you are logged in the same account as the Monte Carlo Collector before proceeding.

  1. From the console, select your username in the upper right corner.
  2. Select β€œMy Account”.
  3. Save the Account Id (without dashes) for later.

Open the S3 event management pane

Follow these steps to help locate the event configuration page for the bucket you want to enable events for.

  1. Open the S3 Console and search for the bucket that you would like to enable events for
  2. Select the bucket
  3. Save the bucket ARN by selecting β€œCopy Bucket ARN” for later.
  4. Select the β€œProperties” tab. Leave this page open you will come back to it later.

Create a SNS Topic

πŸ“˜

What region should I create my topic in?

Make sure you are in the same region as the bucket you want to add an event for.

  1. Open the SNS console and select β€œTopics”
  2. Select β€œCreate Topic”. Choose "Standard" type, enter a meaningful name and fill any optional fields

🚧

It’s highly recommended to enable delivery status logging for SQS.

  1. Select β€œCreate Topic” and save the Topic ARN for later.
  2. Update (append) the topic you just created with the following policy statement:
  • SNS_ARN is the the ARN from above
  • S3_ARN is the bucket ARN, which you saved in the "Locate the S3 event management pane" subsection
{
	"Effect": "Allow",
	"Principal": {
		"AWS": "*"
	},
	"Action": "SNS:Publish",
	"Resource": "<SNS_ARN>",
	"Condition": {
		"StringEquals": {
			"aws:SourceArn": "<S3_ARN>"
		}
	}
}

You may need to include a "Sid" here too.

  1. Save changes

Update the SQS access policy

Follow these steps to allow your SNS topic to write to the relevant queue. If the data collector is managed by Monte Carlo these steps can be skipped by just sending the SNS Topic ARN to your representative. Your representative will in turn send you the SQS ARN and relevant account ID.

  1. Open the SQS console in the account the Monte Carlo Collector was deployed to
  2. Search for the queue. The name follows this structure: {CF_STACK}-MetadataEventQueue-{RANDOM_STR}
  3. Select the queue and confirm the the ARN matches the ARN you saved previously
  4. Select the β€œAccess Policy” Tab and Select β€œEdit”.

If the access policy is empty or looks something like this:

{
  "Version": "2012-10-17",
  "Id": "arn:aws:sqs:<region>:<account>:<name>/SQSDefaultPolicy"
}

Paste the following (replacing any values in brackets):

  • The COLLECTOR_ACCOUNT_ID is the account ID you saved in the "Retrieve your account ID" subsection
  • The EVENT_QUEUE_ARN is the ARN you saved in the "Retrieve relevant SQS ARNs subsection"
  • The SNS_ARN is the SNS ARN, which you saved in the "Create a SNS Topic" subsection

🚧

Be sure to use the SNS topic ARN and not the S3 bucket ARN here

{
   "Version":"2008-10-17",
   "Statement":[
      {
         "Sid":"__owner",
         "Effect":"Allow",
         "Principal":{
            "AWS":"arn:aws:iam::<COLLECTOR_ACCOUNT_ID>:root"
         },
         "Action":"SQS:*",
         "Resource":"<EVENT_QUEUE_ARN>"
      },
      {
         "Sid":"__sender",
         "Effect":"Allow",
         "Principal":{
            "AWS":"*"
         },
         "Action":"SQS:SendMessage",
         "Resource":"<EVENT_QUEUE_ARN>",
         "Condition":{
            "ArnLike":{
               "aws:SourceArn":[
                  "<SNS_ARN>"
               ]
            }
         }
      }
   ]
}

But, if the access policy already has a SID with β€œ__sender” (i.e. looks like above) append your SNS_ARN to the SourceArn list instead. The SNS_ARN was saved in the "Create a SNS Topic" subsection.

"aws:SourceArn": [
            "arn:aws:s3:::existing_bucket",
            "<SNS_ARN>"
          ]

Create event notification

Follow these steps to create an event notification in S3.

  1. Return to the page you had opened in step 4 of the "Open the S3 event management pane" subsection.
  2. Select β€œCreate event notification” under Event notifications.
  3. Fill in a meaningful name.
  4. Optionally specify a prefix and/or suffix.
  5. Select β€œAll object create events” and β€œAll object delete events” under Event types.
1590
  1. Enter the SNS queue ARN you had saved from the "Create a SNS Topic" subsection as the Destination topic ARN.
1562
  1. Save changes.

Create a SNS subscription

Follow these steps subscribe SQS to the SNS topic you created and enabled notifications for.

  1. Open the SNS console and select β€œSubscriptions”.
  2. Select β€œCreate Subscription”.
  3. Select the topic ARN you saved above in the "Create a SNS Topic" subsection.
  4. Select Amazon SQS as the protocol
  5. Select (or paste) the SQS ARN you saved above in the "Retrieve relevant SQS ARNs" subsection.
2206

🚧

Be sure to select β€œEnable raw message delivery”

  1. Select β€œCreate Subscription”
  2. Validate the status is β€œConfirmed”