Example: Fix a failed connection with Claude
A connection that worked has stopped. Claude uses Monte Carlo's validation to find which part fails, and gives you AWS CLI checks that pinpoint the cause without exposing any secret.
In this example a Postgres connection on an AWS collection agent stopped working after someone edited the agent's IAM policy by hand. Claude narrows the problem down with Monte Carlo's validation tools, you confirm the cause in AWS, and the connection validates again. Nothing changes in Monte Carlo: the fix is on the AWS side.
| Who | What | How |
|---|---|---|
| Claude | Validate the connection and its credentials, and compare with a healthy connection on the same agent | validate_connection, validate_aws_secrets_manager_credentials, get_validation_run |
| You | Run the checks Claude gives you: what changed, and can the agent read the secret | AWS CLI (CloudTrail, IAM policy simulator) |
| You | Apply the fix Claude writes, after reviewing a diff | AWS CLI (IAM) |
| Claude | Re-validate and confirm | validate_connection, get_validation_run |
1. Ask Claude
You don't need a special command; describe the problem:
Our Postgres connection in Monte Carlo (<connection-name>) stopped working. Can you find out why and help me fix it?
2. Claude finds which part fails
Claude runs the connection's validation, then validates its credentials on their own. In this example:
| Check | Result |
|---|---|
| Customer managed credentials | ⚠️ "Customer managed credentials are invalid or unreachable. Check that the agent has permissions to read the secret and that the credentials JSON matches the documented schema." |
| Connect to database | ❌ "Could not connect to the database." |
| Storage, projects, tables | Skipped, because the connect check failed |
It then validates another connection on the same agent. That one passed, which rules out the agent itself and points at this connection's secret: it's missing, its content changed, the agent can no longer read it, or the database became unreachable.
3. Find what changed and confirm the cause
These checks read only metadata and audit logs, never the secret's value.
The secret itself didn't change. Check that it exists, when it last changed, and whether rotation is on:
REGION=<region>
aws secretsmanager describe-secret --region "$REGION" --secret-id <secret-name> \
--query '{ARN:ARN,Deleted:DeletedDate,LastChanged:LastChangedDate,Rotation:RotationEnabled}'What changed since the connection last worked. CloudTrail lists every write in that window. Use fixed timestamps in UTC:
aws cloudtrail lookup-events --region "$REGION" \
--lookup-attributes AttributeKey=EventName,AttributeValue=PutRolePolicy \
--start-time <last-worked-utc> \
--query 'Events[].CloudTrailEvent' --output text \
| jq -r '[.eventTime, .requestParameters.roleName, .requestParameters.policyName] | @tsv'To see every change, not just IAM policy writes, look up by AttributeKey=Username and filter out the read-only events (Describe*, Get*, List*). In this example it showed a PutRolePolicy on the agent's execution role one minute before the connection first failed.
Can the agent read each secret right now? The IAM policy simulator answers directly. With more than one secret, read ResourceSpecificResults:
ROLE=<agent-execution-role-name>
aws iam simulate-principal-policy \
--policy-source-arn "$(aws iam get-role --role-name "$ROLE" --query Role.Arn --output text)" \
--action-names secretsmanager:GetSecretValue \
--resource-arns <failing-secret-arn> <working-secret-arn> \
--query 'EvaluationResults[].ResourceSpecificResults[].[EvalResourceName,EvalResourceDecision]' \
--output textHere the failing secret showed implicitDeny and the working one allowed: the policy edit had dropped the Postgres secret's ARN.
4. Apply the fix
First list the role's inline policies (aws iam list-role-policies --role-name "$ROLE") and tell Claude which one grants the secrets: grants are often split across several policies. Claude then gives you an edit you review before anything is written:
POLICY=<inline-policy-name>
PG_ARN=<failing-secret-arn>
# 1. Back up the current policy
aws iam get-role-policy --role-name "$ROLE" --policy-name "$POLICY" \
--query PolicyDocument --output json > policy-backup.json
# 2. Add the missing ARN, keeping what's there
jq --arg pg "$PG_ARN" '
.Statement[0].Resource = ([.Statement[0].Resource] | flatten + [$pg] | unique)
' policy-backup.json > policy-new.json
# 3. Review: the only change should be the added ARN
diff <(jq -S . policy-backup.json) <(jq -S . policy-new.json)
# 4. Apply, then check every secret the agent uses
aws iam put-role-policy --role-name "$ROLE" --policy-name "$POLICY" \
--policy-document file://policy-new.jsonRun the simulator from step 3 again with all of the agent's secrets; each should show allowed. To undo, run put-role-policy with file://policy-backup.json. If the policy is managed by Terraform or CloudFormation, make the change there instead, or the next apply reverts it.
5. Claude re-validates
Tell Claude the fix is applied. It re-runs the validation:
| Check | Result |
|---|---|
| Customer managed credentials | ✅ Valid |
| Connect to database | ✅ |
| Storage bucket | ✅ |
| Projects | ✅ Found database <database> |
| Tables | ✅ Found schema <schema> and its tables |
The connection's metadata and query jobs pick up again on their next scheduled run.
What to avoid while troubleshooting
- Don't run
aws secretsmanager get-secret-valueto test access, even piped intojq: it pulls the password into your shell. The IAM simulator and Monte Carlo's validation test access without the value. - Don't rely on the agent's Lambda logs to find the cause: the agent redacts its error messages. Monte Carlo's validation results and CloudTrail are the useful sources.
- Don't change the connection in Monte Carlo unless validation points at it: most failures of a connection that used to work are on the cloud side.
Troubleshooting guide
| Validation shows | Likely cause | Check |
|---|---|---|
| Credentials warning + "Could not connect", another connection on the same agent works | The agent can no longer read this secret: an IAM policy edit, a customer-managed KMS key, or a secret resource policy | simulate-principal-policy per secret; describe-secret (KmsKeyId); get-resource-policy; CloudTrail PutRolePolicy |
| Same, and the secret's ARN changed | The secret was deleted and recreated (new 6-character suffix) | describe-secret; ask Claude to point the credentials at the new ARN |
| Every connection on the agent fails | The agent itself: stack update, VPC or NAT change | CloudTrail for the stack and the agent's subnets |
| "Could not connect" and a slow failure, credentials valid | The network path: a security group or route changed, or the database moved | The database's security group inbound rules; the database's status and endpoint |
| Connects but fails Tables | Grants were revoked, or the tables moved | The database's grants for the user in the secret |
If none of these explains it, Claude gives you the connection and validation run IDs to share with Monte Carlo support, who can see the agent's unredacted errors.
Updated about 1 hour ago
