Lambda Infinite Loop with S3: How to Prevent Recursive Triggers
A Lambda infinite loop with S3 occurs when your function writes output back to the same bucket that triggers it, creating a self-perpetuating cycle that burns through invocations and cost.
TL;DR: Stopping the Lambda Infinite Loop with S3
| Solution | Mechanism | Best For | Complexity |
|---|---|---|---|
| Separate input/output prefixes with scoped event filter | S3 event notification prefix filter limits trigger to one folder | Same-bucket architecture requirement | Low |
| Separate input/output S3 buckets | Output bucket has no event notification configured | Clean architectural separation | Low |
| Idempotency guard via object metadata or DynamoDB | Lambda checks a processed flag before acting | Complex pipelines where prefix separation is insufficient | Medium |
Why the Lambda Infinite Loop with S3 Happens
The trigger model is deceptively simple: S3 detects a new object, fires an event notification, and Lambda responds. The problem is that S3 event notifications are object-level, not intent-level. When your function writes its output back to the same bucket — even to a different key — S3 sees a new ObjectCreated event and fires the notification again. Lambda has no built-in awareness that it caused the write.
This is the cycle that forms.
- Upload: A user or process puts an object into the S3 bucket.
- Event fires: S3 emits an
ObjectCreatednotification to Lambda. - Lambda processes: The function reads the object and writes output back to the same bucket.
- Loop re-triggers: The output write generates a new
ObjectCreatedevent, invoking Lambda again. - Runaway state: Each invocation produces another write, and the cycle continues until you intervene or hit a concurrency limit.
Analogy: Imagine a photocopier that automatically copies every document placed in its output tray. The first original produces one copy — which lands in the output tray, triggering another copy, which triggers another. The machine never stops because it cannot distinguish originals from its own output.
Emergency Stop: Halt the Runaway Loop First
Before applying any architectural fix, stop the bleeding. An active recursive loop will continue consuming concurrency and generating cost while you work on the solution.
The fastest lever is throttling the function to zero concurrent executions. This does not delete the function or its configuration — it simply prevents new invocations from starting.
aws lambda put-function-concurrency \
--function-name your-function-name \
--reserved-concurrent-executions 0Once the loop is stopped, verify no invocations are still running by checking the Lambda console or CloudWatch metrics for active executions. Then apply one of the solutions below before restoring concurrency.
To restore concurrency after your fix is in place:
aws lambda delete-function-concurrency \
--function-name your-function-nameDecision Guide: Choosing the Right Fix
Solution 1: Separate Input and Output Buckets (Recommended)
The cleanest architectural fix is to write Lambda output to a different S3 bucket — one that has no event notification configured. This eliminates the feedback loop at the infrastructure level without requiring any code changes inside the function.
Structural separation is the correct default. Only deviate from it when a business or operational constraint forces same-bucket writes.
Update your Lambda environment variable or application configuration to point output writes at the destination bucket, and confirm that the destination bucket has no S3 event notification that targets your Lambda function.
Verify the destination bucket's notification configuration is empty:
aws s3api get-bucket-notification-configuration \
--bucket your-output-bucket-nameAn empty response (or a response with no LambdaFunctionConfigurations key) confirms no recursive trigger exists on that bucket.
Solution 2: Prefix Filtering on the Same Bucket
When same-bucket architecture is a hard requirement, S3 event notification prefix filters are the standard mitigation. You configure the notification to fire only for objects under a specific prefix (e.g., input/), and your Lambda writes output exclusively to a different prefix (e.g., output/). Because the output prefix is not covered by the event filter, no notification fires for those writes.
This approach works — but it depends entirely on the Lambda function never writing to the input prefix. That discipline must be enforced in code, not just assumed.
🔽 Click to expand — S3 notification configuration with prefix filter (AWS CLI)
aws s3api put-bucket-notification-configuration \
--bucket your-bucket-name \
--notification-configuration '{
"LambdaFunctionConfigurations": [
{
"LambdaFunctionArn": "arn:aws:lambda:us-east-1:123456789012:function:your-function-name",
"Events": ["s3:ObjectCreated:*"],
"Filter": {
"Key": {
"FilterRules": [
{
"Name": "prefix",
"Value": "input/"
}
]
}
}
}
]
}'After applying this configuration, verify it was saved correctly:
aws s3api get-bucket-notification-configuration \
--bucket your-bucket-nameConfirm the FilterRules show prefix: input/ in the response before restoring Lambda concurrency.
Solution 3: Idempotency Guard in Lambda Code
Prefix filtering stops the trigger at the infrastructure layer. But for pipelines where multiple Lambda functions write to overlapping prefixes, or where the output prefix structure cannot be cleanly separated, an in-function idempotency check provides a code-level safety net.
The pattern: before processing any object, Lambda checks whether it has already been processed. If yes, it exits immediately without writing output.
Two common implementations:
Option A — S3 Object Metadata tag: After processing, tag the output object with a custom metadata key (e.g., processed=true). On each invocation, read the object's tags first. If the tag exists, return early.
Option B — DynamoDB processed-key table: Write the S3 object key to a DynamoDB table after successful processing. On each invocation, check the table first. If the key exists, return early. This approach survives Lambda retries and concurrent invocations more reliably than metadata tagging alone.
🔽 Click to expand — Python idempotency guard using S3 object tagging
import boto3
s3 = boto3.client('s3')
def lambda_handler(event, context):
bucket = event['Records'][0]['s3']['bucket']['name']
key = event['Records'][0]['s3']['object']['key']
# Check if already processed
try:
tagging = s3.get_object_tagging(Bucket=bucket, Key=key)
tags = {t['Key']: t['Value'] for t in tagging['TagSet']}
if tags.get('processed') == 'true':
print(f'Skipping already-processed object: {key}')
return
except s3.exceptions.NoSuchKey:
return
# --- Your processing logic here ---
# Mark as processed after successful write
s3.put_object_tagging(
Bucket=bucket,
Key=key,
Tagging={'TagSet': [{'Key': 'processed', 'Value': 'true'}]}
)The idempotency guard is a defense-in-depth layer, not a replacement for prefix filtering or bucket separation. Use it in addition to an infrastructure-level fix, not instead of one.
IAM Least Privilege for This Pattern
Whichever solution you choose, scope the Lambda execution role to the minimum required permissions. A function that only needs to read from the input prefix and write to the output prefix should not have broad s3:* access across the entire bucket.
🔽 Click to expand — Least-privilege IAM policy example
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": ["s3:GetObject"],
"Resource": "arn:aws:s3:::your-bucket-name/input/*"
},
{
"Effect": "Allow",
"Action": ["s3:PutObject"],
"Resource": "arn:aws:s3:::your-bucket-name/output/*"
}
]
}Restricting s3:PutObject to the output/ prefix means that even if a code bug attempts to write to input/, the IAM policy blocks it. This is a structural enforcement of the prefix discipline that prefix filtering alone cannot provide.
Production Gotcha: The Loop That Looks Like Normal Traffic
Engineers debugging this for the first time typically open CloudWatch and see Lambda invocation counts climbing steadily. The natural first assumption is a traffic spike — maybe a batch upload job ran. They check S3 access logs, see hundreds of PUT requests, and assume external clients are uploading at high volume.
What the logs actually show is Lambda writing to itself. The PUT requests in S3 access logs will show the Lambda execution role as the requester principal — not an external IAM user or assumed role from an application. That single detail is the diagnostic signal that separates a real traffic spike from a recursive loop.
Check the requester field in S3 server access logs or CloudTrail PutObject events. If the principal matches your Lambda execution role ARN, the function is the source of its own triggers.
The loop is always visible in the logs — engineers just look at the wrong column first.
Depth: Why Suffix Filters Alone Are Insufficient
A pattern that appears frequently in online examples is using a suffix filter (e.g., .csv) to limit which uploads trigger Lambda, with the assumption that writing a .json output file won't match the filter. This works only if the output file format never matches the input suffix.
When a processing pipeline transforms input.csv into output.csv — both with the same extension — the suffix filter provides zero protection. The output write matches the filter and re-triggers the function. Prefix filtering on a dedicated input folder is structurally safer than suffix filtering on file type, because it ties the trigger scope to an explicit organizational boundary rather than a file naming convention that can drift.
Suffix filters are not a reliable recursive-loop prevention mechanism when input and output share the same file extension.
Wrap-Up: Eliminating the Lambda Infinite Loop with S3 Permanently
The recursive trigger problem is architectural, not a Lambda bug. The correct resolution hierarchy is: separate buckets first, prefix filtering second, idempotency guard as defense-in-depth. Applying all three layers gives you infrastructure-level prevention, code-level safety, and IAM enforcement — any one of which would stop the loop independently.
For further reading, see the official AWS documentation on S3 event notification filtering and Lambda reserved concurrency.
Glossary
| Term | Definition |
|---|---|
| S3 Event Notification | A configuration that instructs S3 to invoke a target (Lambda, SQS, SNS) when specific object operations occur in a bucket. |
| Prefix Filter | An S3 event notification rule that restricts trigger scope to objects whose keys begin with a specified string (e.g., input/). |
| Reserved Concurrency | A Lambda setting that caps the maximum number of simultaneous executions for a specific function, including setting it to zero to halt all invocations. |
| Idempotency Guard | A code or data pattern that ensures a Lambda function produces no side effects when invoked more than once for the same input event. |
| Execution Role | The IAM role assumed by a Lambda function at runtime, which determines what AWS API actions the function is authorized to perform. |
Comments
Post a Comment