Lambda Connecting to Private RDS: VPC Configuration Explained

One of the most common connectivity failures in serverless architectures is a Lambda function timing out when trying to reach an RDS instance sitting inside a private subnet — the root cause almost always comes down to missing or misconfigured VPC settings on the Lambda side.

TL;DR

Question Answer
Does Lambda need VPC config to reach private RDS? ✅ Yes — you must attach Lambda to the same VPC and configure subnets + security groups.
Which subnets should Lambda use? Private subnets (same or routable to RDS subnets). Use multiple AZs for resilience.
What security group rule is required? RDS security group must allow inbound on the DB port (e.g., 5432/3306) from the Lambda security group.
Does VPC-attached Lambda lose internet access? Yes, by default. Add a NAT Gateway if outbound internet is needed.
What IAM permission is needed? Lambda execution role needs ec2:CreateNetworkInterface, ec2:DescribeNetworkInterfaces, ec2:DeleteNetworkInterface — provided by the AWS managed policy AWSLambdaVPCAccessExecutionRole.

Why This Happens: The Network Isolation Problem

By default, a Lambda function runs inside an AWS-managed VPC that has no visibility into your private VPCs. Your RDS instance in a private subnet has no public IP and no route to the outside world — it is intentionally isolated. These two worlds cannot communicate unless you explicitly bridge them by placing Lambda inside your VPC.

Real-World Analogy: Think of your private VPC as a gated corporate office building. RDS is a server locked in the internal server room. A default Lambda function is like a contractor working from a coffee shop — they can reach the public internet, but the server room door won't open for them. Attaching Lambda to your VPC is like giving that contractor a building access badge and assigning them a desk inside the office. Only then can they walk to the server room.

Architecture: How VPC-Attached Lambda Reaches Private RDS

graph LR Trigger["Trigger
(API GW / EventBridge)"] --> Lambda["Lambda Function"] Lambda -->|"AWS injects ENI"| ENI["ENI
(Private IP in your VPC)"] ENI -->|"Private subnet routing"| SG_Check["RDS Security Group
Inbound Rule Check"] SG_Check -->|"Port 5432/3306 allowed
from Lambda SG"| RDS[("RDS Instance
Private Subnet")] subgraph "Your VPC" ENI SG_Check RDS end subgraph "AWS Managed VPC" Lambda end
  1. Lambda Invocation: Your function is triggered (API Gateway, EventBridge, etc.).
  2. ENI Injection: AWS creates an Elastic Network Interface (ENI) in your specified private subnet, giving Lambda a private IP inside your VPC.
  3. Security Group Evaluation: Traffic from the Lambda ENI hits the RDS security group. The inbound rule must explicitly allow the DB port from the Lambda security group (or its CIDR).
  4. Private DNS Resolution: Lambda resolves the RDS endpoint (e.g., mydb.xxxx.us-east-1.rds.amazonaws.com) to a private IP via Route 53 private hosted zones — no public routing involved.
  5. TCP Connection Established: The database connection is made entirely within the VPC's private network fabric.

Security Group Configuration: The Critical Rule

This is where most engineers make mistakes. You need two security groups configured correctly:

graph TD LambdaSG["sg-lambda
(Lambda Security Group)"] RDSSL["sg-rds
(RDS Security Group)"] LambdaSG -->|"Outbound: TCP 5432
Destination: sg-rds"| RDSSL RDSSL -->|"Inbound Rule:
TCP 5432
Source: sg-lambda ✅"| RDS[("RDS Instance")] style LambdaSG fill:#f0ad4e,color:#000 style RDSSL fill:#5bc0de,color:#000 style RDS fill:#5cb85c,color:#fff
  1. Lambda Security Group (sg-lambda): Controls outbound traffic from Lambda. By default, AWS allows all outbound — ensure port 5432 (PostgreSQL) or 3306 (MySQL) outbound to the RDS security group is permitted.
  2. RDS Security Group (sg-rds): Must have an explicit inbound rule allowing traffic on the DB port, with the source set to the Lambda security group ID (not a CIDR block). This is the most secure and maintainable approach — it uses security group referencing instead of IP ranges.

Step-by-Step Implementation

Step 1: Create Security Groups

🔽 AWS CLI — Create Lambda and RDS Security Groups 
# Create Lambda Security Group
aws ec2 create-security-group \
  --group-name sg-lambda-rds-access \
  --description "Security group for Lambda accessing RDS" \
  --vpc-id vpc-0abc12345def67890

# Output: { "GroupId": "sg-0lambda1234567890" }

# Create RDS Security Group
aws ec2 create-security-group \
  --group-name sg-rds-private \
  --description "Security group for private RDS instance" \
  --vpc-id vpc-0abc12345def67890

# Output: { "GroupId": "sg-0rds1234567890" }

# Add inbound rule to RDS SG: allow PostgreSQL (5432) from Lambda SG
aws ec2 authorize-security-group-ingress \
  --group-id sg-0rds1234567890 \
  --protocol tcp \
  --port 5432 \
  --source-group sg-0lambda1234567890

Step 2: Attach VPC Configuration to Lambda

🔽 AWS CLI — Update Lambda VPC Config 
aws lambda update-function-configuration \
  --function-name my-rds-function \
  --vpc-config SubnetIds=subnet-0private1a,subnet-0private1b,SecurityGroupIds=sg-0lambda1234567890

Step 3: Attach the Required IAM Managed Policy

Lambda needs permission to create and manage ENIs in your VPC. Attach the AWS managed policy AWSLambdaVPCAccessExecutionRole to your Lambda execution role.

🔽 AWS CLI — Attach IAM Policy to Lambda Execution Role 
aws iam attach-role-policy \
  --role-name my-lambda-execution-role \
  --policy-arn arn:aws:iam::aws:policy/service-role/AWSLambdaVPCAccessExecutionRole

This managed policy grants exactly these permissions (least privilege for VPC access):

🔽 IAM Policy — AWSLambdaVPCAccessExecutionRole (effective permissions) 
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "ec2:CreateNetworkInterface",
        "ec2:DescribeNetworkInterfaces",
        "ec2:DeleteNetworkInterface",
        "ec2:AssignPrivateIpAddresses",
        "ec2:UnassignPrivateIpAddresses"
      ],
      "Resource": "*"
    }
  ]
}

Step 4: Lambda Function — Database Connection (Python Example)

🔽 Python — Lambda Handler Connecting to RDS (psycopg2) 
import os
import psycopg2

# Best practice: store credentials in AWS Secrets Manager or SSM Parameter Store
# Retrieve at cold start, cache for warm invocations
DB_HOST = os.environ["DB_HOST"]   # RDS endpoint DNS name
DB_NAME = os.environ["DB_NAME"]
DB_USER = os.environ["DB_USER"]
DB_PASSWORD = os.environ["DB_PASSWORD"]
DB_PORT = int(os.environ.get("DB_PORT", 5432))

# Connection cached outside handler for reuse across warm invocations
connection = None

def get_connection():
    global connection
    if connection is None or connection.closed:
        connection = psycopg2.connect(
            host=DB_HOST,
            database=DB_NAME,
            user=DB_USER,
            password=DB_PASSWORD,
            port=DB_PORT,
            connect_timeout=5
        )
    return connection

def lambda_handler(event, context):
    conn = get_connection()
    with conn.cursor() as cur:
        cur.execute("SELECT version();")
        result = cur.fetchone()
    return {"db_version": result[0]}

Internet Access After VPC Attachment

A critical side effect: once Lambda is VPC-attached, it loses all outbound internet access by default. If your function needs to call external APIs or AWS services via public endpoints, you have two options:

graph LR Lambda["VPC-Attached Lambda"] --> PrivateSubnet["Private Subnet"] PrivateSubnet -->|"Option A"| NAT["NAT Gateway
(Public Subnet)"] NAT --> IGW["Internet Gateway"] IGW --> Internet["Public Internet / External APIs"] PrivateSubnet -->|"Option B"| VPCE["VPC Interface Endpoint
(AWS PrivateLink)"] VPCE --> AWSService["AWS Service
(Secrets Manager, S3, SSM)"] style NAT fill:#f0ad4e,color:#000 style VPCE fill:#5bc0de,color:#000 style Internet fill:#d9534f,color:#fff style AWSService fill:#5cb85c,color:#fff
  1. NAT Gateway (Option A): Place Lambda in a private subnet with a route to a NAT Gateway in a public subnet. The NAT Gateway provides outbound internet access. This incurs additional cost.
  2. VPC Endpoints (Option B — Preferred for AWS Services): Use Interface VPC Endpoints (AWS PrivateLink) for AWS services like Secrets Manager, S3, or SSM. Traffic stays within the AWS network — no NAT Gateway needed for those services, and it is more cost-effective and secure.

Common Pitfalls Checklist

  • ❌ Wrong subnet type: Placing Lambda in a public subnet does not grant internet access — Lambda ENIs do not get public IPs. Use private subnets with NAT for internet, or public subnets only if you understand the routing implications.
  • ❌ Security group source is a CIDR, not SG reference: Using a CIDR block is fragile. Always reference the Lambda security group ID directly in the RDS inbound rule.
  • ❌ Single AZ subnet: Specifying only one subnet creates an availability risk. Always specify subnets across at least two Availability Zones.
  • ❌ Missing IAM policy: Without AWSLambdaVPCAccessExecutionRole, Lambda cannot create ENIs and the deployment will fail with an access denied error.
  • ❌ RDS not in the same VPC: Lambda VPC config and RDS must be in the same VPC, or connected via VPC Peering/Transit Gateway with appropriate route tables.

Glossary

Term Definition
ENI (Elastic Network Interface) A virtual network card attached to a resource inside a VPC, giving it a private IP address and network identity.
Private Subnet A VPC subnet with no direct route to an Internet Gateway — resources inside are not reachable from the public internet.
Security Group A stateful virtual firewall controlling inbound and outbound traffic at the resource level within a VPC.
VPC Endpoint (Interface) An AWS PrivateLink-powered endpoint that allows private connectivity to AWS services without traversing the public internet.
NAT Gateway A managed AWS service in a public subnet that enables outbound internet traffic from resources in private subnets.

Next Steps

For production workloads, consider using RDS Proxy in front of your RDS instance — it pools and manages database connections, which is critical for Lambda's highly concurrent and ephemeral execution model. Also evaluate AWS Secrets Manager for rotating database credentials automatically without redeploying your function.

Comments

Post a Comment

Popular posts from this blog

IAM User vs. IAM Role: Why Your EC2 Instance Should Never Use a User

Choosing between an IAM User and an IAM Role is one of the most consequential security decisions in AWS — get it wrong and you either expose long-lived credentials or block your application entirely. This post cuts through the confusion and gives you a definitive answer for the EC2-to-S3 access pattern. TL;DR — Quick Comparison Dimension IAM User IAM Role Identity type A person or a static service account An assumable identity for AWS services, users, or external principals Credential type Long-lived Access Key ID + Secret Access Key Short-lived, auto-rotating STS tokens (max 12 h by default) Credential rotation Manual — you must rotate keys yourself Automatic — AWS rotates them transparently Stored on EC2? Yes — in ~/.aws/credentials or env vars (risky) No — delivered via the In...
Read more

EC2 No Internet Access in Custom VPC: Attaching an Internet Gateway and Fixing Route Tables

Launching an EC2 instance in a custom VPC only to find it cannot reach the internet is one of the most common networking pitfalls in AWS — and it almost always comes down to two missing pieces: an Internet Gateway (IGW) and a correctly configured Route Table. This post walks you through the exact architecture, the root cause, and the precise steps to fix it. TL;DR Step Action Why It Matters 1 Create and attach an Internet Gateway to your VPC Without an IGW, there is no path between your VPC and the public internet 2 Add a 0.0.0.0/0 route pointing to the IGW in your subnet's Route Table The route table is the traffic director — without this entry, outbound packets have nowhere to go 3 Assign a Public IP or Elastic IP to the EC2 instance The IGW performs NAT only for instances with a public IP address 4 Verify Security Group allows outbound traffic Security Groups are stateful firewalls; outbound rules...
Read more

Lambda Infinite Loop with S3: How to Prevent Recursive Triggers

Writing a processed file back to the same S3 bucket that triggered your Lambda function creates a recursive loop — each output file fires a new invocation, which writes another file, ad infinitum. Left unchecked, this silently drains your AWS budget and exhausts Lambda concurrency limits within minutes. TL;DR Strategy Mechanism Best For Complexity Separate Output Prefix/Bucket S3 event filter on prefix Simple pipelines Low Object Metadata Check Lambda reads S3 object metadata before processing Single-bucket requirement Medium Separate Output Bucket Trigger only on source bucket Clean architectural separation Low DynamoDB Idempotency Guard Track processed ETag/key in DynamoDB Exactly-once guarantees Medium-High Understanding the Recursive Loop Before fixing the problem, you need to understand exactly why it happens. The S3 → Lambda trigger is an event-driven subscription: S3 publishes an ObjectCreat...
Read more