How to View and Filter VPC Flow Logs in CloudWatch Logs Insights

You enabled VPC Flow Logs, stared at the raw output in S3, and immediately understood why nobody reads those files directly. The fields are space-delimited, the timestamps are epoch, and finding rejected traffic to a specific port means grepping through millions of records. Sending flow logs to CloudWatch Logs and querying them with Logs Insights is the practical path — this post walks through the exact setup and the queries that actually surface what you need.

TL;DR: VPC Flow Logs to CloudWatch Logs Insights

Step	What You're Doing	Why It Matters
1	Create IAM role for flow log delivery	Flow Logs service needs permission to write to CloudWatch Logs
2	Enable VPC Flow Logs → CloudWatch destination	Raw packets become structured log events
3	Verify log group receives data	Confirm delivery before writing queries
4	Run Logs Insights query for REJECT + port	Filter millions of records in seconds
5	Pin query to CloudWatch Dashboard	Operational visibility without re-running manually

How VPC Flow Logs Work Before You Touch CloudWatch

Flow Logs capture network traffic metadata at the ENI level — not packet contents, just the 5-tuple (source IP, destination IP, source port, destination port, protocol) plus action (ACCEPT or REJECT) and a few other fields. The log record format depends on which fields you select when creating the flow log; the default format includes a fixed set of fields in a fixed order.

When you target CloudWatch Logs as the destination, the VPC Flow Logs service assumes an IAM role you provide, creates log streams per ENI under the log group you specify, and writes records as CloudWatch log events. Each log event is a single flow record — one line of space-delimited text. Logs Insights parses that text using the parse command or the built-in flow log field names if you use a custom format with JSON output. For the default format, you parse fields manually.

One thing that catches engineers off guard: there is an aggregation interval. Flow records are not real-time — by default, the capture window is 10 minutes, though you can configure a 1-minute aggregation interval when creating the flow log. If you're chasing a transient connection issue and see no records, check whether the aggregation window has elapsed.

graph LR ENI["ENI (Elastic Network Interface)"] --> FLS["VPC Flow Logs Service (aggregation window: 1 or 10 min)"] FLS --> IAM["IAM Role Assumption vpc-flow-logs.amazonaws.com"] IAM --> LG["CloudWatch Log Group /aws/vpc/flowlogs"] LG --> LS1["Log Stream eni-0abc123.."] LG --> LS2["Log Stream eni-0def456.."] LS1 --> LI["Logs Insights Query Engine"] LS2 --> LI LI --> R["Query Results (REJECT, port, srcAddr)"]

1. ENI — Traffic enters or leaves the elastic network interface attached to your EC2 instance, NAT gateway, or other resource.
2. VPC Flow Logs service — AWS captures metadata for each flow and aggregates records within the capture window (1 or 10 minutes).
3. IAM Role assumption — The Flow Logs service assumes the delivery role you configured to authenticate against CloudWatch Logs.
4. CloudWatch Log Group / Log Streams — Each ENI gets its own log stream. Records land as individual log events.
5. Logs Insights — You run queries against the log group; Insights scans log events and returns structured results.

Step 1: Create the IAM Role for Flow Log Delivery

Flow Logs needs a role it can assume, with permissions to create log groups, create log streams, and put log events. This is a prerequisite — without it, the flow log creation will succeed in the console but delivery will silently fail.

Create the trust policy file first:

cat > flow-logs-trust-policy.json <<'EOF'
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "vpc-flow-logs.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}
EOF

aws iam create-role \
  --role-name VPCFlowLogsDeliveryRole \
  --assume-role-policy-document file://flow-logs-trust-policy.json \
  --region us-east-1

Now attach the permissions policy. The role needs these specific CloudWatch Logs actions:

cat > flow-logs-permissions.json <<'EOF'
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents",
        "logs:DescribeLogGroups",
        "logs:DescribeLogStreams"
      ],
      "Resource": "*"
    }
  ]
}
EOF

aws iam put-role-policy \
  --role-name VPCFlowLogsDeliveryRole \
  --policy-name VPCFlowLogsDeliveryPolicy \
  --policy-document file://flow-logs-permissions.json

Retrieve the role ARN — you'll need it in the next step:

aws iam get-role \
  --role-name VPCFlowLogsDeliveryRole \
  --query 'Role.Arn' \
  --output text

Step 2: Enable VPC Flow Logs with CloudWatch Destination

With the role in place, create the flow log targeting your VPC. The command below uses the default log format and a 1-minute aggregation interval — adjust --vpc-id and the role ARN to match your environment.

aws ec2 create-flow-logs \
  --resource-type VPC \
  --resource-ids vpc-0abc123def456789 \
  --traffic-type ALL \
  --log-destination-type cloud-watch-logs \
  --log-group-name /aws/vpc/flowlogs \
  --deliver-logs-permission-arn arn:aws:iam::123456789012:role/VPCFlowLogsDeliveryRole \
  --max-aggregation-interval 60 \
  --region us-east-1

The response includes a FlowLogIds array and an Unsuccessful array. If Unsuccessful is non-empty, the error message will tell you exactly what failed — usually a missing IAM permission or a malformed role ARN. Don't skip reading that field.

Confirm the flow log is active:

aws ec2 describe-flow-logs \
  --filter Name=resource-id,Values=vpc-0abc123def456789 \
  --query 'FlowLogs[*].{ID:FlowLogId,Status:FlowLogStatus,Destination:LogGroupName}' \
  --output table \
  --region us-east-1

You want FlowLogStatus: ACTIVE. A status of FAILED means delivery is broken — check the IAM role trust relationship and permissions before proceeding.

Step 3: Verify Log Group Receives Data

Wait at least 2-3 minutes after creating the flow log (longer if you kept the default 10-minute aggregation interval), then confirm log streams are appearing. Each ENI in your VPC gets its own stream, named after the ENI ID and account/region identifiers.

aws logs describe-log-streams \
  --log-group-name /aws/vpc/flowlogs \
  --order-by LastEventTime \
  --descending \
  --max-items 5 \
  --region us-east-1

If no streams appear after 15 minutes, the delivery role is almost certainly the problem. The most common failure: the trust policy references vpc-flow-logs.amazonaws.com but the role was created with a typo, or the permissions policy was attached to the wrong role name. Verify both with aws iam get-role and aws iam get-role-policy before assuming a service issue.

Pull a few raw records to confirm the format before writing queries:

aws logs get-log-events \
  --log-group-name /aws/vpc/flowlogs \
  --log-stream-name $(aws logs describe-log-streams \
    --log-group-name /aws/vpc/flowlogs \
    --order-by LastEventTime \
    --descending \
    --max-items 1 \
    --query 'logStreams[0].logStreamName' \
    --output text \
    --region us-east-1) \
  --limit 5 \
  --region us-east-1

The raw event messages look like: 2 123456789012 eni-0abc123 10.0.1.5 10.0.2.10 54321 443 6 10 840 1620000000 1620000060 ACCEPT OK. That's the default format field order — version, account-id, interface-id, srcaddr, dstaddr, srcport, dstport, protocol, packets, bytes, start, end, action, log-status.

Step 4: Query Rejected Traffic to a Specific Port with Logs Insights

This is where the investment pays off. Logs Insights can scan gigabytes of flow log data in seconds. The default flow log format is not JSON, so you need to parse the fields explicitly using the parse command.

Finding All REJECT Traffic to Port 22

🔽 Click to expand — Logs Insights query for rejected SSH traffic

fields @timestamp, @message
| parse @message '* * * * * * * * * * * * * *' as
    version, accountId, interfaceId, srcAddr, dstAddr,
    srcPort, dstPort, protocol, packets, bytes,
    startTime, endTime, action, logStatus
| filter action = 'REJECT' and dstPort = '22'
| stats count(*) as rejectedCount by srcAddr
| sort rejectedCount desc
| limit 20

Run this query in the CloudWatch Logs Insights console against the /aws/vpc/flowlogs log group, or via CLI:

aws logs start-query \
  --log-group-name /aws/vpc/flowlogs \
  --start-time $(date -d '1 hour ago' +%s) \
  --end-time $(date +%s) \
  --query-string "fields @timestamp, @message | parse @message '* * * * * * * * * * * * * *' as version, accountId, interfaceId, srcAddr, dstAddr, srcPort, dstPort, protocol, packets, bytes, startTime, endTime, action, logStatus | filter action = 'REJECT' and dstPort = '22' | stats count(*) as rejectedCount by srcAddr | sort rejectedCount desc | limit 20" \
  --region us-east-1

The command returns a queryId. Poll for results:

aws logs get-query-results \
  --query-id <queryId-from-above> \
  --region us-east-1

Broader Rejection Analysis — All Ports, Grouped

🔽 Click to expand — Logs Insights query for all rejected traffic by destination port

fields @timestamp, @message
| parse @message '* * * * * * * * * * * * * *' as
    version, accountId, interfaceId, srcAddr, dstAddr,
    srcPort, dstPort, protocol, packets, bytes,
    startTime, endTime, action, logStatus
| filter action = 'REJECT'
| stats count(*) as rejectedCount by dstPort, srcAddr
| sort rejectedCount desc
| limit 50

This surfaces which ports are being probed most aggressively and from which source IPs — useful for identifying scanning activity or misconfigured clients that should have access but don't.

graph TD TR["Select Time Range"] --> PQ["parse @message '* * * * * * ...' as fields"] PQ --> FI["filter action = 'REJECT' and dstPort = '22'"] FI --> ST["stats count(*) as rejectedCount by srcAddr"] ST --> SO["sort rejectedCount desc"] SO --> LIM["limit 20"] LIM --> OUT["Results: Top source IPs rejected on port 22"]

1. Time range selection — Logs Insights scans only the specified window. Narrower ranges cost less and return faster.
2. parse command — Extracts named fields from the raw space-delimited message string. Field order must match the flow log format exactly.
3. filter — Applies after parsing. Filters on action and dstPort reduce the working set before aggregation.
4. stats + sort — Aggregates by source IP and orders by rejection count, surfacing the noisiest offenders first.

The Misdiagnosis That Wastes an Hour

Here's a pattern that shows up repeatedly: you run the REJECT query, get zero results, and assume the security group is allowing the traffic. It isn't. The actual cause is that the flow log was created against the VPC but the specific ENI you're investigating belongs to an interface that was created before the flow log was enabled — and you're looking at the wrong log stream.

Flow Logs on a VPC resource capture traffic on all ENIs within that VPC, including ENIs created after the flow log is enabled. But if you created the flow log and then immediately tried to query, the aggregation window hasn't closed yet. No records means no closed windows, not no traffic.

The confirmation: check the lastIngestionTime on the log stream for that ENI.

aws logs describe-log-streams \
  --log-group-name /aws/vpc/flowlogs \
  --log-stream-name-prefix eni-0abc123 \
  --query 'logStreams[*].{Stream:logStreamName,LastIngestion:lastIngestionTime}' \
  --output table \
  --region us-east-1

If lastIngestionTime is recent, data is flowing. If the stream doesn't exist at all, the ENI ID prefix is wrong — use describe-log-streams without a prefix filter to see what's actually there. The fix is almost always: wait for the aggregation window, or check that you're querying the right log group.

Step 5: Pin Queries to a CloudWatch Dashboard

Re-running Insights queries manually every time someone asks 'is port 3389 being scanned?' gets old fast. Save the query and add it to a dashboard widget for persistent visibility.

Save the query first:

aws logs put-query-definition \
  --name 'VPC-Rejected-Traffic-By-Port' \
  --log-group-names /aws/vpc/flowlogs \
  --query-string "fields @timestamp, @message | parse @message '* * * * * * * * * * * * * *' as version, accountId, interfaceId, srcAddr, dstAddr, srcPort, dstPort, protocol, packets, bytes, startTime, endTime, action, logStatus | filter action = 'REJECT' | stats count(*) as rejectedCount by dstPort | sort rejectedCount desc | limit 20" \
  --region us-east-1

In the CloudWatch console, navigate to Logs Insights, load the saved query, run it, then choose 'Add to dashboard'. Select an existing dashboard or create one named something like VPC-Security-Overview. The widget auto-refreshes on the dashboard's refresh interval.

IAM Policy for Engineers Running Logs Insights Queries

Engineers who need to query flow logs but shouldn't have broader CloudWatch access need a scoped policy. Read and query actions on Logs Insights require these permissions:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "logs:StartQuery",
        "logs:GetQueryResults",
        "logs:StopQuery",
        "logs:DescribeLogGroups",
        "logs:DescribeLogStreams",
        "logs:GetLogEvents",
        "logs:FilterLogEvents"
      ],
      "Resource": "arn:aws:logs:us-east-1:123456789012:log-group:/aws/vpc/flowlogs:*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "logs:DescribeLogGroups"
      ],
      "Resource": "*"
    }
  ]
}

Note that logs:DescribeLogGroups requires Resource: * for the Logs Insights console to enumerate available log groups. The query execution actions can be scoped to the specific log group ARN.

Wrap-Up: VPC Flow Logs in CloudWatch Logs Insights

The setup is straightforward once the IAM delivery role is correct — that's where most failed deployments stall. After data lands in CloudWatch Logs, Logs Insights queries give you the ability to filter VPC Flow Logs by action, port, protocol, or source IP across any time window in seconds. For ongoing security visibility, saved queries pinned to a dashboard eliminate the manual re-run cycle.

Next steps worth considering:

• Configure a CloudWatch Metric Filter on the flow log group to create a metric for REJECT events, then attach an alarm for threshold breaches — documented in the CloudWatch Logs Metric Filters documentation.
• If log volume grows large, evaluate whether S3 with Athena is more cost-effective for long-retention queries — CloudWatch Logs ingestion and storage pricing scales with volume.
• Review the VPC Flow Logs official documentation for custom log formats, which allow JSON output and eliminate the need for manual field parsing in Insights.

Glossary

Term	Definition
VPC Flow Logs	An AWS feature that captures IP traffic metadata for network interfaces in a VPC. Does not capture packet payloads.
ENI (Elastic Network Interface)	A virtual network interface attached to an EC2 instance or other AWS resource. Flow logs are captured at the ENI level.
Logs Insights	CloudWatch's interactive query engine for log data. Supports field parsing, filtering, aggregation, and visualization.
Aggregation Interval	The window over which flow records are collected before being published. Configurable as 1 minute or 10 minutes when creating the flow log.
REJECT action	A flow log field value indicating that traffic was blocked by a security group or network ACL rule.

Related Posts

• 📄 EC2 High Network I/O but Low CPU: Diagnosing Traffic with CloudWatch and VPC Flow Logs
• 📄 How to Use CloudWatch Logs Insights to Search Lambda Error Logs