Overview
AWS VPC Flow Logs is a feature that enables you to capture information about the IP traffic going to and from network interfaces in your VPC. Flow log data is collected outside of the path of your network traffic, therefore it does not affect network throughput or latency. You can create or delete flow logs without any risk of impact to network performance.
A flow log record represents a network flow in your VPC. By default, each record captures a network internet protocol (IP) traffic flow that occurs within an “aggregation interval”, also referred to as a “capture window”.
Each record is a string with fields separated by spaces. A record includes values for the different components of the IP flow e.g. source, destination, and protocol.
Collection
AWS VPC Flow Logs can be configured to publish flow log data to Amazon CloudWatch Logs log group.
While AWS CloudWatch Logs log group can be configured to have a subscription filter and have it deliver the published flow logs data received to Amazon Kinesis Data Stream.
Then Snare Central can be configured to periodically call AWS API to pull and get the log data stored in AWS Kinesis Data Stream.
Sample Flow Log
Table Fields
Field | Description |
|---|---|
DATE | end (index 11) - The date extracted from converting the end integer representing seconds in UTC since the epoch (1970-01-01 00:00:00 UTC) to "2006-01-02T15:04:05Z" in YYYY-MM-DD format. end is the time in Unix seconds, when the last packet of the flow was received within the aggregation interval. This might be up to 60 seconds after the packet was transmitted or received on the network interface. |
TIME | end (index 11) - The time extracted from converting the end integer representing seconds in UTC since the epoch (1970-01-01 00:00:00 UTC) to "2006-01-02T15:04:05Z" in HH:MM:SS format. end is the time in Unix seconds, when the last packet of the flow was received within the aggregation interval. This might be up to 60 seconds after the packet was transmitted or received on the network interface. |
SYSTEM | AWS Internal |
TABLE | AWSVPCFlowLog |
COLLECTIONDATETIME | Snare Central’s local date and time of the actual log collection from AWS Kinesis Data Stream in RFC3339Nano format. |
CWLDATETIME | AWS CloudWatch Log’s timestamp when it receives the event log from other AWS services in RFC3339Nano format. |
DATETIME | end (index 11) - Date and Time extracted from converting the end integer representing seconds in UTC since the epoch (1970-01-01 00:00:00 UTC) to RFC3339Nano format. end is the time in Unix seconds, when the last packet of the flow was received within the aggregation interval. This might be up to 60 seconds after the packet was transmitted or received on the network interface. |
VERSION | version (index 0) - The VPC Flow Logs version. If you use the default format, the version will be 2. If you use a custom format, the version is the highest version among the specified fields. |
ACCOUNTID | account id (index 1) - The AWS account ID of the owner of the source network interface for which traffic is recorded. If the network interface is created by an AWS service, e.g. when creating a VPC endpoint, the record might display unknown for this field. |
INTERFACEID | interface id (index 2) - The ID of the network interface for which the traffic is recorded. |
SRCADDR | srcaddr (index 3) - The source address for incoming traffic, or the IPv4 or IPv6 address of the network interface for outgoing traffic on the network interface. The IPv4 address of the network interface is always its private IPv4 address. |
DSTADDR | dstaddr (index 4) - The destination address for outgoing traffic, or the IPv4 or IPv6 address of the network interface for incoming traffic on the network interface. The IPv4 address of the network interface is always its private IPv4 address. |
SRCPORT | srcport (index 5) - The source port of the traffic. |
DSTPORT | dstport (index 6) - The destination port of the traffic. |
PROTOCOL | protocol (index 7) - The IANA protocol number of the traffic. |
PACKETS | packets (index 8) - The number of packets transferred during the flow. |
BYTES | bytes (index 9) - The number of bytes transferred during the flow. |
START | start (index 10) - Date and Time extracted from converting the start integer representing seconds in UTC since the epoch (1970-01-01 00:00:00 UTC) to RFC3339Nano format. |
ACTION | action (index 12) - The action that is associated with the traffic: ACCEPT - The traffic was accepted. REJECT - The traffic was rejected. e.g. the traffic was not allowed by the security groups or network ACLs, or packets arrived after the connection was closed. |
LOGSTATUS | log status (index 13) - The logging status of the flow log: OK - Data is logging normally to the chosen destinations. NODATA - There was no network traffic to or from the network interface during the aggregation interval. SKIPDATA - Some flow log records were skipped during the aggregation interval. This might be because of an internal capacity constraint, or an internal error. |
SNAREDATAMAP | All unclassified field/s in the log will be pushed into the SNAREDATAMAP. |
Notes
https://docs.aws.amazon.com/vpc/latest/userguide/flow-logs.html