This document was translated by ChatGPT

Without inserting any code into the application, DeepFlow automatically generates network flow logs for all services. Database table name: flow_log.l4_flow_log.

1. Tags

List of automatically injected tags: IP, protocol, port, network header fields, collection location, cloud resources, K8s resources, K8s custom labels. Detailed field descriptions are as follows.

Name	DisplayName	Description
_id	UID
time	Time	Round end_time to seconds.
region	Region
az	Availability Zone
host	VM Hypervisor	Host running virtual machine.
chost	Cloud Host	Including virtual machines
vpc	VPC
l2_vpc	Forwarding VPC	VPC where the MAC address is located.
subnet	Subnet
router	Router
dhcpgw	DHCP Gateway
lb	Load Balancer
lb_listener	Load Balancer Listener
natgw	NAT Gateway
redis	Redis
rds	RDS
pod_cluster	K8s Cluster
pod_ns	K8s Namespace
pod_node	K8s Node
pod_ingress	K8s Ingress
pod_service	K8s Service
pod_group_type	K8s Workload Type
pod_group	K8s Workload	Such as Deployment
pod	K8s POD
service	Service	Deprecated，please use pod_service
resource_gl0_type	Auto Instance Type	Deprecated，please use auto_instance_type.
resource_gl0	Auto Instance Tag	Deprecated，please use auto_instance.
resource_gl1_type	Type - K8s Workload First	Deprecated，please use auto_service_type.
resource_gl1	Instance - K8s Workload First	Deprecated，please use auto_service.
resource_gl2_type	Auto Service Type	Deprecated，please use auto_service_type.
resource_gl2	Auto Service Tag	Deprecated，please use auto_service.
auto_instance_type	Auto Instance Type	The type of 'auto_instance'.
auto_instance	Auto Instance Tag	The instance of IP
auto_service_type	Auto Service Type	The type of 'auto_service'.
auto_service	Auto Service Tag	On the basis of 'auto_instance'
gprocess	Process
tap_port_host	Tap Port Host	Deprecated，please use capture_nic_host.
tap_port_chost	Tap Port Cloud Host	Deprecated，please use capture_nic_chost.
tap_port_pod_node	Tap Port K8s Node	Deprecated，please use capture_nic_pod_node.
capture_nic_host	Host of Capture NIC
capture_nic_chost	Cloud Host of Capture NIC
capture_nic_pod_node	K8s Node of Capture NIC
host_ip	VM Hypervisor	The management IP address of VM Hypervisor.
host_hostname	VM Hypervisor	The hostname of VM Hypervisor.
chost_ip	Cloud Host	The primary IP address of Cloud Host.
chost_hostname	Cloud Host	The hostname of Cloud Host.
pod_node_ip	K8s Node	The primary IP address of K8s Node.
pod_node_hostname	K8s Node	The hostname of K8s Node.
k8s.label	K8s Label
k8s.annotation	K8s Annotation
k8s.env	K8s Env
cloud.tag	Cloud Tag
os.app	OS APP
eth_type	Ether Type
vlan	VLAN TAG
mac	MAC Address
ip	IP Address
is_ipv4	IPv4 Flag
is_internet	Internet IP Flag	Whether the IP address is an external Internet address.
province	Province	The province to which the Internet IP address belongs.
protocol	Network Protocol
tunnel_tier	Tunnel Tiers
tunnel_type	Tunnel Type
tunnel_tx_id	TX Tunnel ID
tunnel_rx_id	RX Tunnel ID
tunnel_tx_ip	TX Tunnel IP Address
tunnel_tx_ip_0	TX Tunnel src IP Address
tunnel_tx_ip_1	TX Tunnel dst IP Address
tunnel_rx_ip	RX Tunnel IP Address
tunnel_rx_ip_0	RX Tunnel src IP Address
tunnel_rx_ip_1	RX Tunnel dst IP Address
tunnel_tx_mac	TX Tunnel MAC Address
tunnel_tx_mac_0	TX Tunnel src MAC Address
tunnel_tx_mac_1	TX Tunnel dst MAC Address
tunnel_rx_mac	RX Tunnel MAC Address
tunnel_rx_mac_0	RX Tunnel src MAC Address
tunnel_rx_mac_1	RX Tunnel dst MAC Address
client_port	Client Port
server_port	Server Port
tcp_flags_bit	TCP Flag Set	The set of TCP flags in all packets in the current natural minute.
syn_seq	Seq no. of SYN Packet
syn_ack_seq	Seq no. of SYN-ACK Packet
last_keepalive_seq	Seq no. of Heartbeat Packet	Seq number in the most recent heartbeat packet.
last_keepalive_ack	Ack no. of Heartbeat Packet	Ack number in the most recent heartbeat packet.
l7_protocol	Application Protocol
request_domain	Request Domain
flow_id	Flow ID
start_time	Start Time	Unit: microseconds. Indicates the start time of the flow within the current natural minute
end_time	End Time	Unit: microseconds. Indicates the end time of the flow within the current natural minute. If the flow is closed within this minute
close_type	Flow Close Type
status	Status	Determined by the close_type and protocol: Normal/ForceReport/Non-TCP timeout/Disconnected* = Normal
is_new_flow	New Flow Flag
signal_source	Signal Source
tap	Traffic Access Point	Deprecated，please use capture_network_type.
capture_network_type	Network Location	The network location for capturing traffic uses a fixed value (Cloud Network) to represent intra-cloud traffic
vtap	DeepFlow Agent	Deprecated，please use agent.
agent	DeepFlow Agent
nat_source	NAT Source
tap_port	TAP Port Identifier	Deprecated
tap_port_name	TAP Port Name	Deprecated
tap_port_type	TAP Port Type	Deprecated
capture_nic	Capture NIC ID	When the value of tap_port_type is 'Local NIC'
capture_nic_name	Capture NIC Name	When the value of tap_port_type is 'Local NIC'
capture_nic_type	Capture NIC Type	Indicates the type of traffic collection location
tap_side	TAP Side	Deprecated
observation_point	Observation Point	The logical location of the collection location in the traffic path
l2_end	Boundary of L2 Network	Indicates whether the traffic is collected on the client NIC or the server NIC.
l3_end	Boundary of L3 Network	Indicates whether the traffic is collected in the Layer 2 network where the client or server is located.
has_pcap	PCAP File	Whether the PCAP file is stored
nat_real_ip	NAT IP Address	The real IP address before (after) NAT
nat_real_port	NAT Port	The real port number before NAT works

generate from csv file: l4_flow_log.en

2. Metrics

List of metrics: throughput, load, latency, TCP anomalies, retransmissions, zero window. Detailed field descriptions are as follows.

Field	DisplayName	Unit	Description
byte	Byte	Byte
byte_tx	Byte TX	Byte
byte_rx	Byte RX	Byte
total_byte_tx	Total Byte TX	Byte
total_byte_rx	Total Byte RX	Byte
packet	Packet	Packet
packet_tx	Packet TX	Packet
packet_rx	Packet RX	Packet
total_packet_tx	Total Packet TX	Packet
total_packet_rx	Total Packet RX	Packet
l3_byte	L3 Payload	Byte
l3_byte_tx	L3 Payload TX	Byte
l3_byte_rx	L3 Payload RX	Byte
bpp	Bytes per Packet	Byte
bpp_tx	Bytes per Packet TX	Byte
bpp_rx	Bytes per Packet RX	Byte
new_flow	New Flow	Flow
closed_flow	Closed Flow	Flow
syn_count	SYN Packet	Packet
synack_count	SYN-ACK Packet	Packet
l4_byte	L4 Payload	Byte
l4_byte_tx	L4 Payload TX	Byte
l4_byte_rx	L4 Payload RX	Byte
direction_score	Direction Score		The higher the score
log_count	Log Count
retrans_syn	SYN Retransmission	Packet
retrans_synack	SYN-ACK Retransmission	Packet
retrans	TCP Retransmission	Packet
retrans_tx	TCP Client Retransmission	Packet
retrans_rx	TCP Server Retransmission	Packet
zero_win	TCP ZeroWindow	Packet
zero_win_tx	TCP Client ZeroWindow	Packet
zero_win_rx	TCP Server ZeroWindow	Packet
retrans_syn_ratio	SYN Retrans. %	%
retrans_synack_ratio	SYN-ACK Retrans. %	%
retrans_ratio	TCP Retrans. %	%
retrans_tx_ratio	TCP Client Retrans. %	%
retrans_rx_ratio	TCP Server Retrans. %	%
zero_win_ratio	TCP ZeroWindow %	%
zero_win_tx_ratio	TCP Client ZeroWindow %	%
zero_win_rx_ratio	TCP Server ZeroWindow %	%
tcp_establish_fail	Error	Flow
client_establish_fail	Client Error	Flow
server_establish_fail	Server Error	Flow
tcp_establish_fail_ratio	Error %	%
client_establish_fail_ratio	Client Error %	%
server_establish_fail_ratio	Client Error %	%
tcp_transfer_fail	Transfer Error	Flow	All transfer errors.
tcp_transfer_fail_ratio	Transfer Error %	%
tcp_rst_fail	RST	Flow	All RST errors.
tcp_rst_fail_ratio	RST %	%
client_source_port_reuse	Est. - Client Port Reuse	Flow
server_syn_miss	Est. - Server SYN Miss	Flow
client_establish_other_rst	Est. - Client Other RST	Flow
client_ack_miss	Est. - Client ACK Miss	Flow
server_reset	Est. - Server Direct RST	Flow
server_establish_other_rst	Est. - Server Other RST	Flow
client_rst_flow	Transfer - Client RST	Flow
server_rst_flow	Transfer - Server RST	Flow
server_queue_lack	Transfer - Server Queue Overflow	Flow
tcp_timeout	Transfer - TCP Timeout	Flow
client_half_close_flow	Close - Client Half Close	Flow
server_half_close_flow	Close - Server Half Close	Flow
rtt	Avg TCP Est. Delay	us
tls_rtt	Avg TLS Est. Delay	us
rtt_client	Avg TCP Est. Client Delay	us
rtt_server	Avg TCP Est. Server Delay	us
srt	Avg TCP/ICMP ACK Delay	us
art	Avg Data Delay	us
cit	Avg Client Idle Delay	us
rtt_max	Max TCP Est. Delay	us
tls_rtt_max	Max TLS Est. Delay	us
rtt_client_max	Max TCP Est. Client Delay	us
rtt_server_max	Max TCP Est. Server Delay	us
srt_max	Max TCP/ICMP ACK Delay	us
art_max	Max Data Delay	us
cit_max	Max Client Idle Delay	us
srt_sum	Total TCP/ICMP ACK Delay	us
srt_count	TCP TCP/ICMP Delay Count
art_sum	Total Data Delay	us
art_count	Data Delay Count
cit_sum	Total Client Idle Delay	us
cit_count	Client Idele Delay Count
duration	Duration	us	The duration from start_time to the last packet (not end_time).
l7_request	Request
l7_response	Response
rrt	Avg App. Delay	us
rrt_sum	Total App. Delay	us
rrt_count	App. Delay Count
rrt_max	Max App. Delay	us
l7_error	App. Error
l7_client_error	App. Client Error
l7_server_error	App. Server Error
l7_server_timeout	App. Server Timeout
l7_error_ratio	App. Error %	%
l7_client_error_ratio	App. Client Error %	%
l7_server_error_ratio	App. Server Error %	%
l7_parse_failed	L7 Protocol Parse Failed	Packet	Cumulative number of application protocol parsing failures
row	Row Count

generate from csv file: l4_flow_log.en

3. Grafana Dashboard

Based on the above data, you can build rich dashboards using Grafana. We have pre-configured a Network - Flow Log dashboard in Grafana, as shown below:

Network Flow Log

You can also visit DeepFlow Online Demo (opens new window) to see the effect.