FreeBSD network architecture is limited to 700kpps (700 000 paquet per second): This is the big difference with the linux kernel used in Vyatta that don't have this limitation. There are lot's of guide about tuning FreeBSD TCP performance (where the FreeBSD host is an end-point of the TCP session), but it's not the same that tunig forwarding performance (where the FreeBSD host don't have to read the TCP information of the packet that is forwarding).
Basics:
Here is a list of sources about optimizing forwarding performance under FreeBSD.
How to bench or tune the network stack:
FreeBSD Experimental high-performance network stacks:
Here are some benchs regarding network forwarding performance of FreeBSD:
Hardware advice:
The bench lab should permit to measure the pps. For obtaining accurate result the RFC 2544 (Benchmarking Methodology for Network Interconnect Devices) is a good reference.
Here are some links about some comparaisons between routing daemon:
Tools:
Display the information regarding packet traffic, with refresh each second
netstat -w 1 -I nfe0
input (vge0) output
packets errs bytes packets errs bytes colls
303 0 21364 711 0 1059721 0
233 0 15276 545 0 816149 0
188 0 14346 508 0 752803 0
298 0 19451 692 0 1040904 0
263 0 18366 531 0 779491 0
239 0 19480 535 0 785283 0
236 0 27161 488 0 715859 0
Or more cool using systat:
systat -ifstat
/0 /1 /2 /3 /4 /5 /6 /7 /8 /9 /10
Load Average |
Interface Traffic Peak Total
lo0 in 0.121 KB/s 3.923 KB/s 3.693 GB
out 0.121 KB/s 3.923 KB/s 3.693 GB
vge0 in 31.579 KB/s 35.850 KB/s 1.039 GB
out 1.424 MB/s 1.593 MB/s 3.906 GB
Display system-wide statistics for each network protocol
netstat -s
tcp:
549838287 packets sent
497401134 data packets (223809603 bytes)
8557796 data packets (3621832021 bytes) retransmitted
122752 data packets unnecessarily retransmitted
4 resends initiated by MTU discovery
30644686 ack-only packets (2815507 delayed)
0 URG only packets
6026 window probe packets
12653388 window update packets
575517 control packets
313868917 packets received
223261175 acks (for 221487405 bytes)
23944038 duplicate acks
0 acks for unsent data
63662409 packets (558708967 bytes) received in-sequence
511546 completely duplicate packets (100621861 bytes)
2604 old duplicate packets
10140 packets with some dup. data (1622759 bytes duped)
3539217 out-of-order packets (373067493 bytes)
2659 packets (2513 bytes) of data after window
2513 window probes
4846447 window update packets
26645 packets received after close
1415 discarded for bad checksums
0 discarded for bad header offset fields
0 discarded because packet too short
22580 discarded due to memory problems
123764 connection requests
353961 connection accepts
0 bad connection attempts
1904 listen queue overflows
1973 ignored RSTs in the windows
456412 connections established (including accepts)
477764 connections closed (including 10055 drops)
76872 connections updated cached RTT on close
77503 connections updated cached RTT variance on close
17518 connections updated cached ssthresh on close
3341 embryonic connections dropped
155166651 segments updated rtt (of 142193749 attempts)
1633302 retransmit timeouts
2642 connections dropped by rexmit timeout
10747 persist timeouts
7 connections dropped by persist timeout
0 Connections (fin_wait_2) dropped because of timeout
9 keepalive timeouts
2 keepalive probes sent
7 connections dropped by keepalive
21436112 correct ACK header predictions
55502379 correct data packet header predictions
355472 syncache entries added
4169 retransmitted
1251 dupsyn
0 dropped
353961 completed
0 bucket overflow
0 cache overflow
139 reset
461 stale
1904 aborted
0 badack
2 unreach
0 zone failures
355472 cookies sent
995 cookies received
2302476 SACK recovery episodes
5464810 segment rexmits in SACK recovery episodes
3538359759 byte rexmits in SACK recovery episodes
24257592 SACK options (SACK blocks) received
6659181 SACK options (SACK blocks) sent
0 SACK scoreboard overflow
udp:
34558 datagrams received
0 with incomplete header
0 with bad data length field
0 with bad checksum
0 with no checksum
6663 dropped due to no socket
0 broadcast/multicast datagrams undelivered
0 dropped due to full socket buffers
0 not for hashed pcb
27895 delivered
27925 datagrams output
0 times multicast source filter matched
(etc...)
Show statistics recorded by the memory management routines. The network manages a private pool of memory buffers.
netstat -m 925/575/1500 mbufs in use (current/cache/total) 679/385/1064/25600 mbuf clusters in use (current/cache/total/max) 679/226 mbuf+clusters out of packet secondary zone in use (current/cache) 244/54/298/12800 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/6400 9k jumbo clusters in use (current/cache/total/max) 0/0/0/3200 16k jumbo clusters in use (current/cache/total/max) 2565K/1129K/3695K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0/5/6656 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile 0 calls to protocol drain routines
Report on the number of interrupts taken by each device since system startup:
vmstat -i interrupt total rate irq18: vge0 517365969 1041 irq20: atapci0 34811697 70 cpu0: timer 994124321 2000 Total 1546301987 3112