Session

Reducing Kernel Queuing Delays with TCP Window Space Events

Speakers

David Goulet, Rob Jansen

Label

Nuts and Bolts

Session Type

Talk

Description

The combination of TCP auto-tuning and asynchronous 
I/O event notifications (e.g., epoll) allows the Linux 
kernel to generally sustain high-volume TCP 
connections---even for connections with high 
bandwidth-delay products (high link bandwidth and/or 
high path latency). However, bufferbloat can quickly 
become an issue when multiple such connections are in 
use. In particular, high outbound kernel queuing 
delays have been observed in the Tor anonymity 
network, a large distributed system whose relays often 
manage thousands of sockets---many of which are 
simultaneously-active, high-volume TCP connections.

In this work, we propose a new notification event that 
triggers when TCP is ready to send data on a socket 
while seeking to better understand how it can help 
applications to better manage network I/O and improve 
performance. The new event supplements and extends the 
current write event that triggers when a socket buffer 
has free space, and the difference in semantics allows 
more precise control over queuing to the application. 
We describe the problem, detail a proposal for 
extending epoll to support the new semantics 
(including a code patch), and show the effect that 
such a change could have on performance through a 
small scale simulation.