Ensuring Strong TCP Performance Over Docsis Modems

Delivering efficient and robust TCP support in a Docsis network in clearly not an easy task. The current Docsis specs provides some support for TCP, but to best handle TRCP traffic designers need to supplement these enhancements by embedding application awareness in the modem designs

In this article, we'll present an overview of the inherent bi-directional nature of TCP and discusses the impact that the Docsis protocols have on TCP. Finally, we will suggest methods to increase the performance of TCP and applications utilizing TCP by embedding application awareness in the cable modem.

TCP Characteristics
TCP is the most common transport protocol for Internet applications and because it is connection based, it can ensure that each data packet transmitted from a server reaches the intended client application. To guarantee that each packet reaches its destination, TCP uses a handshake protocol. Both the server and client keep track of the packets being transmitted/received.

The server sends several packets at one time to the client and then waits for an acknowledgement that the packets have been received. If an acknowledgement (ACK) does not arrive back to the server in a given time, the server "stalls" and will not send the next block of packets.

Eventually, if an ACK is not received, the server will retransmit the unacknowledged packets. The number of packets that the server sends before waiting for ACKs to arrive is determined by the "window size". The window size has a significant affect on TCP performance—pthe smaller the window size, the higher the chance that the server will have to stall the transmission waiting for ACKs to arrive.

Figure 1 shows an example of a TCP session from client to server that has a "bursty" nature due to its small window size. Even though the physical channel enables a high data rate, the actual throughput seen by the application at the client side is limited by the TCP protocol to a fraction of that rate. It is throughput, not data rate, which has the most dramatic impact on TCP application performance. When configured to a larger window size, the number of packets increases and the traffic is "smoother".

Docsis Basics
CableLabs' Docsis specification defines both the physical layer (PHY) aspects of the transmission of a cable modem and media access control (MAC) protocols for accessing the cable channel. Docsis defines different transmission characteristics for the downstream transmission (transmission from the cable modem termination system [CMTS] to the cable modem at home) and the upstream transmission (from the home back to the CMTS). The differences are in both the PHY and the MAC layers, and cause the DOCSIS channel to perform asymmetrically.

Under all of the Docsis specifications, the downstream channel (see Figure 2) is defined to support up to 40 Mbit/s in a continuous transmission. The CMTS is responsible for receiving packets from the core network and sending them to the cable modems over the cable network. The CMTS determines the order and priority of packet transmission. And, because the CMTS has full ownership of the downstream media, negotiation is not required to access it.

The upstream channel (also shown in Figure 2), on the other hand, is quite different. In the upstream channel, all modems sharing the media compete for access to the upstream. Cable modems that want to send data need to first request the CMTS for a transmission opportunity. The CMTS then collects requests from the modems and sends a message to indicating the time at which each modem can send data on the upstream channel. A modem may request only one transmit opportunity at a time, thus limiting the number of upstream transmissions the modem can perform per second.

TCP Performance over Docsis 1.0 Modems
Docsis 1.0 enables data communication between cable modems and the CMTS. Cable modems compete equally to utilize the upstream channel. A cable modem with data for transmission in the upstream must first request permission from the CMTS. The CMTS then processes this message along with similar requests coming from all other cable modems on the network. The CMTS then decides how to allocate the upstream channel to the requesting cable modems and sends a message that "maps" the usage of the upstream for the coming time segment.

The number of upstream transmissions that the cable modem can handle is limited to a few hundred per second. In Docsis 1.0, the cable modem can send a single packet in each upstream transmission burst. In terms of TCP, this means that the number of ACKs a client application can send to the server is limited. Figure 3 illustrates the cycle that goes from packet reception in the downstream until the acknowledgement is then sent to the server.

The bottleneck affect on the bandwidth for applications running over TCP is illustrated in the following example:

Example 1: TCP Performance in a Docsis 1.1 Plant

Example 1 shows that the maximum TCP throughput achieved by the a Docsis 1.0 cable modem is limited to 7.2 Mbit/s even though the downstream channel is capable of much greater bandwidth.

Docsis 1.1 Provides Improvements
Docsis 1.1 provides several different enhancements over Docsis 1.0 that improve TCP performance. Though these enhancements in the MAC protocol were triggered by the desire to enable voice applications, the tools that were added to the protocol provide means to significantly improve data transmission over TCP. These enhancements included multiple service flows, payload header suppression, concatenation and others.

Concatenation is one of the biggest wins for developers looking to support TCP. Concatenation was an optional feature in Docsis 1.0 but is a mandatory part of Docsis 1.1. Concatenation allows a cable modem to combine together several packets and transmit them together as one. This means that the number of packets per second the modem can send is no longer limited by the number transmission opportunities provided by the CMTS. This is especially important for the performance of TCP. Since ACKs are short packets, several ACKs can be combined and sent in a single transmission opportunity (or burst).

Figure 4 illustrates how several ACKs are combined in a single burst, to be sent together in the upstream.

Concatenation enables bypassing of the bottleneck that was highlighted above in Example 1. Example 2 illustrates that with concatenation, the maximum bandwidth of the downstream may be reached.

Example 2: TCP Performance in a Docsis 1.1 Plant

From Example 2, designers may conclude that the TCP performance bottleneck is solved by the mere usage of concatenation. While concatenation does provide some improvement, the improvement is limited to cases where only a single TCP session is active and does not address all real world scenarios.

Docsis 2.0 isn't the Answer
While the migration from Docsis 1.0 to 1.1 introduced several new MAC mechanisms meant to enable quality of service (QoS) required by applications such as voice services, the DOCSIS 2.0 specification focused on improving the upstream PHY. The improvement addresses two issues: throughput and robustness. The upstream modulation was enhanced to support 64-level quadrature amplitude modulation (64QAM) and a baud rate of 5.12 Mbaud/sec was added, enabling upstream transmission of approximately 30 Mbit/s.

Improving the capacity of the upstream channel enables greater data transmission. This is a significant improvement and allows users more bandwidth for sending information, which is important for applications such as file sharing and gaming. It is also important to note that the additional upstream bandwidth introduced by DOCSIS 2.0 does not directly improve TCP performance. As was shown above, the bottleneck for sending the ACKs is the MAC protocol that enables only a limited number of bursts per second.

Affect of Bi-Directional Traffic on TCP Performance
So far we discussed the performance of a single TCP session assuming the modem is not doing other upstream transmissions. In the presence of upstream traffic, the performance of TCP sessions sending downstream data degrades. Upstream traffic can be any type of traffic, TCP or other, which originates from the client side and is then transmitted via the cable modem to the Internet. Upstream transmissions result from emails and attachments, clicking on a web site, file sharing, and gaming applications and in many other scenarios.

Example 3 re-examines TCP performance by assuming that the cable modem is also sending traffic upstream.

Example 3: TCP Performance—Docsis 1.0+Upstream

The resulting performance degradation shown in Example 3 for TCP applications is significant. In Example 1 all bursts were used to transmit ACKs, in Example 3 only one third of the bursts are used for ACKs. Two-thirds of the bursts are used to send other data. The number of ACKs per second is reduced to one third of its original number. This translates directly to a drop of two-thirds in the downstream TCP performance.

In Example 4, a DOCSIS 1.1 cable modem is operating with downstream and upstream traffic. It is assumed that the data packets are defined to be 1518 bytes in length eliminating the possibility to concatenate ACKs with the data packets being sent in the upstream.

Example 4: TCP Performance—Docsis1.1+Upstream

The number of ACKs that can be sent in the upstream per second will vary. It now depends on the number of ACKs that are concatenated together. Furthermore, it is assumed that the number of ACKs that are concatenated is approximately three.

As Example 4 points out, the effect of the upstream traffic is very significant on the TCP performance even on DOCSIS 1.1 cable modems. Performance of the DOCSIS 1.1 cable modems is expected to be significantly better than that of DOCSIS 1.0 cable modems, but there is still significant degradation of performance compared to the case when the cable modem is doing downstream only.

Filtering ACKs
As defined in Docsis, a cable modem does not inspect the contents of packets that it transmits. By having a cable modem inspect the packets—called adding awareness— it is possible to improve the cable modem performance for TCP applications.

For example, if a cable modem has multiple ACKs to transmit for previously received packets, it can transmit only the most recent ACK. The TCP protocol considers this ACK to indicate that all previous packets were properly received by the client. Thus, this ACK includes all the information conveyed by all the ACKs preceeding it and these ACKs can be discarded. The process of sending only the most recent ACK and discarding the ACKs not yet transmitted is called ACK filtering.

Example 5 illustrates the improvement expected in downstream TCP performance when there is bi-directional traffic (simultaneous upstream and downstream).

Example 5: TCP Performance—Docsis1.0+Upstream+ACK Filtering

Example 5 shows that a DOCSIS 1.0 cable modem with ACK filtering enabled achieves performance of 30 Mbit/s in the presence of upstream traffic. In the example, only 100 of 300 bursts are transmitted. With ACK filtering enabled, this is the equivalent to transmitting 1000 ACKs. The performance impact is that the cable modem is now able to achieve 30 Mbit/s downstream TCP.

It's important to note that ACK filtering does not provide a tradeoff in the system architecture. There is typically a tradeoff between performance improvement and the additional bandwidth required to achieve the improvement. This is not the case for ACK filtering. Performance is improved while eliminating the need to transmit some packets that were scheduled. Dropping unnecessary packets increases upstream bandwidth.

About the Author
Lior Storfer is a senior member emeritus of the technical staff at Texas Instruments. He currently serves as the product marketing and architectures manager for Texas Instruments Broadband Cable Group. Lior received a bachelor's degree in computer engineering from Technion (Israeli Institute of Technology) and a master's degree in business from Tel-Aviv University. He can be reached at slior@ti.com.