Physical-Layer Solutions for Switch and Router Backplane Interconnect

As the speed and port density of switches and routers increase, the design strain on interconnect within these systems become more difficult. This article discusses the various options today's designers have to help circumvent this bottleneck.

Switch and router backplane design is one of the most difficult problems faced by system engineers. Increasing port speeds from 10 Mbps Ethernet to 100 Mbps, next to 1 Gbps (and soon to 10 Gbps), increases the strain on the interconnect or backplane within the box.

The backplanes of most switches and routers are redesigned for every generation. These backplanes must support the aggregate bandwidth of the ports on all installed linecards. To extend the life of the backplane, designers require a solution that is scalable, allowing bandwidth for both present and next-generation systems and linecards.

Backplanes must also meet the numerous size requirements of the system. Most backplanes are constrained by the size of the connector and the number of signals required to support the target aggregate bandwidth. The chassis width and card spacing limits the number of cards on the backplane. These factors must be considered in order to be both technically sound and cost effective.

Traditionally, the interconnect topologies used for backplane connections have been bus architectures. Bus architectures use multiple data lines and in-band or out-of-band addressing and control signals. The architectures usually connect to multiple different source and termination points. In these buses, the bandwidth is shared. An example of a bus used for backplanes is PCI. PCI has 32 data signals operating at 33 MHz. While total bandwidth is approximately 1 Gbps, this bandwidth is shared between multiple linecards. In reality, the effective bandwidth for each linecard is 1/n, where n is the number of linecards on the bus. There is a problem with this type of "solution". As port speeds increase, it becomes increasingly difficult for bus architectures to scale. Designers can increase the bandwidth of a bus architecture by increasing the number of signals and increasing the data-carrying capacity of each signal. The problem with increasing the number of signals is that connector size becomes an issue. The problem with increasing the signaling speed of the signals is increased cross talk and noise from adjacent signals and reflections due to multiple termination points.

In recent years, many designers have migrated their backplanes from traditional bus to a point-to-point topology. This new topology allows each linecard to have dedicated bandwidth to the switch fabric. It also allows each signal to carry more information since there is only a single termination point reducing the amount of noise due to crosstalk and reflections.

Most system designers use differential signaling to implement point-to-point connections because of the capability of differential signaling to transfer data at high speeds. Two of the most popular driver choices are Low-Voltage Differential Signaling (LVDS) and Differential Current Mode Logic (CML). This paper describes each of these technologies along with their benefits and weaknesses as they apply to backplane interconnect.

The ANSI/TIA/EIA-644-1995 and IEEE 1596.3-1996 standards both define LVDS. In addition, LVDS is also the signaling used for the Optical Internetworking Forum SERDES"Framer Interface (OIF-SFI-4-01.0), System Physical Interface (OIF-SPI-4-02.0), Packet over SONET (POS) Level 4, UTOPIA level 4, Rapid I/O, and other standards. These bus standards use LVDS as the signaling type and typically refer to either the ANSI/TIA/EIA or IEEE standard for all or part of the signaling requirements. The ANSI/TIA/EIA standard is generally considered a superset of the IEEE 1596.3-1996 standard. The maximum signaling rate defined by these standards is 200 Mbps and 655 Mbps respectively.

LVDS I/O buffers are often built with CMOS transistors. Figure 1 shows an output buffer configuration for LVDS. This buffer operates by steering current through the 100W termination resistor (located at the remote receiver). For a logical "one", the current flows from the current source through the top transistor indicated with the "+", through the termination resistor, and to ground through the bottom transistor indicated with the +. For a logical "zero", the current flows from the current source through the transistors indicated with a "-" and the termination resistor. During steady-state conditions, these transistors are either in saturation or cutoff, and current will not flow from the current source to ground without going through the termination resistor. While this provides for low-power operation at low frequencies, it limits the speed at which the transistors are able to switch.

Figure 1: The LVDS output buffer, 50W termination, 100W termination resistor, and LVDS receive buffer

LVDS is defined to operate DC-coupled into 50W interconnect with 100W balanced termination. Most LVDS receivers have a common-mode range of approximately 1.0V. LVDS drivers typically require a 2.5V supply and generate a single-ended output swing of 400mV centered on 1.2V.

Differential CML is the signaling technology used for many newer interfaces. It is now found in optical modules, 1G/10G Ethernet, XAUI, Fibre Channel, ESCON, InfiniBand, and other interfaces, often as a replacement for ECL/PECL. Many of the physical-layer devices that service these standards operate at greater than a 10 GBd signaling rate.

You can implement Differential CML with either CMOS or bipolar transistors. Figure 2 shows both implementations of output buffers for Differential CML. For a logical "one", current travels from the current source, through one of the resistors in the output buffer, through the termination resistor (located at the receiver) and through the transistor and into the current source. For a logical "zero" the current flows through the other resistor and transistor in the output buffer. This is similar to LVDS except that the transistors for Differential CML operate in the linear range. Because of this linear operation, there is always a small current that flows from the current source to ground through both resistors and transistors in the output buffer. By operating the transistors in their linear range, the buffer can operate at very high frequencies. Unfortunately, the static currents used to keep these transistors in their linear region tend to increase their overall power dissipation at low frequencies. The ability to use bipolar transistors also gives chip designers the option to build higher speed buffers with higher drive capability.

Figure 2: The top figure shows a bipolar implementation of a CML-based output buffer. The bottom figure shows a CMOS implementation of a CML buffer.

Similar to LVDS, Differential CML also has a relatively low signal swing. The single-ended swing for most Differential CML drivers is between 500mV and 800mV, but can be set to almost any level by simply adjusting the current source. Differential CML outputs typically switch between V_cc and V_cc-0.6V. Typical receivers have a common-mode range of approximately 1.0V. Differential CML drivers can operate into a wide array of load impedances, including the 50W or 75W impedance lines specified by most high-performance serial interface standards. It is possible to implement these buffers with V_cc supplies as low as 1.2V.

Unlike LVDS (which by definition is DC coupled), Differential CML works equally as well in DC- or AC-coupled connections. AC coupling is done by placing coupling capacitors next to the transmit or receive buffers. Some interfaces such as Fibre Channel copper connections, require AC coupling at both ends of the link. AC coupling allows the use of different power supplies for the transmitter and the receiver and also allows the voltage swing at the receive buffer to be "centered" within the common-mode operating range of the receiver to maximize the common-mode noise tolerance of the receive buffer. This is called DC restoration and helps prevent errors due to noise.

A Differential CML output buffer can transmit data over relatively long distances. The specification for 1 Gbps Ethernet over twinax (IEEE 802.3z) is 25 meters at 1.25 GBd. The draft specification for 10 Gbps Ethernet (IEEE 802.3ae) requires a 4x3.125 GBd interface (XAUI) to transmit 0.5m over FR4. Differential CML also allows a seamless connection with most optical modules, which can extend the transmit distance to many kilometers.

Following are two examples of how a designer may implement a backplane requiring 10 Gbps of bandwidth using LVDS or Differential CML. The examples assume that no overhead is required, so that the total bandwidth required is 10 Gbps.

The first example implements a backplane with the System Physical Interface 4 phase 2 (SPI-4 phase 2; OIF-SPI4-02.0). SPI-4 phase 2 specifies LVDS signaling for the 16-bit data bus, clock, and control pins. The bus requires a total of 78 pins for bi-directional communication operating at 622 Mbps, with some of the clock signals operating at 622 MHz. Due to the number of required pins and the tight setup-and-hold window, skew control and timing require major design effort. Power consumption is relatively moderate and the transmission distance is relatively short due to the number of pins.

The second example implements a backplane with the 10G Ethernet standard XAUI (10G Attachment Unit Interface). XAUI specifies Differential CML signaling levels for the four differential signals with an embedded clock, each signal operating at a 3.125GBd signaling rate. This bus requires 16 pins for bi-directional transmission. The protocol layer manages lane-to-lane skew, while the clock and data recovery (CDR) in the SERDES manages the timing recovery. This allows system engineers to distribute the clock on their board, while the embedded clock-tolerance management functions in the XAUI interface allow local functions to remain in a common time domain. The transmission distance is relatively long and the power level is relatively low.

While LVDS provides an excellent interface for relatively slow-to-medium signaling rates for chip-to-chip communications, Differential CML provides the best solution for high- to ultra-high-bandwidth backplane interconnection. This driver delivers much higher bandwidth per pin, relatively long transmission distances (including support for pre- or post-transmission equalization), allows interfacing to optical modules, consumes relatively low power, and can be both AC- or DC-coupled. Many of the standards-compliant products available on the market today, such as XAUI, are implemented with Differential CML for the high-performance serial I/O.

About the Author Michael Bollesen is director of strategic marketing for the data communications division at Cypress Semiconductor, San Jose, CA. You can contact him at mvb@cypress.com or at 408-943-2782.