Data flow must be 'seen' in router debug

Debugging a prototype of a router based on a network processor unit requires visibility into the details of the data flow through the various subsystems. And when simulation doesn't work or isn't practical, the tool that provides the best visibility is a logic analyzer.

Routers can be broken into three main subsystems: packet-forwarding cards, switch fabric and system control cards. Packet-forwarding cards perform data transmit and receive operations, a portion of the routing-table lookup and scheduling. The switch-fabric subsystem contains a high-speed crosspoint switch and the system controllers perform routing decisions and system management and monitoring operations.

The system board typically communicates with the line cards through the control bus, often a CompactPCI interface. The "traffic manager -a part of packet-forwarding cards-can be any combination of FPGAs, ASICs and programmable network processors. This is where route lookup, policy management and priority scheduling occur.

Logic analyzers can be connected to signals at the interfaces between chips, and sometimes they can be connected to on-chip debugging interfaces to view data flow inside of FPGAs and network processors. The other value of a logic analyzer is the ability to correlate data from multiple interfaces onto a single time domain. This makes it possible to track the flow of data through a system, rather than debugging from a single bus. Data from data path buses, memory interfaces and microprocessor buses can all be viewed in a single time domain.

Some next-generation I/O buses, however, have changed the rules of logic analysis. Traditionally, a new revision of a bus standard would quadruple its aggregate bandwidth by doubling both the bus width and the clock speed. For example, to go from OC-12 (622 Mbits/second) to OC-48 (2.5 Gbits/s), the POS-PHY Level 3 bus standard doubled the bus width and clock speed over its predecessor, Level 2.

However, OC-192 system designers realized that routing a 64-bit-wide bus with a 200-MHz clock would come with board-routing, power-consumption and crosstalk problems. Low-voltage differential signaling, or LVDS, has been shown to be effective at much higher clock rates.

Using LVDS at over 800 Mbits/s, 16 channels (32 wire pairs) can provide the same aggregate bandwidth as 64 channels running at 200 MHz, with less power consumed and less crosstalk between channels. These buses typically have a clock speed of one-half the bit rate. For example, a bus with 800 Mbits/s per channel would use a 400-MHz clock and transfer data on both rising and falling edges. Example implementations include POS-PHY Level 4 (also known as SPI-4), RapidIO and HyperTransport.

Capturing 1.25 Gbit/s

To capture data flow on these buses requires a logic analyzer capable of capturing state-mode data at up to 1.25 Gbits/s (625-MHz clock, using both edges). Also, with a high-sensitivity probing front end, passive probing is possible with minimal interference. And with a memory depth of 64M on 34 channels, up to 2.6 million 100-byte frames can be captured in real-time.

At high speed and low voltage, traditional logic analyzer probes would interfere with the successful transmission of signals, which defeats the purpose of testing. Even small amounts of additional metal on a trace add capacitance, which reduces bandwidth.

For this reason, a special low-capacitance surface-mount connector is the preferred way to probe LVDS signals up to 1.25 Gbits/s. The best way is a probing system with only 1.5 picofarads of probe tip capacitance, including the connector, which also must be optimized-especially for logic analysis measurements. In this probing solution, ground pins located between every pair of signal pins provide excellent channel-to-channel isolation at high speeds.

The 1.5-pF capacitance is added by the surface-mount mating connector, and the remaining resistors and capacitor come from the probe's termination network and pod cable. What isn't included in this equivalent load is the effect of the added stubs that lead from the signal trace to the probe connector. The stub, acting as a transmission line, can degrade the bandwidth of the target signal. The trace can be isolated from the transmission-line capacitance by placing a series resistor as close as possible to the signal trace. This resistor and the transmission line to the probe tip degrade the bandwidth of the signal arriving at the probe tip, so resistor values and stub lengths must be chosen carefully to avoid degrading the signal arriving at the probe.

There are two requirements to meet in designing in the probing: adequate isolation of the target trace from the probing system and a minimum required eye opening at the probe tip of 500 ps wide and 200 mV high.

At high speed, capturing data within very small data-valid windows becomes key. The sampling position relative to the clock edge must be fine-tuned individually for each channel. The logic analyzer used must allow for adjustment of the setup-and-hold position in increments of 10 ps, and sampling on a minimum data-valid time of 500 ps.

A new way to do this involves automatically computing the optimal sampling position on all channels. This technique uses the logic analyzer's comparators and sampling hardware to find stable regions on each channel. The appropriately configured software automatically positions the analyzer's sampling position in the center of the stable region. This approach, while not a replacement for a high-performance oscilloscope, allows for a quick perspective on signal integrity on many channels simultaneously, in addition to easier configuration of the logic analyzer.

Decoded view

Logic analyzers have many features for providing a high-level decoded perspective of data flow on standard data path buses. If you can connect your logic analyzer to a standard bus you may be able to see very high-level analysis of data traffic. For example, data from a Gigabit Ethernet 10-bit interface decoded from 8B/10B codes into packet data can be displayed in a text-based decode.

The decoded view can help solve problems in chips that do not comply with bus specs; when a bus does not comply, the decode display will fail. Examples of such failures includes preambles or carrier extension words of invalid length; invalid 10-bit codes; sequencing errors (Start of Packet found between the Start of Packet and End of Packet); and data corruption (invalid FCS checksums).

Decoding data from a single bus is useful, but the true power of a logic analyzer lies in the ability to bring data from multiple buses together on a single time scale. For example, with a logic analyzer, it is possible to see a packet traveling through a Gigabit Ethernet router. Suppose that packet then enters a programmable network processor, which stores it in DDR buffer memory while deciding where to send it and when to schedule it before reading it back out and sending it on its way. A logic analyzer's time correlation allows measurement of latencies between subsystems and components, in addition to providing basic visibility. This type of measurement can be used to solve problems such as excessive latency in packet forwarding and dropped or lost packets.

Often the symptom of a design problem can be seen in another test instrument, such as a protocol analyzer or traffic generator and analyzer. However, since those instruments can only see packet traffic on the external interfaces of the system, they cannot pinpoint the source of the problem on their own.

In such a case, an external trigger signal can be passed from the other instrument to the logic analyzer to capture data and signal flow during the period leading up to the failure. Using the deep acquisition memory of the logic analyzer, a designer can then look back in time before the failure to track the system's behavior. This technique can be used for solving problems such as incorrect packet forwarding and insufficient system throughput.

The logic analyzer is a key tool in problem solving during the turn-on of new router prototypes. It provides high-level visibility of data flow between chips. Concurrent visibility across multiple domains makes it possible to pinpoint the source of failures and performance issues. As speeds increase and voltages decrease, logic analyzer probing and acquisition technology is keeping pace, and will continue to do so in the future.

Scott Ferguson is a software engineer at Agilent Technologies Inc. (Colorado Springs, Colo.).

See related chart

http://www.eet.com