Testing a network device in a live network

As complexity grows, software simulation of a hardware design becomes more time-consuming and more difficult. Also, real-life situations and corner cases are more difficult to model using software simulation. Microprocessor companies have faced these problems for some time and almost unanimously have turned to hardware emulation. But emulation also has its drawbacks: Along with the need for specialized, expensive hardware with inflexible physical-layer interfaces, it requires the target system to be slowed down, thereby eliminating any semblance of real-world situations.

A third option exists: Using a standard PC running speed-matching software, many of the drawbacks can be overcome to speed and simplify the emulation process while offering more realistic emulation. As with most novel approaches, however, some caveats and precautions must be observed.

Emulation involves loading a design, usually at the RTL level, into an emulator from a company such as Axis Systems Inc., Quickturn Design Systems or Mentor Graphics Corp. The emulated design is plugged into a real motherboard, and-fingers crossed-an operating system is booted. The ultimate test is to run a real application, such as a word processor. Of course, the emulator runs orders of magnitude slower than actual silicon, but the procedure really puts a design under pressure to perform. With FPGA densities, speed and complexity having grown in recent years, companies can now create custom FPGA-based prototypes of their chips. (The terms emulation and prototype are used interchangeably here to mean either a commercial hardware emulator or a custom prototype.)

Slow going

Emulating a microprocessor, graphics chip or other complex design usually requires slowing down the target system to work with the slower emulation. Even fast FPGAs may not be fast enough to keep pace with high-speed targets. That presents problems for functions, such as buses, that have a fixed frequency independent of the processor, or timer interrupts that must occur at a fixed interval. Typically, a special, slowed-down target board is populated with peripherals to communicate with the slow emulator.

But that slowed-down solution doesn't work well, if at all, with network devices. Using emulation, the goal is to test the emulated network device in a live network with real packets to see how the design works in a real environment. A network usually consists of dozens or hundreds of devices-or millions, in the case of the Internet. It wouldn't be possible to slow them all down. Also, there are low-level and high-level timing constraints associated with a network. The Reduced Media Independent Interface (RMII) for connecting an Ethernet PHY to a switch ASIC, for example, specifies a 50-MHz clock with a specific duty cycle. Most commercial emulators can't run at that high frequency. Higher-level protocols require an acknowledgment that the packet was received. If there is no acknowledgment within a specified time, the packet is re-sent. If the emulator cannot respond quickly enough, it gets flooded with duplicate packets.

Another issue is the analog interface. An emulator is a purely digital device. Yet a physical network connection requires a PHY that shapes the analog signal so that it can be received correctly at the other end. Similarly the PHY receives an analog signal and converts it to digital. Connecting an emulator directly to a network would require a special analog interface to convert the slow digital signals to the correct analog ones.

Often, this speed mismatch is solved in hardware using a board with an Ethernet adapter on one end and a digital interface to the emulator on the other, but there are difficulties with that solution. First, hardware is expensive to manufacture and requires a long development time. It doesn't make sense to put a large effort and significant resources into a prototype that will be discarded when the actual chip is ready.

Second, the analog circuitry is unique to each network interface, so a different board would be needed for each interface to be tested.

Third, memories need to be designed into the interface board to store large bursts of data that may come from the network too quickly for the emulator to handle. The buffers must store the packet data and feed them slowly to the emulator. Managing the buffers requires fairly complex state machines in hardware. And the maximum buffer size is determined when the board is designed and cannot be increased without a redesign.

There is, however, a software solution to the speed-matching problem. The solution connects an emulated network device to a network via a standard PC running the speed-matching software. The emulator connects through a simple parallel cable to the standard PC parallel port. The network connects to the PC through a standard network interface card (NIC). The speed-matching software handles all of the low-level handshaking to keep the network happy and the low-level interface on the parallel port to keep the emulator happy. The speed-matching software buffers packets from the network in the PC's memory so that large packet bursts are not lost but can be fed slowly into the emulator. The memory management needed for the buffering is handled by the PC operating system so the buffers can be as large as the PC RAM and can utilize the virtual memory of the operating system to store packets on disk, if necessary.

This sophisticated memory management would be difficult, and expensive, to implement in hardware, but it is relatively easy to implement in software.

The speed-matching software sits on top of the operating system, as does any application software (see figure, page 92). Normally when an application program needs to access hardware, it does so through a call to an API of the operating system. The speed-matching software, however, has direct control of the drivers for the parallel port and the NIC. Thus the software can use fully tested hardware driver software and manipulate data without bothering the OS.

When using speed-matching software, there are a few issues of which to be aware. The first is the interface to the emulated device. The parallel port makes a nice interface because it is simple and is available on every PC manufactured. There are a number of limitations, though. First, the throughput is limited to a theoretical maximum of 2 Mbits/second, though reaching that rate is unlikely because of overhead in servicing the port. Faster interfaces can be used, such as SCSI or USB, but those are much more complex. The physical interface requires special transceivers on the network device prototype, which means extra hardware and extra debugging. Also, the protocol is complex, which means the logic in the prototype to transmit and receive packets over these interfaces will be more difficult to write and to debug. Some of the advantages of using a software solution are reduced with such approaches.

A better interface may be a special high-speed parallel port. Several companies sell fast, wide parallel ports on cards that can be inserted into the PC or on a PCMCIA card. These ports still have a simple interface in terms of circuitry and protocol, but they may be 16 or 32 bits wide, doubling or quadrupling the throughput of a standard parallel port.

When using the PC parallel port for the interface, be aware that the parallel port comes in different flavors, including bidirectional, enhanced parallel port (EPP) and extended capabilities port (ECP). These protocols are controlled in hardware, and certain chips implement some protocols but not others. In many cases, the user will need to change the CMOS settings to enable a particular parallel-port protocol. The more advanced protocols include timeouts that cannot be overridden. If the prototype network device cannot respond in time, the chip will time out, resulting in dropped bytes. It is usually possible to manipulate the signals directly to create your own protocol on the parallel port, without a timeout. In any case, accessing the parallel port at this low a level requires code that can access the OS kernel, a task that is possible but sometimes difficult and differs from one OS to another, making it less portable.

On the network side of the emulation, be sure to turn off all networking on the PC. You do not want any of the packets that are being transmitted back and forth to be intercepted by the PC operating system. Sometimes low-level networking may take place that is difficult to turn off. A symptom of this situation will be packets that seem to come out of nowhere and are transmitted to the prototype or to the network. If that happens, it means that some background task is pinging the network. Thus it is best to turn off all background tasks.

Implementation example

Implementing a network bridge using a hardware emulator provides an example of the speed-matching software ap-proach detailed above. The bridge connects two PCs in a peer-to-peer network configuration. Both PC1 and PC2 are connected to I/O ports of the emulator through separate PCs, each of which are running speed-matching software. PC1 will see a direct connection to PC2 and vice-versa, without seeing either of the PCs running the speed-matching software or the emulator. Files can be transferred directly between PC1 and PC2 through the emulated network bridge.

For that to work, a network bridge can be written in Verilog. A diagram of the bridge is shown (see figure, page 93). Packets coming from the parallel ports of two interface PCs have been converted by the speed-matching software to a protocol that is optimized for the parallel port, then sent to the emulator via the PC parallel port. In the emulator, the packets are then loaded into a 1,514-byte buffer that is part of a FIFO. The number 1,514 is used, as Ethernet specifies this as the maximum packet size. Unfortunately, packets less than the maximum size will not use up all bytes, resulting in inefficient use of the emulator memory. But memory management in Verilog is more difficult and complex than in software, as mentioned earlier. Creating this simplified FIFO structure reduces the complexity and, therefore, reduces the overall development time.

After going through the FIFOs, the packets are read out the other end, converted from the RMII protocol back to the non-standard parallel port protocol used by the speed-matching software, and sent on to the other PC.

Not shown in the figure on page 93 are additional status registers that can be accessed by the PC via the parallel port. The registers allow the speed-matching software on the PC to know when the FIFOs are full or empty and can be written or read. This allows the speed-matching software to interrogate the status of the bridge and throttle the packet transfer to make it more efficient. Network protocols allow packets to be dropped without serious consequences, but dropping as few packets as possible should be a goal of the design.

Also left off the figure are the state machine that controls the reading and writing of packets as well as the logic required to convert packet protocols.

Bob Zeidman (Bob@zeidmanconsulting.com) is the founder and president of Zeidman Consulting.

See related chart

See related chart