<B>Exclusive:</B> Developing a flexible FPD inspection system

Exclusive: Developing a flexible FPD inspection system
The popularity of flat-panel displays (FPDs), including plasma display panels and liquid-crystal displays (LCDs), has given rise to the need to streamline their production. FPDs are increasingly replacing cathode-ray tube (CRT) screens, and are at a critical juncture where high availability and lower prices will drive further customer adoption. As a result, FPD manufacturers such as Pioneer, NEC, Fujitsu, Matsushita and Sony are currently making large investments to drive their costs down.

One of the most important, potentially time-consuming and costly phases of FPD production is the inspection process, in which automated optical inspection and image processing techniques are used to identify production defects.

Historically, the DSP functions required by this application would be implemented in DSP processors. However, much higher speed DSP functions can be built using programmable logic, which retains the needed flexibility of DSP processors while achieving several magnitudes of a performance advantage compared to normal multi-DSP processor systems.

If performed efficiently and economically, the FPD inspection process can reduce manufacturing costs significantly. For example, in the case of PDPs, an inspection is performed on two glass panels before they are bonded together to form a single complete panel. If defects are found in either of the two panels, they can often be repaired and reintroduced to the manufacturing process, resulting in yield recovery. If a defective panel is bonded to a good panel, the resulting complete panel must be scrapped in its entirety.

During the prototyping and early manufacturing stages of PDP products, inspections were generally performed by a single charge-coupled device (CCD) sensor moving snake-like across the surface of the panel, taking as long as 60 seconds to complete. This kind of inspection produced a data stream of about 160 Mbytes per second. Currently, these types of inspections are being replaced with multiple parallel sensors inspecting a panel in a single pass.

Further, the resolution of the optical inspections is being increased to identify smaller defects and increase the quality of the resulting product; whereas earlier PDP inspections had a resolution of 5 - 7 microns, current inspections on LCD panels are being done at 1-micron resolutions. These factors are increasing the bandwidth of visual data that must be processed to several gigabytes per second.

Early FPD inspection systems relied on DSP and other special purpose processors to implement the inspection algorithms. An early system developed by Avvida consisted of 21 function-specific boards in a VME chassis using discrete logic, transputers, and DSP processors, and cost over $100K US at the time. As the inspections grew more demanding and data bandwidths increased beyond the capability of DSP processors, Avvida turned to programmable logic and a new processing architecture called JEDI to accelerate the DSP functions.

The following system Avvida developed was based on 12 Altera FLEX 10K FPGAs occupying three boards, and as a result benefited from a 50 percent reduction in the amount of hardware complexity and corresponding cost. In addition, the move to programmable logic allowed Avvida to develop a product with which they could focus on developing the algorithms as opposed to recreating the hardware with each new generation or version of the product. This reliance on hardware reconfigurability makes investment in algorithm and application development easily transferable to the next generation of programmable logic as it becomes available.

In Avvida's latest image processing system (called Tsunami), one Altera Stratix FPGA resides on a Tsunami board, and additional Stratix FPGAs can be added for additional processing power via JEDI II daughter card modules. The FPGAs allow an input bandwidth of over 3 Gbytes/s and processing speed in the tera-ops range at a total system cost of less than $10K US. The algorithms involved in FDP inspection require large amounts of rapid-access integrated memory and several high-speed image processing functions such as convolution, erosion, dilation, and comparison. A next generation FPD inspection algorithm implemented with the on-board FPGA and another FPGA on a JEDI II module is shown in Figure 1.

Figure 1: A next-generation FPD inspection algorithm consists of several stages that are heavily reliant on large amounts of integrated, high-speed memory and dedicated DSP building blocks that are present in the latest high-density FPGAs.

Next-generation inspection algorithm

In the Pixel Reorder block, data from multiple digital cameras is brought into the system in many independent streams called "camera taps". The individual taps are re-formatted to produce a single contiguous image. This block relies on high-speed data storage and uses the FPGA's M4K memory blocks. To compensate for irregularities in optical configuration, the Normalization block performs individual pixel correction, gain and offset. Normalization uses the FPGA's DSP blocks as well as the large on-board M-RAM blocks for look-up tables and correction curves. Next, the Feature Processor identifies the location of certain features (end points, crosses and T-junctions) in the structure of the fine line conductors linking each pixel to the driver circuitry. This technique, called "skeletonization," uses a M4K block, an M-RAM block, and an external DRAM to store feature data that the host can read from and write to. The Feature Processor then flags additional or missing features as defects.

In parallel with the Feature Processor, the data also undergoes a template comparison, in which the individual display cells are compared one to the next. Any cell-to-cell irregularity is flagged a defect. The comparison algorithm uses M4K blocks and two external DRAM memory banks to store and delay the image data corresponding to the prior cell. In the Tsunami system, this template comparison can also be performed using a known good or "golden" template. Also in parallel with the Feature Processor and Template Comparison is the Design Rule Checking (DRC) stage. The DRC stage uses DSP blocks to gauge the interconnect trace dimensions to predefined minimum and maximum limits. Features smaller or larger than the design rule are considered defects.

The output of the Feature Processor, Template Compare and DRC blocks is then fed into the Defect Alignment block, which aligns the data steams using the FPGA's M-RAMs and compares them to further identify defects. The resulting defect data is then routed to the Blobbing and Defect Statistics block, where a blobbing function extracts detailed statistics about found defects. Defect type, location, orientation and size are passed on to the host application for defect analysis and reporting. The identified defects are also passed on to the Defect Overlay and Capture block, which allows the equipment operator to quickly identify defective areas on the panel under inspection. Captured defect images are also sent to the host and stored with the defect statistical data for further review and analysis.

Overseeing the real-time operation of the Tsunami board is the Nios embedded processor. A local controller such as the Nios processor is needed since the host processor OS in a PCI-based system may not be able to respond in real time. The Nios processor controls the overall image processing portions of the inspection process and is responsible for initiating, idling or stopping the process as needed. The Nios processor also handles passing the data to the host. Integrating the processor into the on-board FPGA eliminates the need for an external processor, saves on board development and other engineering costs and keeps the system complexity in check.

Critical reconfigurability

The reconfigurability of Avvida's inspection systems allows them to be optimized to a high degree for specific applications. In the case of FPD inspection, the algorithms in the FPGAs are developed to correspond to the specific requirements of the environment at the manufacturer's inspection site. These algorithms are further tuned iteratively until the desired results are obtained. Differences in CCD sensors, emphasis on detecting different types of defects, and variations resulting from processes changes can be addressed with refinements to the algorithms programmed into the FPGAs. Further, the system can be modified via FPGA reconfiguration to handle inspection of different sizes or types of panels, allowing the end user to a high degree of flexibility with respect to the system's application. By building a modem into the system, Avvida can also reconfigure their systems remotely, addressing customer needs for upgrades, system adjustments, enhanced functionality, or completely new functions.

Building reconfigurability into these systems also decreases their associated development costs. Avvida can focus R&D investment on algorithm development rather than hardware development, then take advantage of the next generation of FPGAs by building new daughter boards rather than rebuilding the whole system. The algorithms are easily migrated from one FPGA design to the next, and new applications can be developed in a matter of days or weeks, allowing Avvida to reduce their engineering costs. Avvida estimates that they are saving 70 percent of the engineering resource time using this approach compared to past hardware design approaches.

Conclusion

Today's FPGAs are ideal for high bandwidth image processing tasks in terms of their feature set and density. With large amounts of flexible, on-board memory and dedicated structures for implementing DSP functions, the latest generation of programmable logic technology allows hardware designers to be more creative in tackling DSP-oriented tasks. With the availability of mature embedded processors for programmable logic, DSP designers can continue enjoying the benefits of a software-based design flow while taking advantage of the hardware acceleration traditionally offered by FPGAs. These advantages, currently enjoyed by FPD inspection systems, can easily be extended to other image processing and DSP applications as well, and DSP designers will increasingly need to consider programmable logic if they want to deliver best-in-class solutions.

Brian Tithecott has a B.Sc. in computer science from the University of Western Ontario and an M.B.A. from Wilfrid Laurier University. He is Director of Marketing and Business Development at Avvida Systems. Avvida Systems is an Altera ACAP partner providing FPGA-based image processing solutions for industrial, medical, broadcast and military applications. Contact Brian at Brian.Tithecott@avvidasystems.com.

Martin S. Won is a senior member of technical staff at Altera Corp. He has over ten years of experience in digital systems design involving programmable logic. He holds a BS in Electrical and Computer Engineering from the University of California at Santa Barbara.