Design Article
Video and image processing design using FPGAs
Brian J. Jentz <br> Altera Corp.
7/7/2006 12:40 AM EDT
New trends in video and image processing are forcing developers to re-examine the design architectures they have used previously when considering the numerous tradeoffs of using different architectures that are key to the decision process.
Consumer demand and exciting innovations, such as HDTV and digital cinema, revolve around video and image processing and the rapid evolution of the technology. Major advancements in image capture and display resolutions, advanced compression techniques, and video intelligence are the driving forces behind these technological innovations. At the same time, rapid change in standards and higher resolutions are pushing designers away from off-the-shelf technology.
Resolutions in particular have increased dramatically in just the last few years. The following table illustrates current state-of-the-art resolutions in different end types of applications.
Table 1: Resolutions by Application Types
The move from standard definition (SD) to high definition (HD) represents a 6X increase in data needing to be processed. Video surveillance is also moving from the Common Intermediate Format (CIF) (352 x 288) to the D1 format (704 x 576) as a standard requirement, with some industrial cameras even moving to HD at 1280 x 720. Military surveillance, medical imaging, and machine vision applications are also moving to very high resolution images.
Advanced compression techniques are replacing previous generation technology, offering enhanced streaming capability, higher compression for a given quality, and lower latency. As the standard for digital cinemas, JPEG 2000 is gaining momentum in military, medical imaging, and surveillance. H.264 is expected to replace MPEG2 in broadcast television applications, MPEG4 Part 2 in video surveillance systems, and H.263 in videoconferencing. Even as these new compression solutions are deployed, ongoing standards activity continues to enhance H.264 and JPEG 2000 standards.
The DICOM medical imaging standard has finalized Supplement 105, including multi-component transformations in Part 2 of JPEG 2000 for the compression of 3D medical imagery. Supplement 106 will include JPIP as a protocol for remote browsing of compressed medical images using JPEG 2000.
The next extension to MPEG 4 Part 10 (H.264 AVC) is Scaleable Video Coding (SVC). SVC addresses coding schemes for reliable delivery of video to diverse clients over heterogeneous networks using available system resources, particularly where the downstream client capabilities, system resources, and network conditions are not known in advance. For example, clients may have different display resolutions, systems may have different caching or intermediate storage resources, and networks may have varying bandwidths, loss rates, and best-effort or quality of service (QoS - refers to the maximum frequency at which the design must operate) capabilities. An extension of AVC/H.264 is being developed by the Joint Video Team (JVT) to provide scalability at the bitstream level, with good compression efficiency, and allowing free combinations of scalable modes (such as spatial, temporal, and SNR/fidelity scalability). Application areas include video surveillance systems, mobile streaming video, wireless multi-channel video production and distribution, and multi-party video telephony/conferencing.
Another rapidly evolving market is video intelligence. Cameras have had the ability to pan, tilt, zoom, and panorama, but these will be driven increasingly by system intelligence rather than manual intervention. Motion detection allows more efficient hard disk storage by only archiving video frames where a motion threshold is passed. The promise of video object recognition would allow for automated surveillance monitoring, which is much more effective than manual surveillance monitoring.
With expanding resolutions and evolving compression, there is a need for high performance while keeping architectures flexible, thus allowing for quick upgradeability. As technologies mature and volumes increase, there will of course be the ever-present need to reduce costs.
Video and Image Processing System Architectures
System architecture choices include standard cell ASICs, ASSPs, and programmable solutions such as digital signal processing (DSP) or media processors and FPGAs. Each of the approaches has advantages and disadvantages, with the ultimate choice depending on end equipment requirements and solution availability. Given the trends discussed, the ideal architecture should have the following characteristics: high performance, flexibility, easy upgradeability, low development cost, and a migration path to lower cost as the application matures and volume ramps.
Next: Performance and development costs
Consumer demand and exciting innovations, such as HDTV and digital cinema, revolve around video and image processing and the rapid evolution of the technology. Major advancements in image capture and display resolutions, advanced compression techniques, and video intelligence are the driving forces behind these technological innovations. At the same time, rapid change in standards and higher resolutions are pushing designers away from off-the-shelf technology.
Resolutions in particular have increased dramatically in just the last few years. The following table illustrates current state-of-the-art resolutions in different end types of applications.
Table 1: Resolutions by Application Types
The move from standard definition (SD) to high definition (HD) represents a 6X increase in data needing to be processed. Video surveillance is also moving from the Common Intermediate Format (CIF) (352 x 288) to the D1 format (704 x 576) as a standard requirement, with some industrial cameras even moving to HD at 1280 x 720. Military surveillance, medical imaging, and machine vision applications are also moving to very high resolution images.
Advanced compression techniques are replacing previous generation technology, offering enhanced streaming capability, higher compression for a given quality, and lower latency. As the standard for digital cinemas, JPEG 2000 is gaining momentum in military, medical imaging, and surveillance. H.264 is expected to replace MPEG2 in broadcast television applications, MPEG4 Part 2 in video surveillance systems, and H.263 in videoconferencing. Even as these new compression solutions are deployed, ongoing standards activity continues to enhance H.264 and JPEG 2000 standards.
The DICOM medical imaging standard has finalized Supplement 105, including multi-component transformations in Part 2 of JPEG 2000 for the compression of 3D medical imagery. Supplement 106 will include JPIP as a protocol for remote browsing of compressed medical images using JPEG 2000.
The next extension to MPEG 4 Part 10 (H.264 AVC) is Scaleable Video Coding (SVC). SVC addresses coding schemes for reliable delivery of video to diverse clients over heterogeneous networks using available system resources, particularly where the downstream client capabilities, system resources, and network conditions are not known in advance. For example, clients may have different display resolutions, systems may have different caching or intermediate storage resources, and networks may have varying bandwidths, loss rates, and best-effort or quality of service (QoS - refers to the maximum frequency at which the design must operate) capabilities. An extension of AVC/H.264 is being developed by the Joint Video Team (JVT) to provide scalability at the bitstream level, with good compression efficiency, and allowing free combinations of scalable modes (such as spatial, temporal, and SNR/fidelity scalability). Application areas include video surveillance systems, mobile streaming video, wireless multi-channel video production and distribution, and multi-party video telephony/conferencing.
Another rapidly evolving market is video intelligence. Cameras have had the ability to pan, tilt, zoom, and panorama, but these will be driven increasingly by system intelligence rather than manual intervention. Motion detection allows more efficient hard disk storage by only archiving video frames where a motion threshold is passed. The promise of video object recognition would allow for automated surveillance monitoring, which is much more effective than manual surveillance monitoring.
With expanding resolutions and evolving compression, there is a need for high performance while keeping architectures flexible, thus allowing for quick upgradeability. As technologies mature and volumes increase, there will of course be the ever-present need to reduce costs.
Video and Image Processing System Architectures
System architecture choices include standard cell ASICs, ASSPs, and programmable solutions such as digital signal processing (DSP) or media processors and FPGAs. Each of the approaches has advantages and disadvantages, with the ultimate choice depending on end equipment requirements and solution availability. Given the trends discussed, the ideal architecture should have the following characteristics: high performance, flexibility, easy upgradeability, low development cost, and a migration path to lower cost as the application matures and volume ramps.
Next: Performance and development costs
|
|
|
|
|
Navigate to related information
