Image capture and processing challenges--and solutions--in portable designs--Part III

The advent of wafer-scale manufacturing techniques has made possible the production of extremely compact camera modules at incredibly low cost. For reasons of physics, a small camera module will have inferior performance to a larger camera module, but the deficiencies can be corrected by exploiting the novel lens structures that become possible by the switch to wafer-scale manufacturing. Nevertheless, this merely preserves the status quo in terms of image quality and does nothing to enhance the user experience.

Designers of higher resolution camera phones have, until recently, been able to sell to consumers solely on the basis of the headline pixel count number. With the proliferation of camera phones (more than 80 percent of models now possessing one or more cameras), consumers have come to realize that picture quality and pixel count are not strongly connected. Indeed, the stunning pictures sent back by the Mars Rover vehicles were taken by ~1Mpixel cameras. Likewise, the designers of professional-grade cameras have known for a long time that to obtain the highest quality digital images requires the combination of optics with software.

As discussed in Part 1 of this article series, customers demand improved image quality and a range of embedded features in new camera phones, the most desirable of which are optical zoom, focus and low-light sensitivity, i.e., photography without flash. All of these features are relatively straightforward to implement provided the camera module is permitted to increase in height and cost, particularly in the case of zoom. Conventional zoom requires two lenses to move independently along the optical axis of the camera. This can be accomplished with miniature actuators, but the final product is large, power-hungry, has slow response time and is somewhat incompatible with the harsh survival envelope of portable electronic devices, particularly the "drop test." So, how can the camera module designer provide all of the features consumers desire and improve picture quality without affecting the form factor, reliability and above all, the cost of the camera module? The answer is software-enhanced optics.

Software-Enhanced Optics
Software-enhanced optics--or "smart optics"--is the technique of correcting for known optical effects by image processing. The optical effect can be an intrinsic defect that must be corrected or a deliberately introduced artifact that provides a feature or special effect. If the objective is merely to boost the picture quality, rather than invest in high-quality, precision optics, it is possible to use software to correct for known aberrations introduced by a lower-cost optical train. For example, if the constraints on size and cost mean that the picture corners will always be blurred to the same degree, software-enhanced optics can apply edge-sharpening algorithms to just those regions. The user is then happy with the picture because the inherent deficiencies in the camera module have been corrected or masked and the picture appears to be good in all areas. To be effective, the adjustments should be completely transparent to the user and require no intervention to use.

Having embraced the concept of software-enhanced optics, a world of new opportunities opens. The basis of the approach is to use a specialty lens that manipulates the optical rays during their passage through the camera to provide an intensity distribution on the imager with desired features. The manipulated image is not used as is; it needs further software correction. However, because the image was manipulated in a known manner, it can be digitally restored so high-quality output can be extracted. It is possible to implement many features with this approach, including full optical zoom with no moving parts, extended field depths, and small F-number optics for low-light environments.

A software-enhanced optics solution for zoom exploits the phenomenon that in a conventional optics train the density of information is not uniform over the field of view. The central region contains more data than the periphery. However an image sensor has a regular, two-dimensional pixels array. This means that a scene is under-sampled in the center of the imager while being over-sampled at the edges. The software-enhanced optics solution uses a specially designed fixed lens that provides intentionally non-uniform optical information density over the image area to match the quantized format of the solid-state imager. This is, in effect, the converse of the approach taken by nature. Many animals with single-aperture eyes, particularly birds of prey, have a standard lens, but a non-linear distribution of rods and cones in the retina. In both cases the resulting image is distorted, but can be rectified because the lens design and pixel distribution of the imager (or retina) are known.

For viewing at unity magnification, the algorithm has to compress details in the central portion of the field of view, where the special lens increases magnification and resolution. Thus, the compression does not degrade image quality and indeed, the software-enhanced lens solution is designed so that, in this mode, the picture quality is as good as in a conventional camera. When zoom is selected, the image borders are cropped off, and the already magnified center is retained. The image is then corrected for distortion. This is fundamentally different than digital zoom because magnification is the result of the lens action and is fixed at image capture, so that the zoomed image retains its high resolution. Software-enhanced optics can achieve up to 3x zoom.

An example of software-enhanced optics providing zoom is shown in Figure 7. This solution has no moving parts, is physically compact, rugged, virtually instantaneous, consumes negligible power and can be implemented at relatively low cost. It is greatly superior to digital zoom. Digital zoom involves cropping and expanding the image to fill the field of view. This decreases resolution because the available information has to be spread over a larger area. In a 3x digital zoom, approximately 90 percent of the information quantity in the captured image is lost, which is why digital zoom can only provide a small amount of magnification before the picture quality becomes unacceptable. The image enhancement solution for zoom is accomplished by a fixed lens and a simple algorithm. This makes it suitable for all imager technologies and all resolutions from QCIF to >10Mpixel, so broad adoption on camera phones is expected in the near term.