A Technical Overview
n most electronic imaging applications, images with high resolution (HR) are desired and often required. HR means that pixel density within an image is high, and therefore an HR image can offer more details that may be critical in various applications. For example, HR medical images are very helpful for a doctor to make a correct diagnosis. It may be easy to distinguish an object from similar ones using HR satellite images, and the performance of pattern recognition in computer vision can be improved if an HR image is provided. Since the 1970s, charge-coupled device (CCD) and CMOS image sensors have been widely used to capture digital images. Although these sensors are suitable for most imaging applications, the current resolution level and consumer price will not satisfy the future demand. For example, people want an inexpensive HR digital camera/camcorder or see the price gradually reduce, and scientists often need a very HR level close to that of an analog 35 mm film that has no visible artifacts when an image is magnified. Thus, finding a way to increase the current resolution level is needed. The most direct solution to increase spatial resolution is to reduce the pixel size (i.e., increase the number of pixels per unit area) by sensor manufacturing techniques. As the pixel size decreases, however, the amount of light available also decreases. It generates shot noise that de-
©DIGITAL VISION, LTD.
Sung Cheol Park, Min Kyu Park, and Moon Gi Kang
grades the image quality severely. To reduce the pixel size without suffering the effects of shot noise, therefore, there exists the limitation of the pixel size reduction, and the optimally limited pixel size is estimated at about 40 µm2 for a 0.35 µm CMOS process. The current image sensor technology has almost reached this level. Another approach for enhancing the spatial resolution is to increase the chip size, which leads to an increase in capacitance . Since large capacitance makes it difficult to speed up a charge transfer rate, this approach is not considered effective. The high cost for high precision optics and image sensors is also an important concern in many commercial applications regarding HR imaging. Therefore, a new approach toward increasing spatial resolution is required to overcome these limitations of the sensors and optics manufacturing technology. One promising approach is to use signal processing techniques to obtain an HR image (or sequence) from observed multiple low-resolution (LR) images. Recently, such a resolution enhancement approach has been one of the most active research areas, and it is called super resolution (SR) (or HR) image reconstruction or simply resolution enhancement in the literature -. In this article, we use the term “SR image reconstruction” to refer to a signal processing approach toward resolution enhancement because the term “super ” in “super 21
IEEE SIGNAL PROCESSING MAGAZINE 1053-5888/03/$17.00©2003IEEE
resolution” represents very well the characteristics of the technique overcoming the inherent resolution limitation of LR imaging systems. The major advantage of the signal processing approach is that it may cost less and the existing LR imaging systems can be still utilized. The SR image reconstruction is proved to be useful in many practical cases where multiple frames of the same scene can be obtained, including medical imaging, satellite imaging, and video applications. One application is to reconstruct a higher quality digital image from LR images obtained with an inexpensive LR camera/camcorder for printing or frame freeze purposes. Typically, with a camcorder, it is also possible to display enlarged frames successively. Synthetic zooming of region of interest (ROI) is another important application in surveillance, forensic, scientific, medical, and satellite imaging. For surveillance or forensic purposes, a digital...