2998 IEEE JOURNAL OF SOLID-STATE CIRCUITS, VOL. 41…imagesensors.org/Awards/Walter Kosonocky...

2998 IEEE JOURNAL OF SOLID-STATE CIRCUITS, VOL. 41, NO. 12, DECEMBER 2006

A 1/1.8-inch 6.4 MPixel 60 frames/s CMOS ImageSensor With Seamless Mode Change

Satoshi Yoshihara, Yoshikazu Nitta, Masaru Kikuchi, Ken Koseki, Yoshiharu Ito, Yoshiaki Inada,Souichiro Kuramochi, Hayato Wakabayashi, Masafumi Okano, Hiromi Kuriyama, Junichi Inutsuka,

Akari Tajima, Tadashi Nakajima, Yoshiharu Kudoh, Fumihiko Koga, Yasuo Kasagi, Shinya Watanabe, andTetsuo Nomoto, Member, IEEE

Abstract—A 1/1.8-inch 6.4 MPixel 60 frames/s CMOS imagesensor fabricated in a 0.18- m single-poly triple-metal (1P3M)process is described. A zigzag-shaped 1.75 T/pixel architecture anda 10-bit counter-type column parallel ADC enables 2.5 2.5 m2

pixels. The resulting pixel has 38% fill factor and 12ke-/lux.ssensitibity. In addition, full frame and 2 2 binning modes areinterchangeable without an extra invalid frame.

Index Terms—A/D converter, CMOS image sensor, dual CDS.

I. INTRODUCTION

COMPACT digital cameras now require a high pixel count,high imaging performance, and low power consumption.

Pixel size miniaturization is necessary to achieve a high pixelcount in an adequate optical format. The trend graph for theframe rate is shown in Fig. 1. Although the number of pixelshas been increased, image sensors have been able to read out allof the pixels at only a few frames per second.

The advantages of a CMOS image sensor are low power andeasy system integration with on-chip circuits. Among CMOSimage sensors, transistor-sharing techniques are widely usedto make small pixels have better imaging performance [1]–[3].High-speed CMOS image sensors with on-chip ADC have beendeveloped [4]–[6], and digital double-sampling architectureis proposed to remove device variation and circuit offset thatcauses vertical fixed pattern noise (FPN) [5], [6]. Using theseadvantages, the readout speed can be made dramatically fasterthan that of CCDs. High speed and high pixel count will greatlyexpand the possibilities of the coming new digital cameraworld.

To realize high-speed imaging, we have developed a6.4 MPixel, 60 frames/s CMOS image sensor. There are threepoints needed to enable a 6.4 MPixel 60 frames/s sensor:10/12 bit column-parallel analog-to-digital (A/D) converters,a zigzag-shaped four pixel sharing technique, and a 12-bitparallel LVDS interface.

Manuscript received June 7, 2006; revised August 3, 2006.S. Yoshihara, Y. Nitta, M. Kikucchi, Y. Ito, Y. Inada, S. Kuramochi,

H. Wakabayashi, M. Okana, Y. Kudoh, F. Koga, and T. Nomoto are with SonyCorporation, Kanagawa 243-0014, Japan.

K. Koseki, H. Kuriyama, J. Inutsuka, and A. Tajima are with Sony LSI DesignInc., Kanagawa 243-0014, Japan.

Y. Kasagi and S. Watanabe are with Sony Semiconductor Kyushu Corpora-tion, Nagasaki 854-0065, Japan.

Digital Object Identifier 10.1109/JSSC.2006.884868

Fig. 1. Trend graph for frame rate.

Fig. 2. Block diagram of the sensor.

II. BASIC DEVICE ARCHITECTURE AND

COLUMN-PARALLEL A/D CONVERTER

The block diagram of the sensor is shown in Fig. 2. The sensorconsists of a pixel block, column parallel counter-type A/D con-verters, control logic and peripheral circuits. The column A/Dconverters are composed of comparators, counters, and latches.Peripheral circuits include a phase-locked loop, generating a216-MHz counter clock from a 54-MHz input clock. A 10/12bit parallel LVDS interface circuit was chosen, enabling datarates up to 432 MHz.

Column parallel A/D converters allow for low bandwidthreadouts which enables image sensor to realize low noisecharacteristics. This is a key advantage over wide bandwidthsingle output amplifiers in CCDs. In order to realize these highframe rates, CMOS image sensors need to achieve both a highpixel throughput and high image quality.

0018-9200/$20.00 © 2006 IEEE

YOSHIHARA et al.: A 1/1.8-inch 6.4 MPIXEL 60 frames/s CMOS IMAGE SENSOR WITH SEAMLESS MODE CHANGE 2999

Fig. 3. Column-parallel A/D converter. (a) Conventional A/D. (b) Column-inline dual CDS.

A conventional column-parallel A/D converter is shown inFig. 3(a). The single-slope A/D converter is composed mainly ofcomparators, data latches, a ramp generator and a synchronouscounter. Single-slope A/D converters have the advantage of alow bandwidth readout, which allows for low noise characteris-tics, and of high accuracy in the A/D conversion.

On the other hand, single-slope A/D converters have adisadvantage in high-speed imaging, because they require along time for the A/D conversion. A synchronous counter isused for digital sampling. Column-to-column variations ofclock skew, which cause conversion error, are generated whena high-speed clock is used. Additionally, analog correlateddouble sampling circuits (CDS) need a large capacitor to keepnoise at a minimum.

The key development aspects of this architecture are shownin Fig. 3(b). A high-speed clock is utilized to reduce the A/Dconversion time. Digital CDS, which performs digital doublesampling and subtraction with column-inline counters, has beenintroduced.

The schematic diagram for a column A/D converter is shownin Fig. 4. Pixel operation requires three control signals, , ,and controlled by row decoders. The column comparatorsare driven by a ramp generator (DAC) and the pixel output

by connecting series capacitor . The column counters,composed of ripple counters, perform the A/D conversion bycounting the number of clocks until the comparatoroutput changes. Ripple counters have the advantage of notneeding to be synchronized with the high-speed clock .Digital CDS is obtained by changing up/down counting of theripple counters using the clock selectors. The timing chart forthe pixels and the comparators is shown in Fig. 5. The analogCDS sequence is as follows. First, the reset signal resets thepixels causing the reset level of the sensor output to appear atthe pixel output (controlled by ) [A1]. After that, theinput and output of the comparators are connected through thetransistor Tcr (controlled by ) [A2]. This eliminates theoffset of the comparators and the pixel outputs which causesFPN when the control signal is turned on. There is still somedeviation remaining in the comparator input voltagewhich corresponds to the [A3]. The signal level appearswhen the control signal opens the transfer gate (controlled by

Fig. 4. Schematic diagram for a column A/D converter.

) [A4]. The comparator output is turned on again in the samemanner as the reset level [A5].

The column ripple counters perform the A/D conversion bycounting the number of digital clock cycles. The counters stopwhen the clock latch negates the PLL clock with the comparatoroutput. The ripple counters are set to down count period duringreset readout [D1]. By changing the select signal, the ripplecounters are set to up counting during the signal readout [D2].The counters digitally subtract the conversion of the reset signalfrom the sensor signal. By using this dual CDS, the analogsensor signal is converted to the corrected digital output signal

in the individual columns in parallel. When the dualCDS is finished, the digital data is transferred to the data latchesincluded in each counter block. This pipelines the A/D conver-sion of the nth row and horizontal data transfer of the throw.

The advantage of this dual CDS architecture is a high noisesuppression capability because FPN cancellation is performedtwice, once in the analog domain and once in the digital domain.An analog CDS is used to eliminate the analog offset of the pixeland comparator [A2] and to reduce the A/D conversion periodfor the reset signal [A3]. Additionally, a high-speed, 216 MHzclock is utilized to reduce the digital double sampling period[A3, A5]. Ripple counters are advantageous in this applicationbecause it is unnecessary for them to be synchronized with thehigh-speed clock. Column-to-column variations of clock skewand counter delay which cause A/D conversion error are cor-rected by digital CDS [D1,D2].

III. TRANSISTOR-SHARING TECHNIQUE

Transistor-sharing techniques are widely used to makesmaller pixels and to get high-imaging performance. The pixelarrangement of a zigzag-shaped four-pixel sharing techniqueis shown in Fig. 6. Bayer-pattern primary color filters andmicrolenses are fabricated on-chip. The layout design and thecircuit diagram are shown in Fig. 7. The resulting transistor


Fig. 5. Timing chart from the pixels to the comparators.

Fig. 6. Pixel arrangement of a zigzag-shaped four-pixel sharing technique.

count is 1.75 transistors/pixel. The upper two pixels share thefirst floating diffusion and the lower two pixels share the secondfloating diffusion. The first and second floating diffusion areasare connected together by wiring. The shared floating diffusionnode is also connected to the reset transistor and the amplifiertransistor. In this configuration, the same color Gr and Gb cor-respond to a single column circuit. So the deviation of verticalneighboring Gr and Gb is negligible.

IV. TIMING SEQUENCE

This sensor has multiple modes of operation (Table I). Thetiming sequences of this sensor are as follows. The horizontalreadout sequence at 6.4 MPixel and 60 frames/s is shown inFig. 8. To obtain the maximum resolution at 60 frames/s, theresolution of VRAMP is set to 10 bits. After the reset opera-tion, the reset level of the pixel is A/D converted. At that time,the A/D converter ramp signal is suppressed to 256 counts be-cause the reset voltage is much smaller than the signal voltage.During A/D conversion of the signal voltage, the full 1024 rangeis used. Further, both the readout of the previously selected row

Fig. 7. Layout design and the circuit diagram.

TABLE IDESIGN SPECIFICATIONS

and the A/D conversions of the currently selected row are per-formed simultaneously. As a result, the horizontal scanning timeis 7.2 s.


Fig. 8. Horizontal readout sequence at 6.4 MPixels and 60 frames/s.

Fig. 9. Vertical accessing at 6.4 MPixels and 60 frame/s.

The vertical timing for 6.4 MPixel 60 frames/s operation isshown in Fig. 9. First, the transfer pulse and reset pulse areopened at the same time. Charge accumulation starts when theyare closed. Then, after the V sync pulse is received, the transferpulse is open again and the charge of the pixel is read out to thevertical source follower output. There are 2310 vertical lines,each read out at 7.2 s, therefore all 6.4 MPixels can be readout within 1/60 of a second.

This sensor can also be operated in high resolution mode. Byincreasing the number of bits during A/D conversion, an outputresolution of 12 bits is achieved. The horizontal scanning timeis extended to 28.8 s and all pixels can be read out within 1/15of a second.

The vertical summation technique is shown in Fig. 10. Ver-tical rows are summed digitally using this column A/D conver-sion technique. Each of the pixels is converted independently.Digital CDS is also accomplished the same way (Section II).The first row of pixels is converted in a similar way as the6.4 MPixel 60 frames/s mode, i.e., down count of the reset leveland up count of the signal level. The counter code is main-tained without resetting the first row’s pixel data. After that,the second row’s pixel data is converted and the counter codesums that data. To prevent the counter code from overflowing,the counter has a 13-bit depth. Consequently, the two signalscan be summed accurately.

Fig. 10. Vertical summation technique.

Fig. 11. Readout sequence of the 2�2 binning mode.

The readout sequence of the 2 2 binning mode is shown inFig. 11. For the summation of the same color pixels of ver-tical rows, these are selected one after the other. Vertical signalsare summed at the column counters, and horizontal signals ofthe pixels are digitally summed in the output circuit. Further-more, this sensor also has a draft mode, in which 1/5 of verticalrows are read out. This draft mode results in a frame rate of300 frames/s.

V. SEAMLESS MODE CHANGE

In conventional CMOS image sensors, it is necessary to insertan invalid frame to obtain a certain integration time for variousreadout modes. An example of a conventional sensor’s modechange is shown in Fig. 12. When the mode is switched from2 2 binning to 6.4 MPixel mode, a mismatch of the integrationtime occurs. The readout sequence is switched to 6.4 MPixel,but shutter operation of the previous mode was 2 2 binning.This is because the integration time of adjacent color rows isdifferent between two modes, and then, the current frame of6.4 MPixel becomes an invalid frame.

The column parallel A/D conversion architecture of thissensor is suitable for the seamless mode change. There isno difference in the A/D conversion without switching theorder of vertical rows. So, whether the signals are summedor not, the data of individual pixels are the same. Moreover,there are three key points to realize seamless mode change. Tochange the order of vertical rows at the shutter timing just afterreadout timing, shutter pulses and readout pulses are controlledindependently, and the two modes can be switched by setting


TABLE IIMEASURED DATA OF THE PIXEL

Fig. 12. Example of a conventional sensor’s mode change.

Fig. 13. Example of seamless sequence from 2�2 binning to 6.4 M.

the serial communication one frame prior to the actual modechange.

An example of seamless sequence from 2 2 binning to6.4 M is shown in Fig. 13. The serial communication of 6.4 Mis set during a previous frame, but the readout sequence con-tinues in the 2 2 binning mode. While the readout operation of2 2 binning continues, the shutter operation for the next frameis switched to the sequence of 6.4 M. As a result, a seamlessmode change can be done without generating a mismatchbetween each row’s integration times. Therefore, there is noinvalid frame between 2 2 binning and 6.4 M. The reversemode transition, from 6.4 M to 2 2 binning, is done the sameway.

VI. MEASUREMENT RESULTS

The design specifications are shown in Table I. The sensoris fabricated using a 0.18- m, single-poly triple-metal process.

Fig. 14. Measured nonlinearity at 60 frames/s.

Pixel size is 2.5 2.5 m . Zigzag-shaped 1.75 transistors/pixelarchitecture enables 2.5 2.5 m pixels with high saturationand sensitivity. The input clock rate is 54 MHz, and the max-imum pixel rate is 432 MHz.

By using this sharing technique, high performance imagingis achieved. The measured pixel data is shown in Table II. Theconversion gain is measured at 40uV/e-. The measured quantumefficiency of the green pixel is 48% at a wavelength of 550 nm.The sensitivity of the green pixels is 14 000e-/lux.s. A saturationsignal of 12 000e- is achieved without image lag. These datainclude the effect of a color filter and an on-chip microlens.The aperture ratio, i.e., fill factor, is 38% without an on-chipmicrolens.

The measured nonlinearity value at 60 frames/s is shown inFig. 14. It is defined as the ratio of the deviation from linearapproximation line to saturation signal. The horizontal and ver-tical axes show the integration time and the digital output whichis normalized with saturation signal, respectively. The linearityof the A/D converter depends on the characteristics of the sourcefollower, comparator and slope-D/A converter. Measured non-linearity at 60 frames/s is 0.3% at 90% of saturation.

The measured fixed pattern noises are shown in Fig. 15. It wastaken with an analog gain of 18 dB and a digital gain of 30 dB.There is significant column-to-column FPN after analog CDSis finished. And after the dual CDS is finished, column FPN


Fig. 15. Measured fixed pattern noises.

TABLE IIIMEASUREMENT RESULTS

Fig. 16. Measured random noise.

is decreased by 15.7 times. Column FPN is negligibly smallbecause digital CDS eliminates offset precisely.

Measurement results are shown in Table III. Measured ver-tical FPN is 0.38e- rms. Vertical FPN is much smaller thanrandom noise at dark. So an image can be obtained without anyvertical noise, even if the gain is raised. Measured random noiseis approximately 7e- rms, as shown in Fig. 16. The random noisecomponents from pixel and circuit are comparable. The outputsignal in 2 2 binning mode is divided by four in the on-chipoutput circuit in order to equalize the signal level. So, mea-sured random noise during 2 2 binning is exactly half of thatat 6.4 MPixel at 10 bits. Random noise of the 1/5 intermittentmode is the same as 6.4 MPixel at 10 bits.

Fig. 17. Total power consumption of all modes.

Fig. 18. Example of a reproduced 6.4 MPixel image at 60 frames/s.

Total power consumption of each mode is approximately360 mW, as shown in Fig. 17, because the column operationsof all modes are almost the same.

An example of a reproduced 6.4 MPixel image at 60 frames/sis shown in Fig. 18. Although the image is captured at60 frames/s, the image quality is still high.

A comparison of image distortion at 60 frames/s and15 frames/s is shown in Fig. 19. An image distortion at60 frames/s is one-fourth of that at 15 frames/s.

An example of seamless mode change is shown in Fig. 20.One 6.4 MPixel image is actually obtained within continuous2 2 binning mode, and the upper right picture is that of6.4 MPixel 60 frames/s mode. Note the difference in resolutionbetween the two modes. The user can freely record movingpictures and still images without any limitations on the imagingperformance. A micrograph of the chip is shown in Fig. 21.

VII. CONCLUSION

Continuous 60 frames/s high-performance imaging has beendemonstrated. By dividing the readout sequence, seamlessimage capture between two modes is also available.

In conclusion, by using the techniques discussed in this paper,both video and still images can be captured from the same sensoroperated in full 6.4 MPixel resolution. New opportunities in thedigital imaging world are being sought for high-speed CMOS


Fig. 19. Comparison of image distortion at 60 frames/s and 15 frames/s. (a) Reproduced image of stationary subject. (b) Reproduced image of moving subject at15 frames/s. (c) Reproduced image of moving subject at 60 frames/s.

Fig. 20 Example of seamless mode change.

Fig. 21. Micrograph of the chip.

image sensors, and this work clearly shows one new applicationfor CMOS image sensors.

ACKNOWLEDGMENT

The authors acknowledge the contributions of T. Shoji,S. Sakane, K. Adachi, K. Kitagata, S. Kamogawa, Y. Yamagata,A. Nishimura, S. Ohki, M. Sato, K. Imamura, K. Masuda,M. Itoh, and K. Kaneko. The authors wish to thank H. Nomura,Y. Muramatsu, K. Amano, T. Toyama, J. Yamamoto,K. Mishina, A. Suzuki, T. Taura, A. Kato, and Y. Yasuifor encouragement in this study, and they also acknowledgeN. Fukushima and Y. Nakamoto for technical support.

REFERENCES

[1] H. Takahashi et al., “A 3.9 um pixel pitch VGA format 10b digitalimage sensor with 1.5-transistor/pixel,” in IEEE ISSCC Dig. Tech. Pa-pers, Feb. 2004, pp. 108–109.

[2] M. Mori et al., “A 1/4 in 2 MPixel CMOS image sensor with 1.75transistor/pixel,” in IEEE ISSCC Dig. Tech. Papers, Feb. 2004, pp.110–111.

[3] K. Mabuchi et al., “CMOS image sensor using a floating diffusiondriving buried photodiode,” in IEEE ISSCC Dig. Tech. Papers, Feb.2004, pp. 112–113.

[4] A. Krymski et al., “A high-speed, 240-frames/s, 4.1-MPixel CMOSsensor,” IEEE Trans. Electron Devices, vol. 50, pp. 130–135, Jan. 2003.

[5] W. Yang et al., “Integrated 800�600 CMOS imaging system,” in IEEEISSCC Dig. Tech. Papers, 1999, pp. 304–305.

[6] Y. Nitta et al., “High-speed digital double sampling with analog CDSon column parallel ADC architecture for low-noise active pixel sensor,”in IEEE ISSCC Dig. Tech. Papers, Feb. 2006, pp. 500–501.

[7] S. Yoshihara et al., “A 1/1.8-inch 6.4MPixel 60 frames/s CMOS ImageSensor with Seamless Mode Change,” in IEEE ISSCC Dig. Tech. Pa-pers, Feb. 2006, pp. 1984–1993.


Satoshi Yoshihara received the B.S. degree in elec-tronic engineering from Kyoto University, Kyoto,Japan, in 1991.

In 1991, he joined Sony Corporation, Japan. Hehas worked on the development of CCD image sensorand CMOS image sensor.

Yoshikazu Nitta was born in Yamanashi, Japan,on November 24, 1963. He received the B.S.and M.S. degrees in electronic engineering fromTohoku University, Sendai, Japan, in 1986 and 1988,respectively.

In 1988, he joined Mitsubishi Electric Corporation,Hyogo, Japan. In 2003, he joined Sony Corporation,Atsugi, Japan, where he is currently working on high-speed CMOS image sensors.

Masaru Kikuchi joined Sony Corporation, Japan, in2004. He has worked on the development of CMOSimage sensors.

Ken Koseki received the B.S. degree in physics fromNiigata University, Niigata, Japan, in 1998.

In 1998, he joined Sony LSI Design Inc., Japan.He has worked on the development of CMOS imagesensors.

Yoshiharu Ito received the B.S. degree in electronicengineering from Aoyama Gakuin University, Tokyo,Japan, in 1989.

In 1989, he joined Sony Corporation, Japan. Hehas worked on the development of CMOS analog anddigital circuit design and CMOS image sensors.

Yoshiaki Inada joined Sony Corporation, Japan, in2004. He has worked on the development of CMOSimage sensors.

Souichiro Kuramochi received the Master of Engi-neering degree in applied physics from the Universityof Tokyo, Japan, in 1996.

In 2003, he joined Sony Corporation, Japan. Hehas worked on the development of CMOS imagesensors.

Hayato Wakabayashi received the M.S. degree inengineering science from Osaka University, Osaka,Japan, in 1996.


Masafumi Okano received the M.S. degree in bio-engineering from Waseda University, Tokyo, Japan,in 2003.


Hiromi Kuriyama joined Sony LSI Design Inc.,Japan, in 1998. He has worked on the developmentof digital circuit for TV, audio and CMOS imagesensors.

Junichi Inutsuka joined Sony Nagasaki Corporation(now Sony Semiconductor Kyushu Corporation) in1992. He has worked on the development of CMOSA/D converter macro and analog macros. In 2006, hetransferred to Sony LSI Design Inc., where he is nowworking on CMOS image sensor design.

Akari Tajima received the B.S. degree from KyusyuInstitute of Technology, Fukuoka, Japan, in 1998.

She joined Sony Nagasaki Corporation (nowSony Semiconductor Kyushu Corporation) in 1998.She has worked on the development of CMOS A/Dconverter macro and analog macros. In 2006, shetransferred to Sony LSI Design Inc., Japan.


Tadashi Nakajima received the M.E. degree inindustrial chemistry from Seikei University, Tokyo,Japan, in 1997.

He joined Sony Corporation, Japan, in 2003, wherehe has been engaged in the development of semicon-ductor devices.

Yoshiharu Kudoh received the B.E. and M.E. de-grees of mechanical engineering from Tohoku Uni-versity, Sendai, Japan, in 1995 and 1997.

He joined Sony Corporation in 2003, where he hasbeen working in research on image sensors devicessuch as CMOS image sensors.

Fumihiko Koga received the M.E. degree in phys-ical electronics from Tokyo Institute of Technology,Tokyo, Japan, in 2002.

He joined Sony Corporation, Japan, in 2003, wherehe has been engaged in the development of semicon-ductor devices.

Yasuo Kasagi received the B.S. degree in electronicengineering from Tokyo University of Agricultureand Technology, Tokyo, Japan, in 1992.

In 2000, he joined Sony Semiconductor KyushuCorporation. He is presently engaged in developmentof process integration for CMOS image sensors.

Shinya Watanabe joined Sony Corporation, Japan,in 1990. He has worked on the development of CCDimage sensor and CMOS image sensors.

Tetsuo Nomoto (M’06) received the B.S. and M.S.degrees in applied physics from Tohoku University,Sendai, Japan, in 1988 and 1990, respectively.

In 1990, he joined Olympus Optical Corporation,Nagano, Japan, where he was involved in the devel-opment of charge modulation device image sensors.He joined Sony Corporation, Kanagawa, Japan, in2001, where he has been engaged in the developmentof CMOS active pixel sensors.

Mr. Nomoto is a member of the Physical Societyof Japan.

2998 IEEE JOURNAL OF SOLID-STATE CIRCUITS, VOL. 41…imagesensors.org/Awards/Walter Kosonocky...

Documents

Transcript of 2998 IEEE JOURNAL OF SOLID-STATE CIRCUITS, VOL. 41…imagesensors.org/Awards/Walter Kosonocky...