― 21 ―

PhotoVision: A Take-Anywhere Digital Photo Album

PhotoVision: A Take-Anywhere Digital Photo Album

携帯型ディジタル写真アルバム

George Rome Borden IV*

* Sharp Laboratries of America, Inc.

Abstract

Various methods have been used to archive, display and

browse digital images. Many of these utilize textual

information describing the image and require users to input

large amounts of data to be able to successfully find their

photos. We have developed an interface that allows users to

locate photos based on a calendar paradigm. Once an image

has been located, the user can easily create albums

manually or allow the software to assist in album creation.

The interface was developed as a prototype for a product,

PhotoVision. PhotoVision is a complete system for

browsing photos. It consists of a lightweight LCD tablet

with some form of persistent storage.

ディジタル画像の蓄積，表示，ブラウズにはさまざまな方法がある。しかし，そのほとんどでは，画像を説明するテキスト情報を利用しており，画像を探し出すためにユーザは大量のデータの入力を必要とするものだった。我々は，カレンダのパラダイムに基づき写真の場所を特定することができるインタフェースを開発した。写真の場所をひとたび特定したあとは，ユーザは，手動で，あるいはソフトウエアの助けを得て，アルバムを容易に作成することができる。このインタフェースはPhotoVisionという商品のプロトタイプとして作成された。PhotoVisionは，軽量のLCDタブレットと不揮発性記憶装置を組み込んだデバイスで，完全な写真ブラウジングシステムである。

Introduction

During the last few years digital cameras have become

increasingly popular. The ease of taking and storing digital

photos has left one problem unsolved. How do we

efficiently manage and view photos? Most cameras come

with some type of photo management software, but many

of these are inadequate and, to be useful, require the user to

input additional information about the image. There are

many interface designs for image retrieval in the works on

information retrieval. Some allow the user to specify

example images that are then used to search for images that

have been indexed by content features [1]. Another

approach is to allow the user to manipulate textual queries

[7]. In all of these cases the user is generally attempting to

retrieve data items that have no inherent relationship.

A unique aspect of a digital photo album is that the

photos all have some temporal relation to the user. As

examples, a user takes a large number of photos during a

trip to Europe, receives a photo of a child's graduation and

takes a photo of a new home. All of these events have some

temporal place in the user's life and it is possible for a user

to find these photos simply based on this criterion. This has

been recognized and utilized in [4]. Where this relation was

used to group photos into albums using a probabilistic

technique that attempted to place photos that were taken

during some event into the same album.

In this paper we propose an interface that takes

advantage of the temporal relation of a user's images to

facilitate searching for images, organizing images and

viewing images.

1. The PhotoVision System

The Sharp Copernicus shown in Fig. 1 is an example of

one possible form for the PhotoVision hardware.

PhotoVision is envisioned as a small tablet that allows one

to view and browse images in a portable manner similar to

that of a traditional photo album. It consists of a color touch

sensitive LCD panel, persistent storage, some means of

connecting to other devices such as personal computers

and the browsing software application. The software

modules shown in Fig. 3 exclude low-level drivers and OS

modules. They are the interface itself, the data persistence

manager, thumbnail manager, target manager and

automatic album generator.

― 22 ―

シャープ技 報 第77号・2000年8月

The interface shown in Fig. 2 consists of four main parts.

The calendar in the upper left corner allows a user to select

a date range to view. The thumbnails along the bottom

allow for quick searching through large numbers of photos.

The intermediate view in the upper right corner allows a

user to view an image at a slightly larger size than the

thumbnails, and a full-screen view (not shown) allows the

user to view an image using the tablet's entire screen space.

The different sized image views serve specific functions for

the user.

Data persistence is maintained by assuming that the

filename of all files is unique. The data persistence

manager is set to search user defined directories for new

files. When a new file is found, it is placed into the database

[Fig. 4] along with its path and a thumbnail is generated

and placed into a thumbnail directory for fast access. If at

anytime the file is removed from the directory, the

persistence manager will detect that the file is missing. If at

any time in the future the file is placed into one of the

search directories, the persistence manager will find it, note

its new location, and the metadata that has been associated

with that image will not be lost. As long as all of the file

names are unique, this scheme works fine. When file

names are not unique, one would guess that it would be

possible to use complete path names, which are guaranteed

to be unique in some file systems. This may work, but fails

Fig. 1 Sharp Copernicus touch panel PC.

Fig. 2 PhotoVision interface.

― 23 ―

PhotoVision: A Take-Anywhere Digital Photo Album

case where a file is removed and then later returned. In this

case it is not possible to determine which file has returned

to the database. A better method is to attach the metadata

information to the file itself as mentioned in section 5.

The thumbnail manager loads all of the thumbnails in the

currently selected date range. It loads those that must be

displayed immediately in parallel. It then begins the

background task of loading one image at a time of those

that are in the selection but not currently displayed. This

allows a user to quickly see all of the images that must be

displayed and to begin interacting with the interface as

soon as possible. The background loading of the images

allows the user to scroll the thumbnails quickly.

The target manager keeps track of current sources and

targets for album generation, full-screen viewing, slide

show viewing and other operations.

The automatic album generator records the total amount

of time that each image has been viewed in the full-screen

mode. During full-screen view mode the amount of time

that the image has been viewed is repeatedly compared to

one standard deviation above the mean of all of the photo's

viewing time. When the current photo's viewing time is

larger, the user is signaled that the current photo is a

candidate for addition to the default preferred photos

album.

2. Search, Differentiate and View

There are three sizes at which a user may view images,

thumbnails, intermediate and full-screen. These three sizes

correspond to three different uses; search, differentiate and

view respectively. When a user is searching for a particular

photo, they are almost certainly looking at and browsing

through the thumbnails displayed at the bottom of the

screen [Fig. 2]. These thumbnails however do not have a

high enough resolution to allow a user to enjoy the photo

and in cases where two photos are similar the thumbnails

may not even allow a user to differentiate between

them[1][4].

In order for a user to differentiate between two similar

photos they will need to utilize the intermediate view in the

upper right corner. This view has enough resolution to

select from two different photos, but not enough to enjoy

the image being displayed.

It is anticipated that user's who wish to view an image in

a satisfying manner will view the image with the full-

screen view. This view fills the screen with the image and

displays some small icons in the upper left-hand corner that

disappear after a couple of seconds. Running the mouse

over them makes them reappear. Using these icons the user

can move to the next or previous photo and return to the

browsing interface.

3. Calendar View

The calendar view in the upper left corner of Fig. 2

allows a user to select a range of dates from which photos

will be viewed. Once a selection is made, the photos that

Fig. 3 PhotoVision architecture.

Fig. 4 Photo viosion database example.

― 24 ―

シャープ技 報 第77号・2000年8月

have creation dates within the range are displayed as

thumbnails. At the bottom of the calendar view is the

selection area scroll bar. The user can scroll the calendar by

either dragging the calendar directly or dragging the

selection area scroll bar too far in either direction. The

selection area or date range can be increased or decreased

by dragging either of the tabs on each end of the selection

area scroll bar.

The calendar can also be dynamic zoomed similar to a

fisheye view [3] to change the date resolution. This is done

by dragging either of the tabs on the selection area scroll

bar to the edge of the calendar or to the center of the

calendar. Dragging to the edge of the calendar causes the

magnification to decrease and dragging to the center causes

the magnification to increase. While interesting to use,

empirical data gathered suggested that what users desired

was a quick way to move between the two most useful

resolutions, month and year. As a result, the month and year

buttons were added.

4. Creating Albums

The functions for viewing albums, creating new albums

and deleting albums are all contained in the album roll-

down menu shown in the center of Fig. 2, and there are two

methods of adding photos to albums; manual and

automatic.

To manually add photos to an album the user selects the

"add photos" item from the album menu. The intermediate

view then displays a list of all of the existing albums. The

user then selects the albums into which to add the currently

selected thumbnails and hits the plus button. Through this

method multiple photos can be added to multiple albums

easily. Dragging thumbnails and dropping them onto

albums is also supported.

The automatic method of adding photos to albums

occurs only when a photo is being viewed in the full-screen

view. Because the system restricts the user to viewing one

photo at a time, it is possible for the system to determine

exactly which photo is being viewed. This allows us to

record the amount of time that a photo has been viewed in a

history and use this information to determine which photos

are most important to the user. When a photo is viewed

enough to surpass one standard deviation above the mean,

the user is signaled and can at that time add the photo to a

default preferred photos album. Later, the user can place

the photo into some other album of their choice. We do not

automatically add the photo to the preferred photos album,

because this could result in some photos being added to the

list unexpectedly and others being deleted. As this is not

desirable [6], we give the user control over which photos

they would like to add. This way a user who has gone

through a complicated browsing based search several times

to locate a particular photo, will be able to quickly find it in

an album the next time.

5. Current and Future Work

During the development of this prototype we have been

involved in several standards bodies dealing with the

metadata described here. During the course of this work the

JPEG group has adopted several mechanisms into the final

committee draft of the JPEG2000 standard that allow the

attachment of metadata directly to the file in a semi-

interoperable manner. It is expected that further refinement

of the standard will offer complete interoperability.

The ability to attach metadata to an image in a persistent

manner will open the door for many new ways of assisting

a user to successfully find and organize images. We hope to

continue research in this area through continued

development of intelligent user interfaces.

Conclusion

We have presented a novel interface for locating images,

viewing images and organizing images with minimal text

input. This interface takes advantage of the temporal

relation that a user has with the images that they own.

Having recognized the usefulness of this relation, there is a

natural fit with the calendar interface. This calendar

paradigm presents users with a familiar model that is easy

to understand and manipulate. We have also introduced a

new and unique tool, the selection area scroll bar, for

manipulating selection ranges and altering resolutions of a

time line and other virtual areas. We believe that the

approaches we have taken offer significant improvements

to existing methods of browsing digital images.

AcknowledgementsJon Clemens, Jeff Sampsell, Ibrahim Sezan and Urano

Naoki have been invaluable in shaping the course of this

work.

References[1] C. A. Burton, L. J. Johnston, and E. A. Sonenberg, An Empirical

Investigation of Thumbnail Image Recognition. In IEEE

― 25 ―

PhotoVision: A Take-Anywhere Digital Photo Album

InfoVis'95 Proceedings, pages 115-121（October 1995）.

[2] C. Faloutosos, R. Barber, M. Flickner, J. Hafner, W. Niblack, D.

Petrovic, and W. Equitz, Efficient and Effective Querying by Im-

age Content, Journal of Intelligent Information Systems, vol 3, pp

231-262（1994）.

[3] G. W. Furnas, Generalized fisheye views, In Proceedings of the

CHI'86 Conference on Human Factors in Computing Systems

and Graphic Interfaces (Boston), Addison-Wesley, pp. 16-23,

（May 1986）.

[4] P. Jacob, A. Mickelson, J. Allen, The Optimal Thumbnail.

SHORE project, University of Maryland, http://

www.otal.umd.edu/SHORE/bs21/ （current 2000） .

[5] C. J. Platt, AutoAlbum: Clustering Digital Photographs using

Probabilistic Model Merging. In proceedings of the IEEE Work-

shop on Content-Based Access of Image and Video Libraries

(CBAIVL2000), IEEE Computer Society, pp. 96-100 （June

2000）.

[6] B. Shneiderman, Direct Manipulation for Comprehensible, Pre-

dictable and Controllable User Interfaces, In proceedings of the

1997 international conference on Intelligent User Interfaces

(IUI'97) , pp. 33-39（1997）.

[7] D. Young, B. Shneiderman, A Graphical Filter/Flow Represen-

tation of Boolean Queries: A Prototype Implementation and

evalaluation. JASIS, vol44, #6, pg 327-339（1992）.

(received June 30, 2000)