1

Applications of video-content analysis and retrieval

IEEE Multimedia Magazine 2002 JUL-SEP

Reporter: 林浩棟

2

Outline Introduction Content-based Video Retrieval Applications

Professional and educational applications Consumer domain application

Conclusion

3

Introduction

This is a survey of technologies and applications for video-content analysis and retrieval.

4

Content-based video retrieval (1/2)

Video indexing should be analogous to text document indexing

To facilitate fast and accurate content access to video data, we should segment a video document into shots and scenes

We should extract keyframes or key sequences as index entries for scenes or stories.

5

Content-based video retrieval (2/2)

The core research in content-based video retrieval is developing technologies to automatically parse video, audio, and text to identify meaningful composition structure and to extract and represent content attributes of any video sources.

6

Four processes involved by video-content analysis and indexing Feature extraction Structure analysis Abstraction Indexing

7

Four processes involved by video-content analysis and indexing

8

Four processes involved by video-content analysis and indexing Feature Extraction

The effectiveness of an indexing scheme depends on the effectiveness of attributes in content presentation

An effective strategy in video-content analysis is to use attributes extractable from multimedia sources.

Much valuable information is also carried in other media components, such as text, audio, and speech that accompany the pictorial component.

9

Four processes involved by video-content analysis and indexing

10

Four processes involved by video-content analysis and indexing Structure analysis

The process of extracting temporal structural information of video sequences or programs

Organizes video data according to their temporal structures and relations and thus build table of content

Stories -> scenes -> shots -> frames

11

Four processes involved by video-content analysis and indexing

Shots are a good choice as the basic unit for video-content indexing, and they provide the basis for constructing a video table of content.

12

Four processes involved by video-content analysis and indexing

13

Four processes involved by video-content analysis and indexing Video abstraction

Video abstraction is the process of creating a presentation of visual information about a landscape or the structure of video, which should be much shorter than the original video.

Example: baseball game Use an MPEG-7-compliant XML description

format for the event segment

14

MPEG-7 Providing a standardized description of

various multimedia descriptions of user preferences and usage

history pertaining to multimedia information.

15

Applications We can broadly classify users into two

extremes: Nontechnical consumer Trained, technical, professional corporate

users who regularly use the products Professional and educational applications Consumer domain application

16

Professional and educational applications Automated authoring of Web content Searching and browsing large video

archives Easy access to educational material Indexing and archiving multimedia

presentation

17

Professional and educational applications Automated authoring of Web content

Pictorial Transcripts AT&T DVL system

18

Professional and educational applications Searching and browsing large video

archives Major news agencies and TV broadcasters

own large archives of video that have been accumulated over many years.

Intelligent video segmentation and sampling techniques can reduce the visual contents of the video program to a small number of static images.

19

Professional and educational applications Easy access to educational material

The availability of large multimedia libraries that we can efficiently search has a strong impact on education.

20

Professional and educational applications Indexing and archiving multimedia

presentation example

21

Consumer domain application The widest audience for video-content

analysis is consumers Differences between large archives and

consumer domain Video overview and access Video content filtering

22

Consumer domain application Video overview and access

example

23

Consumer domain application Video content filtering

example

24

Conclusion It’s important to distinguish between research

activities, experiments, and real application that have made, or likely to make, the transition from research labs into the real world

The targeted users are the ultimate judges of the technology’s usefulness in meeting their need.

Users might have a tendency to resist new tools and methods.