Semantic Scenes Detection and Classification

in Sports Videos

Soo-Chang Pei ( 貝蘇章 ) and Fan Chen ( 陳 凡 )

Conference on Computer Vision, Graphics and Image Processing (CVGIP 2003)

指導老師：吳宗憲 教授 組員 p76974050 王志宏 p76974238 趙郁婷 p76974555 蔡佩珊

1

Outline

× Introduction× Method

■ Scene change detection■ Semantic scenes detection in baseball

game videos

× Experimental Result× Discussion

2

Introduction

× Motivation

3

Introduction (Cont.)

4

Infield

Outfield

Player

Pitching

？ Pitching scene

Introduction (Cont.)

× Index and retrieval the baseball video× Semantic scene detection methods

■ Combining with domain-specific knowledge■ Index keyframes by low-level features

5

Skeleton outline of our semantic scenes detection

Introduction (Cont.)

× Video content analysis methods may be classified into the following three categories:■ Syntactic structurization of video■ Video classification■ Extraction of semantics

× Don’t use video object tracking in object level verification in order to reduce the complexity

× Use object-location to verify which view type the keyframe belongs to

6

Method

× Scene change detection

× Semantic scenes detection in baseball game

videos

7

Scene change detection

× Execute the action of scene change detection and key frame extraction

■ IBM VideoAnnEx Annotation Tool

•annotate video sequences with MPEG-7 metadata

8

Semantic scenes detection

× Analysis and structuring in baseball■ A play usually starts with a pitching scene■ After the play starts, if after a scene change the

camera is shooting the field, then the current play should continue; otherwise, the current play ends when switched to a player scene

9The model of baseball broadcasting videos

the play continuousthe play is end

Semantic scenes detection (Cont.)

× Block diagram

10

Semantic scenes detection (Cont.)

× Field color percentageField color percentage■ Detected field color distribution and percentage

■ Three situations• Medium(20%~45%) ： pitching scene• Large (>=45%) ： outfield scene or an infield scene• Small (<20%) ： close-up scene or others

11

grass color range ： 0.19<H<0.46 、 0.2<S<0.7 、 V>100soil color range ： 0.06<H<0.15 、 0.25<S<0.8 、 V>100

grass color range ：34.2<H<82.8 、 51<S<178.5 、 V>50soil color range ： 0<H<22 、 63.75<S<204 、 V>100

Semantic scenes detection (Cont.)

× Block diagram --- Field color percentage

12

20~45%>45%

<20%

Semantic scenes detection (Cont.)

× Pitching scene verificationPitching scene verification■ First build a binary image by assigning field color to

1-pixel and non-field color to 0-pixel

13

(a) original pitching scene (b) the result of binarizing(c) histogram of horizontal projection

T = 0.15

Horizontal projection histogram

Semantic scenes detection (Cont.)

× Pitching scene verificationPitching scene verification

14

(a) original pitching scene (b) the result of binarizing(d) histogram of vertical projection

Vertical projection histogram

m = 15M = 100

× Block diagram --- Pitching scene verification

15

Semantic scenes detection (Cont.)

Semantic scenes detection (Cont.)(Cont.)

× Close-up scene detectionClose-up scene detection■ A close-up scene always target on one’s

face • Face detection is a key point

■ Skin color can be segmented out of an image

• Hues are between 0 and 50 degrees and saturation between 0.23 and 0.68

16

Semantic scenes detection (Cont.)

× Close-up scene detectionClose-up scene detection■ Two step

• Step 1 ： Label skin color to 1-pixel, or label to 0-pixel

• Step 2 ： Find the largest region in our defined “red-block”

17

depicts the skin color distribution

depicts the region considered as face region

Semantic scenes detection Semantic scenes detection (Cont.)(Cont.)

× Block diagram --- Close-up scene detection

18

Semantic scenes detection (Cont.)(Cont.)

× Player scene detectionPlayer scene detection■ Player scene ： Lead role is a figure but

background is composed of lots of field components

19

Can be taken as a scene between infield or outfieldand close-up scene

Semantic scenes detection (Cont.)(Cont.)

× Player scene detectionPlayer scene detection■ Field color percentage is large and there are

some big concaves in the vertical projection diagram

20

K

Semantic scenes detection (Cont.)

× Block diagram --- Player scene detection

21

Semantic scenes detection (Cont.)(Cont.)

× Infield and outfield scene detectionInfield and outfield scene detection■ Calculate the ratio of grass to soil

Infield scene Outfield scene 22

GrassRatio

Soil

>5≦ 5

Experimental Result

× Data : MPEG-1× Frame size : 360*240× Frame rate : 30Hz

23

Experimental Result (Cont.)

× Result

24

Original Binarizing

Infield_scene

Experimental Result (Cont.)

× Result

25

Original Binarizing

Outfield_scene

Discussion

× pitching 畫面會因為土和草的比率而偵測有誤× 當投手站於右方時，可能造成錯誤

26

27