Criminal Court Case Facial Comparison and Facial...
Transcript of Criminal Court Case Facial Comparison and Facial...
Criminal Court Case Facial Comparison and Facial Recognition Limits
2011© AFIS and Biometrics Consulting Inc
Ben Bavarian,
Mehrad Tavakoli
Principal Consultants
AFIS and Biometrics Consulting Inc.
96th IAI Educational Conference
Milwaukee, Wisconsin
August 10, 2011
Workshop Agenda
2
The Case Background
The data
The defense expert testimony
The FBI expert counter arguments
Open discussion
The Case
3
THE SUPERIOR COURT OF THE STATE OF CALIFORNIA
IN AND FOR THE CITY AND COUNTY OF SAN FRANCISCO
HONORABLE JEROME T. BENSON, JUDGE
DEPARTMENT NO. 21, SCN 210246 COURT NO. 2429070
PEOPLE OF THE STATE OF CALIFORNIA,(PLAINTIFF, )
VS.
CHARLES HEARD, (DEFENDANT. )
APPEARANCES OF COUNSEL:
J. MICHAEL SWART, ASSISTANT DISTRICT ATTORNEY
FOR THE DEFENDANT: ERIC M. SAFIRE, ATTORNEY AT LAW
LAW OFFICES OF ERIC M. SAFIRE
June 2010
The Assignment
4
Task Description
The objective is to carry out an independent scientific examination of
landmark features that are used in technical matching techniques for
face recognition biometric to compare the recently captured photos of
Mr. Charles Heard with the unknown person (the one with the black
hoodie), in the video from the security camera on Montgomery Street
showing two figures, Afro-American males, appear at 56:18:04, or
perhaps a few seconds sooner.
Conclusions
Based on the analysis detailed out in this report the comparison is
inconclusive. In other words it is not possible to assess that the figure in
the said video is that of Mr. Charles Head based on comparing the
biometric landmark features.
Methods for the one-to-one facial image comparison Facial Identification Scientific Working Group (FISWG)
5
Holistic comparison – the process of comparing faces by looking at the
face as a whole and not component parts;
Morphological analysis – applying the classification and description of the
form and structure of facial features;
Photo-anthropometric – the measurement of dimensions and angles of the
anthropologic landmarks in order to quantify characteristics and proportions
of a person in an image;
Superimposition – the process of creating a scaled overlay of one image
and aligning it with a second image.
The biometrics identification community has added the 5th method for the
comparison:
Biometric Facial Recognition – applies mathematical algorithms which
use principal abstract components of the digital image and calculates a
degree of likelihood or a probability match score between two images.
Component-based Face Matching; Facial Image Component
decomposition and Analysis (PCA, ICA).
Feature-based Face Recognition; 2D landmarks can be defined
and tracked on face images
A note on Features used in Face recognition
6
Biometric Data: 2D Facial Images : Challenges:
Sensitive to changes in Light & Make-Up Sensitive to Angle of Capture Inter-class and intra-class variability
7
www.marykateandashley.com news.bbc.co.uk/hi/english/in_depth/americ
as/2000/us_elections
Identical twins Almost identical father and son
Pose and make-up variations
Professor Massimo Testarelli, Biosecure Industry Committee Workshop, Feb. 2006, Computer Vision Laboratory University of Sassari – Ital
MONTGONERY – 56:16:74
The Data
8
MONTGOMERY – 56:17:83
The Data
9
MONTGONERY – 56:18:04
The Data
10
MONTGONERY – 56:18:26
The Data
11
MONTGONERY – 56:18:48
The Data
12
MONTGONERY – 56:18:70
The Data
13
The Data
14
15
The Data ANSI/NIST-ITL Standards Data Format for Interchange of Fingerprint , Facial & other Biometric Information Section 15.1.27 Field 10.029: Facial feature points. MPEG4 Standard; Annex C of ISO/IEC 14496-2
Methodology used for the Analysis
16
The following four steps will be used in our
scientific analysis study:
Step1: Data Gathering and Preparation
Step 2: Feature point measurements and tabulations
Step 3: Face feature calculations from facial feature points and
data normalization
Step 4: Data Analysis
The science applied Probability of complementary event
17
Note:
Our method is to answer the question of “non-matching”, “dissimilarity” between
two samples or solving the problem of “complementary event” in probability
theory.
The complementary probability, determining “non-match”, problem that we have
solved here is order of magnitude easier and straight forward to do than the
facial recognition finding the “match” problem.
A simple explanation.
Suppose one throws an ordinary six-sided die. The probability of number “2”, or
any number, coming out is (1/6)th. The probability of any other number coming
out is (5/6)th. That is five times more than for a single event. That is called the
complementary event or complementary probability of an event. In layman
terms it is much easier to get the complementary event that the event itself. Or
you will be better off betting on the complementary event with higher chances of
winning.
Biometric Analysis Approach
Step 1
Data Gathering and Preparation
Step 2
Feature point measurements and
tabulations Step 3
Face feature calculations from facial
feature points and data normalization
Step 4
Data Analysis ≠ ? 18
Biometric Analysis Approach Reference Photos Security Video Captures
MONTGOMERY – 56:16:74
MONTGOMERY – 56:17:83
MONTGOMERY – 56:18:04
Biometric Analysis Approach
20
Biometric Analysis Approach
Reference Photos
Security Video Capture
21
Biometric Analysis Approach
L4
Measurement
L0
Measurement
L3
Measurement
L1
Measurement
L2
Measurement
22
L3
L0
Biometric Analysis Approach - Normalization
Video Footage
1
2 .5
L3
L0
L3
L0
Reference Photo
10
20 .5
L3
L0
23
L3
L0
Video Footage
13
37 .35
L3
L0
Reference Photo
282
914 .31
L3
L0
L3
L0
Biometric Analysis Approach - Normalization
24
Video
Face is
Wider
Biometric Analysis Approach
L1-
forehead-to-nose length
L2-
nose-to-chin
length
L3-right eye-to-
left-eye length
L4-
Mouth
length
target jail photos Average Length 0.54 0.48 0.32 0.27
Standard Deviation 0.03 0.03 0.02 0.01
the video scene
pictures
Average Length 0.52 0.48 0.36 0.30
Standard Deviation 0.03 0.03 0.05 0.02
Video
Head is
Shorter
Video
Face is
Wider
Table 12- The final set of feature lengths and the
statistical variations for both data sets
25
26
The FBI expert counter arguments
The resolution of the video images are not sufficient for facial
recognition algorithms
The data elements are not marked correctly on the sample data
The lighting and subject position in the video lend itself to
photographic issues such as perspective distortions, angle of
pose issues, etc.
Statistical error analysis are missing in the expert witness work.
The defendant expert is not an “expert” in photography and
Photo-anthropometric area.
The data points used are not reliable points.
Open discussion
What is the scientific basis of facial recognition?
The features used;
The statistical characterization and uniqueness;
Reliability; permanence; and uniqueness characterization of the features;
Applications of facial recognition in law enforcement;
Evolutionary process for adoptation
27
Dr. Bavarian is one of the pioneers and industry leading authority in the field of Biometric
Identification with over 26 years of R&D and management experience in industry and academics.
He founded the AFIS and Biometrics Consulting Inc. in 2007, providing subject matter expert
consulting and strategic management services in Biometrics Identification Industry. The company
has successfully completed more than two dozen contracts in the last three years.
Prior to ABC Inc. Dr. Bavarian was the Vice President of Motorola Biometrics, where he led the
business turn around and four fold increase in Sales by directing the development of the industry
leading Automated Biometric Identification System products with over 100 large scale
deployments.
Before moving to the industry in 1992, Dr. Bavarian was a professor in the Department of
Electrical and Computer Engineering at the University of California, Irvine, where he conducted
original research in image processing, computer vision, intelligent systems and published over
120 technical papers and received several awards for outstanding research and distinguished
teaching.
Dr. Behnam (Ben) Bavarian received his Ph.D. in Electrical and Computer Engineering from The
Ohio State University, Columbus Ohio in 1984.
Author Biography - Dr. Ben Bavarian
28
Mr. Mehrad Tavakoli is PhD graduate student at University of California, Irvine California (UCI),
where he is pursuing his research in advancing Biometrics Identification science and its
application for Law Enforcement, Security and Commercial applications.
Mr. Tavakoli is the lead research scientist for the UCI project on Biometrics Identification on the
Move System™ (BIMS) where he has developed new techniques for using high resolution
imaging and the practical applications to capture of face, iris and fingerprint and palmprint friction
ridge details.
In his prior experience Mehrad has designed and build a Geo-Spatial Imaging system similar to
Google Streets, where massive amount of image database where captured and managed for
instant access. He has also led the R&D efforts for new projects in IT and Communications areas
such as new ad-hoc routing protocols, MAC-layer protocols, and underwater communication
which included the underwater surveillance security systems and solutions.
Mr. Tavakoli received his Masters of Science Degree from UCI, specializing in Computer
Architecture, Image processing, Signal and Systems and completing his thesis on Stand-Off
Biometrics Identification.
Author Biography - Mr. Mehrad Tavakoli
29