Post on 15-Jan-2016
description
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Optimal Experiment Design (OED)for kernel ridge regression
and
the Minimum Volume Covering Ellipsoid (MVCE)
Tijl De Bie(K.U.Leuven)
Joint work with: Alexander Dolia John Shawe-TaylorMichael Titterington
Chris Harris
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 2
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
The next hour…
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 3
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Optimal experiment design?OED?
Notation
Examples
Deliverables…
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 4
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Optimal experiment design?OED?
Notation
Examples
Deliverables…
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 5
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Optimal experiment design?OED?
Notation
Examples
Deliverables…
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 6
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
NotationOED?
Notation
Examples
Deliverables…
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 7
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
ExamplesOED?
Notation
Examples
Deliverables…
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 8
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
ExamplesOED?
Notation
Examples
Deliverables…
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 9
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Deliverables in this talk…OED?
Notation
Examples
Deliverables…
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 10
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Least squares regression (LS)
Least squares
Ridge regression
Kernel RR
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 11
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Least squares regression (LS)
Least squares
Ridge regression
Kernel RR
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 12
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Optimal experiment design for LS
Least squares
Ridge regression
Kernel RR
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 13
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Optimal experiment design for LS
Least squares
Ridge regression
Kernel RR
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 14
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Optimal experiment design for LS
Least squares
Ridge regression
Kernel RR
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 15
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Optimal experiment design for LS
Least squares
Ridge regression
Kernel RR
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 16
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Ridge regression (RR)
Least squares
Ridge regression
Kernel RR
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 17
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Ridge regression (RR)
Least squares
Ridge regression
Kernel RR
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 18
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Ridge regression (RR)
Least squares
Ridge regression
Kernel RR
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 19
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Optimal experiment design for RR
Least squares
Ridge regression
Kernel RR
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 20
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Optimal experiment design for RR
Least squares
Ridge regression
Kernel RR
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 21
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Optimal experiment design for RR
Least squares
Ridge regression
Kernel RR
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 22
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Kernel ridge regression (KRR)
Least squares
Ridge regression
Kernel RR
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 23
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Kernel ridge regression (KRR)
Least squares
Ridge regression
Kernel RR
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 24
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Kernel ridge regression (KRR)
Least squares
Ridge regression
Kernel RR
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 25
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Kernel D-OED
Least squares
Ridge regression
Kernel RR
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 26
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Kernel D-OED
Least squares
Ridge regression
Kernel RR
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 27
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
OED: summaryD-OED MVCE
stan
dard
regu
lariz
edke
rnel
Least squares
Ridge regression
Kernel RR
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 28
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Now over to novelty detection!
Novelty detection
MVCE and duality
Regularized MVCE
Kernel MVCE
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 29
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Now over to novelty detection!
Novelty detection
MVCE and duality
Regularized MVCE
Kernel MVCE
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 30
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Regularized MVCE
Novelty detection
MVCE and duality
Regularized MVCE
Kernel MVCE
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 31
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Kernel MVCE
Novelty detection
MVCE and duality
Regularized MVCE
Kernel MVCE
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 32
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Kernel MVCE
Novelty detection
MVCE and duality
Regularized MVCE
Kernel MVCE
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 33
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Kernel MVCE
Novelty detection
MVCE and duality
Regularized MVCE
Kernel MVCE
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 34
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
MVCE: summary
Novelty detection
MVCE and duality
Regularized MVCE
Kernel MVCE
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 35
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
MVCE: summaryD-OED MVCE
stan
dard
regu
lariz
edke
rnel
Novelty detection
MVCE and duality
Regularized MVCE
Kernel MVCE
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 36
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
MVCE: dealing with outliers
Novelty detection
MVCE and duality
Regularized MVCE
Kernel MVCE
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 37
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Experiments – MVCE
Linear
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 38
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Experiments – MVCE
Linear, centered
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 39
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Experiments – MVCE
RBF-kernel
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 40
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Experiments – MVCE
RBF-kernel
Soft-margin
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 41
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Experiments – D-OED
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 42
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Experiments – D-OED
2-norm
infinity norm
1-norm
Costs:
Random vs
Uniform vs
OED (blue)
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 43
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Conclusions