The Retrospective Testing of Stochastic Loss Reserve ......The Retrospective Testing of Stochastic...
26
The Retrospective Testing of Stochastic Loss Reserve Models Glenn Meyers –FCAS, MAAA, Ph.D. ISO Innovative Analytics CAS Annual Meeting, November 7, 2011
Transcript of The Retrospective Testing of Stochastic Loss Reserve ......The Retrospective Testing of Stochastic...
Th
e R
etr
osp
ect
ive
Te
stin
g o
f
Sto
cha
stic
Lo
ss R
ese
rve
Mo
de
lsS
toch
ast
ic L
oss
Re
serv
e M
od
els
Gle
nn
Me
yers
–FC
AS
, M
AA
A,
Ph
.D.
ISO
In
no
vati
ve A
na
lyti
cs
CA
S A
nn
ua
l M
ee
tin
g,
No
vem
be
r 7
, 2
01
1
“Do
n’t
Bli
nk
–T
he
Ha
zard
s o
f O
verc
on
fid
en
ce”
Da
nie
l K
ah
ne
ma
n–
NY
T M
ag
azi
ne
, O
ct.
23
•“C
on
fid
en
ce is
a f
ee
lin
g,
on
e d
ete
rmin
ed
mo
stly
by t
he
co
he
ren
ce o
f th
e s
tory
an
d b
y t
he
ea
se w
ith
wh
ich
it
com
es
to m
ind
, e
ven
wh
en
th
e e
vid
en
ce f
or
the
sto
ry i
s sp
ars
e a
nd
un
reli
ab
le.”
–S
ub
stit
ute
th
e w
ord
“m
od
el”
fo
r “s
tory
”–
Su
bst
itu
te t
he
wo
rd “
mo
de
l” f
or
“sto
ry”
•“W
e o
fte
n in
tera
ct w
ith
pro
fess
ion
als
wh
o e
xerc
ise
th
eir
ju
dg
me
nt
wit
h e
vid
en
t co
nfi
de
nce
, so
me
tim
es
pri
din
g
the
mse
lve
s o
n t
he
po
we
r o
f th
eir
intu
itio
n.
… C
an
we
tr
ust
th
em
?”
•“
Tru
e in
tuit
ive
exp
ert
ise
is le
arn
ed
fro
m p
rolo
ng
ed
e
xpe
rie
nce
wit
h g
oo
d f
ee
db
ack
on
mis
take
s. “
Ba
ckg
rou
nd
•R
isk
ba
sed
ca
pit
al
pro
po
sals
, e
.g.
EU
So
lve
ncy
II
an
d U
SA
SM
I re
ly o
n s
toch
ast
ic m
od
els
.
–V
aR
@9
9.5
% a
nd
TV
aR
@9
9%
•T
he
re a
re m
an
y s
toch
ast
ic l
oss
re
serv
e m
od
els
•
Th
ere
are
ma
ny
sto
cha
stic
lo
ss r
ese
rve
mo
de
ls
tha
t cl
aim
to
pre
dic
t th
e d
istr
ibu
tio
n o
f u
ltim
ate
loss
es.
Are
an
y o
f th
ese
mo
de
ls r
igh
t?
E-F
oru
m P
ap
er
Join
t w
ith
Pe
ng
Sh
i –
No
rth
ern
Ill
ino
is U
niv
ers
ity
•D
esc
rib
es
a d
ata
ba
se–
Da
ta f
rom
se
vera
l Am
eri
can
In
sure
rs
–D
ata
fo
r si
x li
ne
s o
f in
sura
nce
–P
aid
an
d i
ncu
rre
d lo
ss t
ria
ng
les
–S
ub
seq
ue
nt
ou
tco
me
s–
Su
bse
qu
en
t o
utc
om
es
–A
vail
ab
le o
nli
ne
(F
ree
)
•P
red
icts
th
e d
istr
ibu
tio
n o
f o
utc
om
es
of
two
m
od
els
fo
r se
vera
l in
sure
rs f
or
Co
mm
erc
ial
Au
to
Insu
ran
ce
•Te
sts
the
pre
dic
tio
ns
ag
ain
st s
ub
seq
ue
nt
rep
ort
ed
o
utc
om
es.
Th
e C
AS
Lo
ss R
ese
rve
Da
tab
ase
•S
che
du
le P
(D
ata
fro
m P
art
s 1
-4)
for
seve
ral U
S
Insu
rers
–P
riva
te P
ass
en
ge
r A
uto
–C
om
me
rcia
l Au
to
–W
ork
ers
’ C
om
pe
nsa
tio
n–
Wo
rke
rs’
Co
mp
en
sati
on
–G
en
era
l Lia
bil
ity
–P
rod
uct
Lia
bil
ity
–M
ed
ica
l Ma
lpra
ctic
e (
Cla
ims
Ma
de
)
•A
vail
ab
le o
n C
AS
We
bsi
te –
Ne
w V
ers
ion
9/1
/20
11
htt
p:/
/ww
w.c
asa
ct.o
rg/r
ese
arc
h/i
nd
ex.
cfm
?fa
=lo
ss_
rese
rve
s_d
ata
Th
e C
AS
Lo
ss R
ese
rve
Da
tab
ase
Acc
iden
t Ye
arP
rem
ium
12
34
56
78
910
19
88
×××
×××
×××
×××
×××
×××
×××
×××
×××
×××
×××
19
89
×××
×××
×××
×××
×××
×××
×××
×××
×××
×××
←1
998
19
90
×××
×××
×××
×××
×××
×××
×××
×××
×××
←1
999
19
91
×××
×××
×××
×××
×××
×××
×××
×××
←2
000
19
92
×××
×××
×××
×××
×××
×××
×××
←2
001
19
93
×××
×××
×××
×××
×××
×××
←2
002
19
94
×××
×××
×××
×××
×××
←2
003
Sett
lem
ent
Lag
•C
an
we
pre
dic
t th
e d
istr
ibu
tio
n o
f o
utc
om
es?
Or
sum
s o
f o
utc
om
es?
19
94
×××
×××
×××
×××
×××
←2
003
19
95
×××
×××
×××
×××
←2
004
19
96
×××
×××
×××
←2
005
19
97
×××
×××
←2
006
Tra
inin
g D
ata
fro
m
19
97
Sch
ed
ule
P
Ou
tco
me
Da
ta f
rom
Late
r S
che
du
le P
s
Exa
mp
les
of
Test
s in
Th
is P
ap
er
•C
om
me
rcia
l A
uto
•5
0 I
nsu
rers
–“S
ele
cte
d”
go
ing
co
nce
rn in
sure
rs
•Te
ste
d t
wo
sto
cha
stic
lo
ss r
ese
rve
mo
de
ls
–B
oo
tstr
ap
ch
ain
la
dd
er
(BC
L) m
od
el
–B
oo
tstr
ap
ch
ain
la
dd
er
(BC
L) m
od
el
•U
sed
th
e “
Ch
ain
Lad
de
r” p
ack
ag
e in
R
•O
verd
isp
ers
ed
Po
isso
n f
or
pro
cess
ris
k.
–B
aye
sia
n A
uto
reg
ress
ive
Tw
ee
die
(BA
T)
mo
de
l
•S
ee
ne
xt s
lid
e
Th
e B
AT
Mo
de
l
•U
ses
ea
rne
d p
rem
ium
an
d in
cre
me
nta
l p
aid
lo
ss d
ata
.
•E
xpe
cte
d L
oss
Ra
tio
(E
LR)
pa
ram
ete
rs f
oll
ow
an
AR
(1)
pro
cess
.
•C
ale
nd
ar
yea
r tr
en
d p
ara
me
ters
fo
llo
w a
n A
R(1
) p
roce
ss.
pro
cess
.
•G
en
era
te p
ara
me
ters
by
a B
aye
sia
n M
CM
C m
eth
od
.
•P
roce
ss r
isk
de
scri
be
d b
y t
he
Tw
ee
die
dis
trib
uti
on
.
•P
rio
r d
istr
ibu
tio
n d
eri
ved
by
exa
min
ing
MLE
est
ima
tes
of
a s
imil
ar
mo
de
l o
n s
eve
ral in
sure
rs.
Pa
ram
ete
rs f
or
Insu
rer
91
4
Pa
ram
ete
rs f
or
Insu
rer
91
4
Pa
ram
ete
rs f
or
Insu
rer
91
4
Cri
teri
a f
or
a “
Go
od
”
Sto
cha
stic
Lo
ss R
ese
rve
Mo
de
l
•U
sin
g t
he
up
pe
r tr
ian
gle
“tr
ain
ing
” d
ata
, p
red
ict
the
dis
trib
uti
on
of
the
ou
tco
me
s in
th
e lo
we
r
tria
ng
le
–C
an
be
ob
serv
ati
on
s fr
om
in
div
idu
al (
AY,
La
g)
cell
s o
r –
Ca
n b
e o
bse
rva
tio
ns
fro
m i
nd
ivid
ua
l (A
Y, L
ag
) ce
lls
or
sum
s o
f o
bse
rva
tio
ns
in d
iffe
ren
t (A
Y,La
g)
cell
s.
•U
sin
g t
he
pre
dic
tive
dis
trib
uti
on
s, f
ind
th
e
pe
rce
nti
les
of
the
ou
tco
me
da
ta.
•T
he
pe
rce
nti
les
sho
uld
be
un
ifo
rmly
dis
trib
ute
d.
–Te
st w
ith
PP
Plo
ts/K
S t
est
s o
r w
ith
his
tog
ram
s.
Test
ing
th
e D
istr
ibu
tio
ns
of
(AY,
Lag
)
Ou
tco
me
Pe
rce
nti
les
for
a S
ing
le I
nsu
rer
BC
L -
Insu
rer
91
4
Test
ing
th
e D
istr
ibu
tio
ns
of
(AY,
Lag
)
Ou
tco
me
Pe
rce
nti
les
for
a S
ing
le I
nsu
rer
BA
T -
Insu
rer
91
4
Test
ing
th
e D
istr
ibu
tio
ns
of
(AY,
Lag
)
Ou
tco
me
Pe
rce
nti
les
for
a S
ing
le I
nsu
rer
BC
L -
Insu
rer
31
0
Test
ing
th
e D
istr
ibu
tio
ns
of
(AY,
Lag
)
Ou
tco
me
Pe
rce
nti
les
for
a S
ing
le I
nsu
rer
BA
T -
Insu
rer
31
0
Test
ing
th
e M
od
el
on
Mu
ltip
le I
nsu
rers
•E
ach
mo
de
l ca
n p
red
ict
the
dis
trib
uti
on
of
the
sum
of
all
ou
tco
me
s in
th
e l
ow
er
tria
ng
le.
•C
om
pa
re t
he
me
an
of
the
pre
dic
ted
dis
trib
uti
on
wit
h t
he
su
m o
f a
ll o
utc
om
es.
d
istr
ibu
tio
n w
ith
th
e s
um
of
all
ou
tco
me
s.
–Fo
r e
ach
mo
de
l
–Fo
r th
e p
ost
ed
re
serv
e
% E
rro
r
Pe
rce
nti
le o
f P
ost
ed
Re
serv
e
for
Ea
ch M
od
el
Test
ing
th
e M
od
el
on
Mu
ltip
le I
nsu
rers
•E
ach
mo
de
l ca
n p
red
ict
the
dis
trib
uti
on
of
the
sum
of
all
ou
tco
me
s in
th
e l
ow
er
tria
ng
le.
•F
ind
th
e p
erc
en
tile
of
the
act
ua
l su
m o
f
ou
tco
me
s fo
r e
ach
in
sure
r.o
utc
om
es
for
ea
ch i
nsu
rer.
•T
he
se p
erc
en
tile
s sh
ou
ld b
e u
nif
orm
ly
dis
trib
ute
d.
•T
his
is
a t
est
of
the
mo
de
l.
Pre
dic
ted
Pe
rce
nti
les
of
Ou
tco
me
s
Sh
ou
ld b
e
Un
ifo
rmly
Dis
trib
ute
d
Ove
rfit
tin
g!
Pre
dic
ted
Pe
rce
nti
les
of
Ou
tco
me
s
Co
ncl
usi
on
s
•N
eit
he
r th
e B
AT
or
the
BC
L d
oe
s a
go
od
jo
b a
t
pre
dic
tin
g t
he
dis
trib
uti
on
of
ou
tco
me
s.
•Tw
o p
oss
ible
re
aso
ns
–W
e d
on
’t h
ave
th
e r
igh
t m
od
el
–W
e d
on
’t h
ave
th
e r
igh
t m
od
el
–C
ha
ng
es
in t
he
cla
im s
ett
lem
en
t e
nvir
on
me
nt
ma
ke t
he
ou
tco
me
s u
np
red
icta
ble
.
Fin
din
g t
he
Rig
ht
Mo
de
l
•T
he
se m
od
els
use
d o
nly
pa
id d
ata
. C
ou
ld w
e
do
a b
ett
er
job
by
in
clu
din
g i
ncu
rre
d l
oss
da
ta?
•B
AT
use
d e
arn
ed
pre
miu
m d
ata
. D
oe
s th
is
•B
AT
use
d e
arn
ed
pre
miu
m d
ata
. D
oe
s th
is
he
lp o
r h
ind
er
the
pre
dic
tio
n?
•Is
th
ere
oth
er
ext
ern
al
da
ta a
vail
ab
le?
•W
ork
wit
h o
the
r li
ne
s o
f in
sura
nce
.
A H
int
–U
se U
np
aid
Lo
ss I
nfo
rma
tio
n
55
.3%
of
Loss
in
Te
st D
ata
55
.3%
of
Loss
in
Te
st D
ata
58
.6%
Pre
dic
ted
Loss
in
Te
st D
ata
Un
pre
dic
tab
le E
nvir
on
me
nta
l C
ha
ng
es
•If
so
, h
ow
do
we
ma
na
ge
in
sure
r ri
sk?
•S
elf
co
rre
ctin
g o
ver
tim
e?
C
an
we
ma
ke
ad
just
me
nts
as
ad
dit
ion
al
da
ta c
om
e in
?
•C
ha
lle
ng
e –
Ou
r n
ew
pro
po
sed
so
lve
ncy
•
Ch
all
en
ge
–O
ur
ne
w p
rop
ose
d s
olv
en
cy
reg
ula
tio
ns
(i.e
. E
U S
olv
en
cy I
I a
nd
Am
eri
can
SM
I) d
ep
en
d o
n o
ur
ab
ilit
y t
o p
red
ict
the
dis
trib
uti
on
of
ou
tco
me
s.
Wh
at
ha
pp
en
s if
we
can
no
t a
ccu
rate
ly p
red
ict
the
dis
trib
uti
on
s?
