Post on 26-Mar-2021
Th
e R
etr
osp
ect
ive
Te
stin
g o
f
Sto
cha
stic
Lo
ss R
ese
rve
Mo
de
lsS
toch
ast
ic L
oss
Re
serv
e M
od
els
Gle
nn
Me
yers
–FC
AS
, M
AA
A,
Ph
.D.
ISO
In
no
vati
ve A
na
lyti
cs
CA
S A
nn
ua
l M
ee
tin
g,
No
vem
be
r 7
, 2
01
1
“Do
n’t
Bli
nk
–T
he
Ha
zard
s o
f O
verc
on
fid
en
ce”
Da
nie
l K
ah
ne
ma
n–
NY
T M
ag
azi
ne
, O
ct.
23
•“C
on
fid
en
ce is
a f
ee
lin
g,
on
e d
ete
rmin
ed
mo
stly
by t
he
co
he
ren
ce o
f th
e s
tory
an
d b
y t
he
ea
se w
ith
wh
ich
it
com
es
to m
ind
, e
ven
wh
en
th
e e
vid
en
ce f
or
the
sto
ry i
s sp
ars
e a
nd
un
reli
ab
le.”
–S
ub
stit
ute
th
e w
ord
“m
od
el”
fo
r “s
tory
”–
Su
bst
itu
te t
he
wo
rd “
mo
de
l” f
or
“sto
ry”
•“W
e o
fte
n in
tera
ct w
ith
pro
fess
ion
als
wh
o e
xerc
ise
th
eir
ju
dg
me
nt
wit
h e
vid
en
t co
nfi
de
nce
, so
me
tim
es
pri
din
g
the
mse
lve
s o
n t
he
po
we
r o
f th
eir
intu
itio
n.
… C
an
we
tr
ust
th
em
?”
•“
Tru
e in
tuit
ive
exp
ert
ise
is le
arn
ed
fro
m p
rolo
ng
ed
e
xpe
rie
nce
wit
h g
oo
d f
ee
db
ack
on
mis
take
s. “
Ba
ckg
rou
nd
•R
isk
ba
sed
ca
pit
al
pro
po
sals
, e
.g.
EU
So
lve
ncy
II
an
d U
SA
SM
I re
ly o
n s
toch
ast
ic m
od
els
.
–V
aR
@9
9.5
% a
nd
TV
aR
@9
9%
•T
he
re a
re m
an
y s
toch
ast
ic l
oss
re
serv
e m
od
els
•
Th
ere
are
ma
ny
sto
cha
stic
lo
ss r
ese
rve
mo
de
ls
tha
t cl
aim
to
pre
dic
t th
e d
istr
ibu
tio
n o
f u
ltim
ate
loss
es.
Are
an
y o
f th
ese
mo
de
ls r
igh
t?
E-F
oru
m P
ap
er
Join
t w
ith
Pe
ng
Sh
i –
No
rth
ern
Ill
ino
is U
niv
ers
ity
•D
esc
rib
es
a d
ata
ba
se–
Da
ta f
rom
se
vera
l Am
eri
can
In
sure
rs
–D
ata
fo
r si
x li
ne
s o
f in
sura
nce
–P
aid
an
d i
ncu
rre
d lo
ss t
ria
ng
les
–S
ub
seq
ue
nt
ou
tco
me
s–
Su
bse
qu
en
t o
utc
om
es
–A
vail
ab
le o
nli
ne
(F
ree
)
•P
red
icts
th
e d
istr
ibu
tio
n o
f o
utc
om
es
of
two
m
od
els
fo
r se
vera
l in
sure
rs f
or
Co
mm
erc
ial
Au
to
Insu
ran
ce
•Te
sts
the
pre
dic
tio
ns
ag
ain
st s
ub
seq
ue
nt
rep
ort
ed
o
utc
om
es.
Th
e C
AS
Lo
ss R
ese
rve
Da
tab
ase
•S
che
du
le P
(D
ata
fro
m P
art
s 1
-4)
for
seve
ral U
S
Insu
rers
–P
riva
te P
ass
en
ge
r A
uto
–C
om
me
rcia
l Au
to
–W
ork
ers
’ C
om
pe
nsa
tio
n–
Wo
rke
rs’
Co
mp
en
sati
on
–G
en
era
l Lia
bil
ity
–P
rod
uct
Lia
bil
ity
–M
ed
ica
l Ma
lpra
ctic
e (
Cla
ims
Ma
de
)
•A
vail
ab
le o
n C
AS
We
bsi
te –
Ne
w V
ers
ion
9/1
/20
11
htt
p:/
/ww
w.c
asa
ct.o
rg/r
ese
arc
h/i
nd
ex.
cfm
?fa
=lo
ss_
rese
rve
s_d
ata
Th
e C
AS
Lo
ss R
ese
rve
Da
tab
ase
Acc
iden
t Ye
arP
rem
ium
12
34
56
78
910
19
88
×××
×××
×××
×××
×××
×××
×××
×××
×××
×××
×××
19
89
×××
×××
×××
×××
×××
×××
×××
×××
×××
×××
←1
998
19
90
×××
×××
×××
×××
×××
×××
×××
×××
×××
←1
999
19
91
×××
×××
×××
×××
×××
×××
×××
×××
←2
000
19
92
×××
×××
×××
×××
×××
×××
×××
←2
001
19
93
×××
×××
×××
×××
×××
×××
←2
002
19
94
×××
×××
×××
×××
×××
←2
003
Sett
lem
ent
Lag
•C
an
we
pre
dic
t th
e d
istr
ibu
tio
n o
f o
utc
om
es?
Or
sum
s o
f o
utc
om
es?
19
94
×××
×××
×××
×××
×××
←2
003
19
95
×××
×××
×××
×××
←2
004
19
96
×××
×××
×××
←2
005
19
97
×××
×××
←2
006
Tra
inin
g D
ata
fro
m
19
97
Sch
ed
ule
P
Ou
tco
me
Da
ta f
rom
Late
r S
che
du
le P
s
Exa
mp
les
of
Test
s in
Th
is P
ap
er
•C
om
me
rcia
l A
uto
•5
0 I
nsu
rers
–“S
ele
cte
d”
go
ing
co
nce
rn in
sure
rs
•Te
ste
d t
wo
sto
cha
stic
lo
ss r
ese
rve
mo
de
ls
–B
oo
tstr
ap
ch
ain
la
dd
er
(BC
L) m
od
el
–B
oo
tstr
ap
ch
ain
la
dd
er
(BC
L) m
od
el
•U
sed
th
e “
Ch
ain
Lad
de
r” p
ack
ag
e in
R
•O
verd
isp
ers
ed
Po
isso
n f
or
pro
cess
ris
k.
–B
aye
sia
n A
uto
reg
ress
ive
Tw
ee
die
(BA
T)
mo
de
l
•S
ee
ne
xt s
lid
e
Th
e B
AT
Mo
de
l
•U
ses
ea
rne
d p
rem
ium
an
d in
cre
me
nta
l p
aid
lo
ss d
ata
.
•E
xpe
cte
d L
oss
Ra
tio
(E
LR)
pa
ram
ete
rs f
oll
ow
an
AR
(1)
pro
cess
.
•C
ale
nd
ar
yea
r tr
en
d p
ara
me
ters
fo
llo
w a
n A
R(1
) p
roce
ss.
pro
cess
.
•G
en
era
te p
ara
me
ters
by
a B
aye
sia
n M
CM
C m
eth
od
.
•P
roce
ss r
isk
de
scri
be
d b
y t
he
Tw
ee
die
dis
trib
uti
on
.
•P
rio
r d
istr
ibu
tio
n d
eri
ved
by
exa
min
ing
MLE
est
ima
tes
of
a s
imil
ar
mo
de
l o
n s
eve
ral in
sure
rs.
Pa
ram
ete
rs f
or
Insu
rer
91
4
Pa
ram
ete
rs f
or
Insu
rer
91
4
Pa
ram
ete
rs f
or
Insu
rer
91
4
Cri
teri
a f
or
a “
Go
od
”
Sto
cha
stic
Lo
ss R
ese
rve
Mo
de
l
•U
sin
g t
he
up
pe
r tr
ian
gle
“tr
ain
ing
” d
ata
, p
red
ict
the
dis
trib
uti
on
of
the
ou
tco
me
s in
th
e lo
we
r
tria
ng
le
–C
an
be
ob
serv
ati
on
s fr
om
in
div
idu
al (
AY,
La
g)
cell
s o
r –
Ca
n b
e o
bse
rva
tio
ns
fro
m i
nd
ivid
ua
l (A
Y, L
ag
) ce
lls
or
sum
s o
f o
bse
rva
tio
ns
in d
iffe
ren
t (A
Y,La
g)
cell
s.
•U
sin
g t
he
pre
dic
tive
dis
trib
uti
on
s, f
ind
th
e
pe
rce
nti
les
of
the
ou
tco
me
da
ta.
•T
he
pe
rce
nti
les
sho
uld
be
un
ifo
rmly
dis
trib
ute
d.
–Te
st w
ith
PP
Plo
ts/K
S t
est
s o
r w
ith
his
tog
ram
s.
Test
ing
th
e D
istr
ibu
tio
ns
of
(AY,
Lag
)
Ou
tco
me
Pe
rce
nti
les
for
a S
ing
le I
nsu
rer
BC
L -
Insu
rer
91
4
Test
ing
th
e D
istr
ibu
tio
ns
of
(AY,
Lag
)
Ou
tco
me
Pe
rce
nti
les
for
a S
ing
le I
nsu
rer
BA
T -
Insu
rer
91
4
Test
ing
th
e D
istr
ibu
tio
ns
of
(AY,
Lag
)
Ou
tco
me
Pe
rce
nti
les
for
a S
ing
le I
nsu
rer
BC
L -
Insu
rer
31
0
Test
ing
th
e D
istr
ibu
tio
ns
of
(AY,
Lag
)
Ou
tco
me
Pe
rce
nti
les
for
a S
ing
le I
nsu
rer
BA
T -
Insu
rer
31
0
Test
ing
th
e M
od
el
on
Mu
ltip
le I
nsu
rers
•E
ach
mo
de
l ca
n p
red
ict
the
dis
trib
uti
on
of
the
sum
of
all
ou
tco
me
s in
th
e l
ow
er
tria
ng
le.
•C
om
pa
re t
he
me
an
of
the
pre
dic
ted
dis
trib
uti
on
wit
h t
he
su
m o
f a
ll o
utc
om
es.
d
istr
ibu
tio
n w
ith
th
e s
um
of
all
ou
tco
me
s.
–Fo
r e
ach
mo
de
l
–Fo
r th
e p
ost
ed
re
serv
e
% E
rro
r
Pe
rce
nti
le o
f P
ost
ed
Re
serv
e
for
Ea
ch M
od
el
Test
ing
th
e M
od
el
on
Mu
ltip
le I
nsu
rers
•E
ach
mo
de
l ca
n p
red
ict
the
dis
trib
uti
on
of
the
sum
of
all
ou
tco
me
s in
th
e l
ow
er
tria
ng
le.
•F
ind
th
e p
erc
en
tile
of
the
act
ua
l su
m o
f
ou
tco
me
s fo
r e
ach
in
sure
r.o
utc
om
es
for
ea
ch i
nsu
rer.
•T
he
se p
erc
en
tile
s sh
ou
ld b
e u
nif
orm
ly
dis
trib
ute
d.
•T
his
is
a t
est
of
the
mo
de
l.
Pre
dic
ted
Pe
rce
nti
les
of
Ou
tco
me
s
Sh
ou
ld b
e
Un
ifo
rmly
Dis
trib
ute
d
Ove
rfit
tin
g!
Pre
dic
ted
Pe
rce
nti
les
of
Ou
tco
me
s
Co
ncl
usi
on
s
•N
eit
he
r th
e B
AT
or
the
BC
L d
oe
s a
go
od
jo
b a
t
pre
dic
tin
g t
he
dis
trib
uti
on
of
ou
tco
me
s.
•Tw
o p
oss
ible
re
aso
ns
–W
e d
on
’t h
ave
th
e r
igh
t m
od
el
–W
e d
on
’t h
ave
th
e r
igh
t m
od
el
–C
ha
ng
es
in t
he
cla
im s
ett
lem
en
t e
nvir
on
me
nt
ma
ke t
he
ou
tco
me
s u
np
red
icta
ble
.
Fin
din
g t
he
Rig
ht
Mo
de
l
•T
he
se m
od
els
use
d o
nly
pa
id d
ata
. C
ou
ld w
e
do
a b
ett
er
job
by
in
clu
din
g i
ncu
rre
d l
oss
da
ta?
•B
AT
use
d e
arn
ed
pre
miu
m d
ata
. D
oe
s th
is
•B
AT
use
d e
arn
ed
pre
miu
m d
ata
. D
oe
s th
is
he
lp o
r h
ind
er
the
pre
dic
tio
n?
•Is
th
ere
oth
er
ext
ern
al
da
ta a
vail
ab
le?
•W
ork
wit
h o
the
r li
ne
s o
f in
sura
nce
.
A H
int
–U
se U
np
aid
Lo
ss I
nfo
rma
tio
n
55
.3%
of
Loss
in
Te
st D
ata
55
.3%
of
Loss
in
Te
st D
ata
58
.6%
Pre
dic
ted
Loss
in
Te
st D
ata
Un
pre
dic
tab
le E
nvir
on
me
nta
l C
ha
ng
es
•If
so
, h
ow
do
we
ma
na
ge
in
sure
r ri
sk?
•S
elf
co
rre
ctin
g o
ver
tim
e?
C
an
we
ma
ke
ad
just
me
nts
as
ad
dit
ion
al
da
ta c
om
e in
?
•C
ha
lle
ng
e –
Ou
r n
ew
pro
po
sed
so
lve
ncy
•
Ch
all
en
ge
–O
ur
ne
w p
rop
ose
d s
olv
en
cy
reg
ula
tio
ns
(i.e
. E
U S
olv
en
cy I
I a
nd
Am
eri
can
SM
I) d
ep
en
d o
n o
ur
ab
ilit
y t
o p
red
ict
the
dis
trib
uti
on
of
ou
tco
me
s.
Wh
at
ha
pp
en
s if
we
can
no
t a
ccu
rate
ly p
red
ict
the
dis
trib
uti
on
s?