G89.2247 Lecture 91 Thinking about an example Pitfalls in measurement models Pitfalls in model...

G89.2247 Lecture 9 1

G89.2247Lecture 9

• Thinking about an example

• Pitfalls in measurement models

• Pitfalls in model specification

• Item Parcel Issues


Review of SEM Notation

• LISREL’s distinction between exogenous and endogenous variables simplifies the computational expressions. Yy X=x

• EQS and AMOS use an approach that does not make this distinctionEssentially, all variables are potentially endogenousThis allows for a wider class of models to be considered,

especially with regard to correlated residuals.


Example of Model that cannot be fit using Exogenous/Endogenous distinction

• Suppose that F1 is a baseline measure and F3 is the same measure at time 2. the biases of E1, E2 and

E3 may be reflected in E7, E8, E9.

• LISREL can also handle this by calling all variables endogenous

V1

V2

V3

V4

V5

V6

V7

V8

V9

F1

F2

F3

E1

E9

E8

E7

E6

E4

E3

E2

D3

D2

E5


How would we model this example?

• Stressful life events(L) both result from and lead to mental dysfunction(D), but coping strategies(C) can reduce the impact of the stressful events, as can support (S)

• Stressful Events, distress, coping, support are typically measured by self-report, which may introduce error

• Panel data is sometimes helpful to study the processes


Suppose we have these measures at three points in time

• L: Life eventsSchool hassles (count per week)Family conflicts (count per week)Urban events (thefts, traffic jams, etc)

• D: DistressAnxietyDepressionAngerLow self esteem


Suppose we have these measures at three points in time (continued)

• C: CopingDenial actionsDistraction actionsProblem-focused actions

• S: SupportPerceived support from confident

• Hugs and kisses, practical helpConfidant's report of support

• Hugs and kisses, practical help


Possible pathways

• L => C• C (-)=>D• C (-)=> L• C => S• D => S• D (-)=> C• D =>C• D => L• S => L


Possible Measurement Models

• Latent variables forDistress?Support?Coping?Life Stress?


Bollen and Lennox(1991)

• We can envision the relation between a construct and three manifest variables in two ways:

G89.2247 Lecture 9 10

Equivalent Models

• Some models that look very different have the same fit

G89.2247 Lecture 9 11

Issues in Defining Measurement Models

• Latent variables are used to correct for error through multiple indicatorsSometimes multiple indicators do not existSome researchers suggest making up quasi multiple

indicators by creating “parcels” of items

• The proper use of parcels is controversialE.g. Little, Cunningham, Shahar, Widaman (2002) To

parcel of not to parcel: Exploring the question, weighing the merits. Structural Equation Modeling 9(2), 151-173.

G89.2247 Lecture 9 12

Example: CES-Depression "Scale"

• DURING THE PAST WEEK I felt depressed. I felt that I could not shake off the blues even with

help from my family or friends. I felt sad. I could not get going.My sleep was restless. I felt that everything I did was an effort. I felt that people dislike me. I thought my life had been a failure. People were unfriendly.

G89.2247 Lecture 9 13

Conventional Use of Scales Such as CES-D

• Items have 0-4 response categories• Factor structure is generally ignored• Items are summed without weights into a

single scale score• Alpha coefficient of .8 to .9 generally

underestimates true reliability• Test retest reliability often high despite explicit

time frame

G89.2247 Lecture 9 14

Relation of Outcome Y and CES-DAccounting for Measurement Error

• Suppose Y is measure of functioning in workplace• SEM approach can be recommended for CES-D, BUT

Studies usually do not have multiple indicators of depression other than items

Sample sizes usually do not allow item-level analyses

• Kishton and Widaman (1994) laid out alternative approaches to forming parcels.Factor-based unidimensional parcels (FBP)Domain Representative parcels (DRP)

G89.2247 Lecture 9 15

Simple SEM Model Relating Depression to Outcome, Y

Y

Depression

Parcel 1

Parcel 2

Parcel 3

G89.2247 Lecture 9 16

Factor-based Unidimensional Parcels

• Form parcels with items that relate to specific subdomains of scale

• E.G., FBP1=A1+A2+A3A1: I felt depressed.

A2: I felt that I could not shake off the blues even with help from my family or friends.

A3: I felt sad.

G89.2247 Lecture 9 17

Domain Representative Parcels

• Form parcels with items that span the subdomains of item set

• E.G., DRP1=A1+B1+C1A1: I felt depressed.B1: I could not get going.C1: I felt that people dislike me.

• Parcels are designed to be close to parallel measures

G89.2247 Lecture 9 18

Possible "True" Structures: Second Order Factor Relation

• Suppose Y is related to second order factor1

1

1

1

1

1

1

1

1

h

hhh

hhh

h

hFA

FB

FC

A1

A2

A3

B1

B2

B3

C1

C2

C3 1

a

g

g

gF2

Y

G89.2247 Lecture 9 19

Possible "True" Structures: First Order Factor Unique Effects

• Graham and Tatterson (2000) considered this1

1

1

1

1

1

1

1

1

h

hhh

hhh

h

hFA

FB

FC

A1

A2

A3

B1

B2

B3

C1

C2

C3 1

a

g

g

gF2

Y

b

b

b

G89.2247 Lecture 9 20

Possible "True" Structures: Item Unique Effects

• Mental health symptoms can have unique effects

h

hhh

hhh

h

hFA

FB

FC

A1

A2

A3

B1

B2

B3

C1

C2

C3 1

a

g

g

gF2

Y

b

b

b

G89.2247 Lecture 9 21

Exploration of Parcel Strategies for Different Assumed Structures

• Parcel models are generally mispecified relative to assumed modelException is FBP parcel model for Second Order

Factor relation

• What is direction and magnitude of bias?Look at R-Square and parameter estimates

• Compare to simple sum of CESD items

G89.2247 Lecture 9 22

Simulated Results Assuming Second Order Factor Relation

• Parameters in structure consideredFirst order factor loadings .7Second order factor loadings .7Structural path .7True R square .50

• R square for simple sum of nine items: 0.31• R square for FBP parcel model: 0.50• R square for DRP parcel model: 0.37

G89.2247 Lecture 9 23

Simulated Results Assuming Second Order Factor Plus Unique First Order Factor Effects

• Parameters in structure consideredFirst order factor loadings .7Second order factor loadings .7Structural paths .55 SOF, .20 unique FOFTrue R square .49


G89.2247 Lecture 9 24

Simulated Results Assuming Second Order Factor Plus Unique Effects for Items

• Parameters in structure consideredFirst order factor loadings .7Second order factor loadings .7Structural paths .45 SOF, .10 unique itemTrue R square .54


G89.2247 Lecture 9 25

Conclusion

• If one has confidence that a second order factor accounts for the relation between scales like the CESD and outcomesFactor Based parcels are idealDomain Representative parcels do better than sum score

• If the relation between an outcome and a heterogeneous item set involves first order unique factors or unique item effectsThe FBP parcels can be misleadingDomain Representative parcels do better than sum score

G89.2247 Lecture 9 26

Comment on Bias of FBP

• The correlations of Y to the FBP1, FBP2 and FBP3 are larger than what is explained by the original conceptual model. Cohen, Cohen, Teresi, Marchi, Velez (1990)

• Y becomes the dominant indicator of the latent variable.

Y

Depression

Parcel 1

Parcel 2

Parcel 3

Y

G89.2247 Lecture 91 Thinking about an example Pitfalls in measurement models Pitfalls in model...

Documents

Transcript of G89.2247 Lecture 91 Thinking about an example Pitfalls in measurement models Pitfalls in model...