Creating Tests that Measure Critical Thinking in Nursing Education
48
Creating Tests that Measure Critical Thinking in Nursing An ExamSoft Client Webinar
Transcript of Creating Tests that Measure Critical Thinking in Nursing Education
Creating Tests that Measure Critical
Thinking in Nursing An ExamSoft Client Webinar
Crea%ng tests that measure
cri%cal thinking in nursing: Test item wri%ng that is both art and science
Ainslie t. Nibert, PHD, RN, FAAN February 17, 2015
Resources for Developing Cri2cal Thinking Test Items and Alternate Format Items:
Na2onal Council Website
• www.nscbn.org – NCLEX Test Plans
• 2013 RN • 2014 PN
– Candidate FAQ – Alternate item formats FAQ – Exam Development FAQ
3
Source: https://www.ncsbn.org/2324.htm
Rela%onship between Tes%ng & the Curriculum
4
Internal and External
Curriculum Evaluation
Outcome Predictors
5
q Standardized Exams
q NCLEX-RN/PN®
External Evaluation Compare Individual Group with Population at Large
6
q Writing Critical Thinking Test Items
q Item Analysis Software & Blueprinting
q Test Item Banking & Exam Delivery
Internal Evaluation Evaluation of course objectives (faculty designed or outsourced)
7
Five Guidelines to Developing Effective Critical Thinking Exams
q Assemble the “basics.”
q Write critical thinking test items. q Pay attention to housekeeping duties.
q Develop a test blueprint. q Scientifically analyze all exams.
8
Sound Instruction
Educator’s
Golden
Triangle Instruction
Evaluation Objectives
Outcomes
9
Philosophy Outcomes Objectives
Instructional Methods
Evaluation Tools
10
Definition
Critical Thinking
The process of analyzing and understanding how and why we reached a certain conclusion.
11
Bloom‘s Taxonomy: Benjamin Bloom, 1956 (revised)
Terminology changes "The graphic is a representation of the NEW verbage associated with the long familiar Bloom's Taxonomy. Note the change from Nouns to Verbs [e.g., Application to Applying] to describe the different levels of the taxonomy. Note that the top two levels are essentially exchanged from the Old to the New version." (Schultz, 2005) (Evaluation moved from the top to Evaluating in the second from the top, Synthesis moved from second on top to the top as Creating.) Source: http://www.odu.edu/educ/llschult/blooms_taxonomy.htm
Post-Exam Item Analysis: An important aspect of item writing
Helps to determine the quality of a test
12
13
Consistency of Scores
Reliability Tools
14
q Kuder-Richardson Formula 20 (KR20)—EXAM
Ø Range from –1 to + 1
q Point Biserial Correlation Coefficient (PBCC)—TEST ITEMS
Ø Range from – 1 to + 1
15
q Item difficulty 30% - 90%
q Item Discrimination Ratio 25% and Above
q PBCC 0.20 and Above
q KR20 0.70 and Above
Standards of Acceptance
Standards of Acceptance Nursing
Nursing-PBCC 0.15 and Above
Nursing-KR20 0.60 - 0.65 and Above
16
17
3-Step Method for Item Analysis
1. Review Difficulty Level
2. Review Discrimination Data
q Item Discrimination Ratio (IDR)
q Point Biserial Correlation Coefficient (PBCC)
3. Review Effectiveness of Alternatives
q Response Frequencies
q Non-distracters Source: Morrison, Nibert, Flick, J. (2006). Critical thinking and test item writing (2nd ed.).Houston, TX: Health Education Systems, Inc.
18
19
20
Content Validity
Does the test measure what it claims to measure?
21
Content Validity
q Test Blueprint Ø Rational/Logical Pre-Assessment Tool
q Database Programs
Ø Storage of item discrimination data
22
Test Blueprints
q Faculty Generated
q Electronically Generated
An electronic blueprint for each exam in each course
23
24
NCLEX-‐RN® Client Needs Percentages of Items 2011 vs. 2014
25
Source: https://www.ncsbn.org/4701.htm
NCLEX-‐RN® Client Needs Percentages of Items 2011 vs. 2014
Increases vs. Decreases
26
27
Cri%cal Thinking Test Items
q Contain Rationale
q Written at the Application Level or Above
q Require Multilogical Thinking to Answer
q Ask for High Level of Discrimination
Source: Morrison, Nibert, & Flick (2006)
28
Test Item Rationales
Test Review
q Learning Experience
q Faculty Friendly
q Legally Defensible
29
Written at the Application Level and Above
q Prepare students for NCLEX®
q Promote thinking about clinical problems
q Cause teaching methods to become creative
30
Require Multilogical Thinking to Answer
Definition
Thinking that requires knowledge of more than one fact to logically and systematically apply concepts to a clinical problem
Highly Discriminating
• Questions should provide viable alternatives that require a high degree of discrimination to answer.
31
Cri%cally-‐thinking Ques%ons
Which interven2on is most important? Which interven2on, plan, assessment data is/are most cri2cal to developing a plan of care?
Which interven2on should be done first?
What ac2on should the nurse take first? Which interven2on, plan, nursing ac2on has the highest priority?
What response is best? 32
NCLEX® AlternaOve Test Item Formats
• Mul2ple-‐response items • Fill-‐in-‐the-‐blank items • Hot spot • Chart/exhibit format • Ordered Response items (Ranking) • Audio item format • Graphic Op2ons (graphics imbedded as answer op2ons)
• Any item formats, including standard mul2ple-‐choice items, may include mul2media, charts, tables or graphic images.
33
Latest NCLEX® Test Item Format ConsideraOons
Units of Measure • Interna2onal Systems of Units (SI) • Metric • Imperial Measurement
Generic vs. Trade Names for Medica2ons • Generic names only in most cases • References to general classifica2ons of medica2ons
34
35
Housekeeping Tips
36
Item Writing Rules
q Get rid of names
q Get rid of ‘multiple’ multiples
q Use non-sexist writing style
q Develop parsimonious writing style
Ø Cross out “of the following”
Ø Delete scenarios
q Write items independent of each other
37
… and More Rules
q Use a question format when possible
q Make distracters plausible and homogeneous
q Equal in length
q No opposites
38
… and More Rules
q Eliminate “all of the above” and “none of the above”
q Rewrite any “all except” questions
q Ensure that alternatives do not overlap
q Vary correct answer
39
… and the MOST IMPORTANT Rule
Develop written testing policies
Ø A Role of the Testing Committee
Ø Guidelines: Writing style & Format
Ø Exam Administration Procedures
40
41
Tes%ng CommiNee Responsibili%es
Recommenda2ons typically made to the Curriculum and Student Affairs Commi]ees to coordinate policy crea2on & enforcement:
◦ Wri2ng style ◦ Format ◦ Pilot items ◦ Grades/scores ◦ Review and analysis ◦ Item revision ◦ Track students ◦ Accredita2on prep
Standardized Tes%ng: Vigilance with Test Security
1. Encourage moral behavior (Academic honesty program at your school with clear language placed in handbooks)
2. Discourage chea2ng a. Before tes2ng
1. Minimize access to exams and viewing of exam content 2. Use highest levels of security available in Blackboard for unit tests and all
security features available in standardized tes2ng plaborm -‐ protect logins & access codes; ac2ve dashboarding
3. Train proctors for live proctoring ac2vi2es
b. During tes2ng 1. Establish secure environment 2. No devia2ons to test procedures or breakdown of environmental security
allowed. Ex: leaving room equates to the test being over for that student regardless of reason
3. Vigilant proctoring
3. Detect chea2ng with Data Forensics and take ac2ons as needed
42
43
Item Writing Tools for Success …
Knowledge
Test Blueprint
Testing Software
45
Faculty
Have QuesOons? Need More Info?
Thanks for your 2me & a]en2on today!
46
866-429-8889
References American Psychological Associa2on. (2004) Code of Fair Tes2ng Prac2ces in Educa2on.
Washington, DC: Joint Commi]ee on Tes2ng Prac2ces. h]p://www.apa.org/science/programs/tes2ng/fair-‐code.aspx
Morrison, S., Nibert, A., & Flick, J. (2006). Cri$cal thinking and test item wri$ng (2nd ed.). Houston, TX: Health Educa2on Systems, Inc.
Morrison, S. (2004). Improving NCLEX-‐RN pass rates through internal and external curriculum evalua2on. In M. Oermann & K. Heinrich (Eds.), Annual review of nursing educa2on (Vol. 3). New York: Springer
Na2onal Council of State Boards of Nursing. (2013) 2013 NCLEX-‐RN test plan. Chicago, IL: Na2onal Council of State Boards of Nursing. h]ps://www.ncsbn.org/3795.htm
Nibert, A. (2010) Benchmarking for student progression throughout a nursing program: Implica$ons for students, faculty, and administrators. In Capu2, L. (Ed.), Teaching nursing: The art and science, 2nd ed. (Vol. 3). (pp.45-‐64). Chicago: College of DuPage Press.
47
Click to edit Master title style
Click to edit Master subtitle style For More Information:
Call: 1.866.429.8889
Email: [email protected]
Visit: learn.examsoft.com
![Cantor Groups, Haar Measure and Lebesgue Measure on · PDF fileCantor Groups, Haar Measure and Lebesgue Measure on [0;1] Michael Mislove Tulane University Domains XI Paris Tuesday,](https://static.fdocument.pub/doc/165x107/5aaaf5b87f8b9a90188ecb94/cantor-groups-haar-measure-and-lebesgue-measure-on-groups-haar-measure-and.jpg)
