Based on 2013-2014 Training in School...
177
Specialist-Level Portfolio Internship Year Based on 2013-2014 Training in School Psychology Katherine Newman, M.Ed. University of Cincinnati
Transcript of Based on 2013-2014 Training in School...
Specialist-Level Portfolio
Internship Year
Based on 2013-2014
Training in School
Psychology
Katherine Newman, M.Ed.
University of Cincinnati
TABLE OF CONTENTS 2
Table of Contents
Introduction to Portfolio………………………………………………………………………......3
Professional Vitae………………………………………………………………………………....4
Systems Change: Grades 6-8 Implementation of NWEA MAP Assessment to Inform
Reading and Math Intervention......................................................................................................8
A Tier I Academic Classwide Peer Tutoring Intervention for Increasing Oral Reading
Fluency in a Second Grade Classroom …..…………………………………………………..….29
A Tier I Behavior Intervention: Educating Ninth Grade Teachers on Positive Behavior
Practices to Decrease Classroom Discipline Referrals………………………………………......45
A Tier II Academic Intervention for Increasing MAZE Comprehension Scores of Six
Ninth Grade Students ………………………………………..………………………………......72
A Tier II Behavior Small Group Intervention to Increase Positive Peer and Positive Adult
Interactions in Eighth Grade Students……………………………………………..…………...105
A Tier III Academic Intervention to Improve the Early Reading Skills of a Fourth
Grade Student …………………………………………………………………………………..127
A Tier III Behavior DBRC Intervention for Decreasing Off-Task Behaviors and Increasing
Academic Engagement in Mathematics for a Sixth Grade Student…………………………….151
INTRO 3
Introduction to the Third Year School Psychology Internship Portfolio
This portfolio is a culmination of my graduate-level training through the University of
Cincinnati School Psychology Specialist- Level program as well as the 1,500 hour internship
with the West Clermont Local School District during the 2013-2014 academic year. A sample of
the experiences gained as an intern school psychologist are included in this portfolio through the
seven entries. Entries included represent both academic and behavioral interventions on each of
the three Tiers, as part of the Response to Intervention (RtI) framework.
PROFESSIONAL VITA 4
Katherine Newman, M.Ed. 163 Old State Route 32, Peebles, Ohio
937-205-5377
EDUCATION:
Educational Specialist (Ed.S.) of School Psychology, Anticipated May 2014 University of Cincinnati, OH
NASP Approved, ABAI Accredited
Masters of Education (M.Ed.), June 2012 University of Cincinnati, OH
GPA: 4.0
Bachelors of Science in Human Development and Family Science, June 2011
The Ohio State University, Columbus, OH
Minors: Education & Psychology
GPA: 3.958, Summa Cum Laude
CERTIFICATION/LICENSURE:
Temporary Pupil Services License, School Psychology, Ohio Department of Education
State ID: OH3175678
August 2013-June 2014
*Have applied for and anticipate professional licensure and national certification
WORK EXPERIENCE:
School Psychologist Intern, August 2013-present
West Clermont Local School District; Batavia Ohio, K-12
Conducted initial multi-factored evaluations through the use of the Response to
Intervention (RtI) evaluation process at the elementary level
Conducted re-evaluations of students with low-incidence and high-incidence
disabilities across grade levels
Created and evaluated academic and behavioral, research-based interventions at
each Tier within the Response to Intervention (RtI) tiered model, across grade
levels
Analyzed reading and math universal screening data, Ohio Achievement
Assessment scores, and MAP assessment scores to provide placement into research
based intervention groups at the elementary and middle school levels
Participated in multiple fall and winter grade-level team data analysis meetings to
match students in need of targeted interventions with appropriate small group
interventions in reading and math at the elementary and middle school levels
Conducted curriculum-based universal screening measures for fall, winter, and
spring benchmarks, including DIBELS Next Assessments and NWEA MAP
assessments at the elementary and middle school levels
PROFESSIONAL VITA 5
Analyzed high school discipline data to inform Positive Behavior Support practices
and provided consultation for Tier 1 Positive Behavior Support at the high school
level
Collaborated with high school building administrators to develop a set of
procedures for managing discipline referrals and collaborated to create a school-
wide form for use in the classroom to manage low-level problem behavior
Completed Functional Behavioral Assessment (FBA) for challenging behavior and
collaborated with school team in developing behavioral interventions for severe
problem behavior at the high school level
School Psychology Practicum Student, August 2012- June 2013
Saint Ignatius Loyola School, Cincinnati, OH
200+ hours of pre-internship supervised experience
Consulted, designed and monitored the progress of individualized interventions
using a problem-solving tiered approach to service delivery, in collaboration with
teachers and parents
Assisted with conducting benchmark progress monitoring K-8th
grade at each
quarter of the school year
Arlitt Child and Family Research and Education Center, Cincinnati, OH
200+ hours of pre-internship supervised experience
Consulted, designed and monitored the progress of individualized interventions
using a problem-solving tiered approach to service delivery, in collaboration with
teachers and parents
Participated in the development and delivery of staff and parent developments
associated with Arlitt Early Childhood Center
Delhi Middle School, Delhi, OH
Conducted a Functional Behavioral Assessment (FBA) on a 14-year old student
20+ hours of pre-internship supervised experienced
Social Skills Group Facilitator, September 2012-November 2012
Delhi Middle School, Delhi, OH
Co-facilitated eight, weekly small group (Tier II) social skills lessons with 8th
grade female students
20+ hours of pre-internship supervised experience
Literacy Tutor, October 2011-May 2012
Winton Woods Primary South School, Cincinnati, OH
Conducted three small group (Tier II) literacy tutoring sessions using Teacher-
Directed KPALS with kindergarten students three times per week
PRESENTATIONS:
PROFESSIONAL VITA 6
McCoy, D., Newman, K. & Schneider, M. (accepted). What is Positive Behavior
Supports (PBS) and what does it look like in a preschool classroom? Poster to be
presented at the annual meeting of the Ohio Association for the Education of
Young Children, Columbus, Ohio. Contributor: Dave Barnett.
McCoy, D., Newman, K. & Schneider, M. (2013). What is Positive Behavior
Supports (PBS) and what does it look like in a preschool classroom? Presentation
presented at the annual meeting of the Cincinnati-Hamilton County Community
Action Agency, Cincinnati, Ohio. Contributor: Dave Barnett.
PROFESSIONAL MEMBERSHIPS:
National Association of School Psychologists (NASP)
Ohio School Psychologist Association (OSPA)
Association of Behavior Analysis International (ABAI)
PROFESSIONAL VITA 7
REFERENCES:
Dr. Renee Hawkins
Associate Professor and Coordinator, School Psychology
University of Cincinnati, College of Education, Criminal Justice, and Human Services
516D, Dyer Hall
University of Cincinnati
Cincinnati, OH 45221
513-556-3342
Dr. Carrie Bunger
School Psychologist
Amelia Middle School & Amelia High School, West Clermont Local Schools
1341Clough Pike
Batavia, OH 45103
513-947-7520
Sarah English
School Psychologist
Summerside Elementary, West Clermont Local Schools
4639 Vermona Drive
Cincinnati, OH 45245
513-404-8804
SYSTEMS CHANGE 8
Systems Change: Grades 6-8 Implementation of NWEA MAP Assessment to Inform Reading
and Math Intervention
The principal of a rural middle school constructed a building-level data team for the
purpose of analyzing universal screening data in reading and math in order to inform decisions
regarding placement into an “Intervention” or “Enrichment” bell. The goal of the building-level
data team was to train the staff to use the Northwest Evaluation Association’s (NWEA)
Measures of Academic Progress (MAP) assessment data, DIBELS reading benchmark
assessment data, and student OAA scores in reading, to support the placement and movement of
students into groups providing Tier II support. The school had a history of using Response to
Intervention (RtI) in the school to support students academically. Historically, the school used
DIBELS ORF and DAZE assessments for reading and AIMSweb Concepts/Applications and
Computation assessments for math alone as a Universal Screening measure for each subject area.
Previous years’ OAA data demonstrated that the school had a need to utilize the additional
resource of the MAP assessment data. This utilization of additional universal screening data,
created a systems-change to better utilize data-based decision making, in turn better supporting
students academically.
Due to the new system of teacher evaluation based on student performance, required by
the Ohio Department of Education, the district had purchased the MAP assessment in order to
serve as a reliable measure of student growth and as a projection tool of Ohio Achievement
Assessment Proficiency. The target middle school was also using the MAP data as an RtI
Universal Screener. The MAP assessment was an online assessment
(http://www.nwea.org/node/98). Students used laptop computers to sign into the NWEA system
and take the assigned assessment. The assessment was given three times during the academic
year (fall, winter, and spring). Data from the assessment was compiled, sorted, and analyzed by
SYSTEMS CHANGE 9
the school psychologist and intern school psychologist. After sorting and analyzing the data
following each benchmark assessment, the data were shared with teachers in grade-level RtI data
meetings for further discussion and analysis. In addition to facilitating grade-level RtI data
meetings, the building-level data team offered several teacher trainings to provide instruction for
teachers on how to input their class lists and assign assessments to students through the NWEA
website, how to administer the MAP assessments, and how to analyze the data in order to inform
decision making.
Methods
Participants and roles
Building-Level Team. The middle school, located in a rural setting in Southwestern
Ohio, constructed a building-level team consisting of the principal, two school counselors,
school psychologist, and intern school psychologist to analyze available universal screening data
in order to inform sixth and seventh grade student placement into an “Intervention” or
“Enrichment” bell. The team met monthly to plan for the assessment periods, compile the
collected data, analyze the data, sort students into two groups, determine decision rules for
movement between groups, and set guidelines for progress monitoring procedures. The intern
school psychologist’s role within the team was to train staff on how to administer the MAP
assessments, provide troubleshooting for teachers when requested during the assessment periods,
sort the data to create “Intervention” and “Enrichment” intervention groups, and aide a sixth
grade team of teachers with understanding and analyzing their student reading data and assist in
making recommendations for student placement changes based on that data. The school
psychologist intern and the sixth grade team met after each of the three MAP assessment periods.
Staff. The staff members in grades six and seven participated in the administration of the
MAP assessments. Three teams of four general education teachers at both participating grade
SYSTEMS CHANGE 10
levels, as well as four special education teachers, participated in the administration of the MAP
assessments. Additionally, the school psychologist, intern school psychologist, and two school
counselors, participated in the administration of make-up assessments for students who were
absent during the initial administration of the assessments. The intern school psychologist
consulted more directly with one sixth grade team on analyzing MAP and other universal
screening data and making suggestions for student placement into or out of reading intervention
and reading enrichment classes.
Students. As per the 2012-2013 Ohio Department of Education Report card for this
middle school, there were 976 students enrolled in the school across three grade levels (sixth,
seventh, eighth). Of those students, 94.2% were Caucasian; 45.4% were economically
disadvantaged, 1.2% had limited English proficiency, and 13.6% were identified as having and
educational disability. The school was identified as having a medium-low poverty status. Only
students in grades six and seven participated in the MAP assessment at the middle school level.
MAP report data indicated that there were 312 students in the sixth grade and 308 students in the
seventh grade, for a total of 620 students between the two grade levels.
Setting.
A rural school in southwestern Ohio implemented the system-wide MAP assessment.
Implementation occurred three times during the 2013-2014 school year (September, January,
March).
Baseline.
Baseline data was obtained via two sources: 2012-2013 Ohio Achievement Assessment
(OAA) scores and Fall MAP data. The math and reading OAA scores for grades six and seven
were gathered from the Ohio Department of Education (ODE) school report card. Grade eight
SYSTEMS CHANGE 11
was not included because they were transitioning to the high school in the following school year.
See Table 1 below for OAA baseline data. MAP assessment data, collected in the fall also
provided baseline data. See Table 2 below for MAP baseline data.
Table 1. Ohio Achievement Assessment Results (2012-2013)
6th Grade 7th Grade
Reading 83.7% 80.0%
Math 75.2% 71.7%
A goal of achieving a 75% or higher passing rate on the Ohio Achievement Assessments
across all subject areas was set for the 2012-2013 school year. A goal of achieving and 80% or
higher passing rate on the OAAs was set for the 2013-2014 school year. Sixth grade students
during the 2012-2013 school year did meet the 75% or higher passing rate goal. However, with
an increase to an 80% or higher passing rate, the goal would not be met in mathematics if scores
did not increase in the 2013-2014 school year. Seventh grade students in the 2012-2013 school
year did meet the reading goal of 75% passing. On the seventh grade math assessment, student
scores did not meet the goal of 75% passing. Looking forward to the 2013-2014 school year, if
trends continued in a similar manner to the 2012-2013 school year, the seventh grade population
would not meet the math goal of 80% or higher passing rate.
School data, indicated on the ODE report card, revealed that 13.6% of the middle school
population (grades 6-8) was identified as a student with a disability. As a school using the RtI
model, the ideal percentage of students receiving intensive Tier III or special education support
should be less than 5%. A rate of 13.6% seen by the school indicates that there was an
overrepresentation of students in special education. With a higher-than-ideal rate of students
SYSTEMS CHANGE 12
being served with special education services, and with average OAA scores below the goal, the
school-based data team saw a need for additional Universal Screening measures to be
implemented to better identify students in need of additional supports.
Table 2. MAP Assessment Results (Fall 2013)
6th Grade 7th Grade
Reading 210 213
Math 214 216
The MAP assessment was said to be indicator of student progress and growth across the
school year as well as a projection of OAA proficiency. The MAP fall assessments for each
subject and grade level provided student projected OAA proficiency. With a goal of 80% or
higher proficiency for each subject area on the OAA, baseline fall MAP data indicated projected
deficits in 6th
grade math, 7th
grade reading and 7th
grade math. See Table 3 for MAP Projected
OAA Proficiency scores.
Table 3. MAP Projected OAA Proficiency
6th Grade 7th Grade
Reading 86.5% 78.5%
Math 68.9% 59.5%
Goals and decision rules
The goal of 100% participation for teachers in grades six and seven for the fall, winter,
and spring administration of MAP assessments was determined by the team. The building-level
team also set a goal for 100% of teachers to be independent in administering and troubleshooting
the assessments by the spring administration of the assessments. Additionally, the building-level
SYSTEMS CHANGE 13
team set the goal for teaching teams to use AIMSweb and DIBELS progress monitoring
assessments to further monitor students in the intervention bell in order to intensify or modify
instruction within the class.
A decision rule for assessment completion was set by the team and it was determined that
all core teachers within the sixth and seventh grade would participate in the universal screening
measure. Due to the time-consuming nature of the assessments, a schedule was created by the
team to determine which test to assess during the “assessment week” and during what times of
the day (See Appendix A). The intern school psychologist reviewed attendance data during
“assessment week” to create a list of students in need of make-up testing due to an absence. The
intern school psychologist also created testing sessions within the MAP system to allow all
students requiring a make-up assessment to test simultaneously; the make-up assessments were
then administered by the intern school psychologist. Additionally, decision rules for moving
students into or out of intervention bells were created. The intern school psychologist created a
decision rules guideline sheet to use in the grade-level data meetings. See Appendix B for
example of the reading decision rule guidelines. Teaching teams were asked to fill in the chart
with a student name and the available information under each column. Student data must have
met the criteria listed at the top of the sheet for at least 2 of the data sources in order for a change
in placement to occur. A sample of the goals for each data source/assessment area in sixth grade
reading and math can be seen in Appendix C.
Implementation Components
Universal Screening: Reading and Math. One crucial step in the success of systems-
change is the collection and use of data to monitor progress and inform future decision making
(Curtis, Castillo, & Cohen, 2010). The goal of universal screening is for the assessment data to
serve as early identification of children with academic problems (Elliot, Huai, Roach, 2007).
SYSTEMS CHANGE 14
Early identification of struggling students allows for prevention and intervention to be
implemented thus reducing the severity or ameliorating the academic concern (Elliot, Huai,
Roach, 2007). Universal screening serves as a prominent component of a problem-solving
service delivery model (Cook, Volpe, Livanis, 2010).
During the 2013-2014 school year, the school began utilizing the NWEA MAP
Mathematics and Reading assessments as universal screening measures during the fall, winter,
and spring. The assessments were given as a class and students each took the assessments using
a laptop computer at each benchmark period. Question difficulty depended on student responses;
a question answered correctly would elicit a more difficult question next. Assessments varied in
length and MAP provided national norms for each assessment as well as district norms, which
could be accessed through computer generated reports. Students’ overall scale scores were
reported by the “RIT Score” in each subject area. Percentile information from NWEA’s
nationally normed groups in each grade level was also provided. Additionally, projected
proficiency on the OAA was also provided from the fall benchmark MAP assessment score.
From the MAP assessment information, student’s scores could be compared to a national norm,
district norm, grade level school norm, and classroom norm. This Universal Screening
information allowed for an additional layer of identification for students struggling academically.
The intern school psychologist was provided with information on implementing the
assessment, troubleshooting the assessment, and analyzing data from the assessment from the
supervising school psychologist who had attended training on MAP administration. The intern
school psychologist participated with the school-level data team in distributing laptops to each
classroom, assisting teachers in administering the assessment online and troubleshooting
technology issues as they arose. The online system immediately scored student assessments and
SYSTEMS CHANGE 15
data was sent to a district representative who then dispersed the information to school
psychologists in the district. The intern school psychologist and school psychologist created
sorted Excel spreadsheets of student data for each grade level and subject area.
Teacher Training. The intern school psychologist held two teacher trainings; one in
September and one in January, both prior to the fall and winter assessment windows. The intern
school psychologist both created handouts and collected existing handout information on MAP
implementation, trouble-shooting technology issues, accessing MAP score reports, and analyzing
MAP data. A detailed script was provided to teachers on how to implement the assessment
(Appendix D) Handout information was reviewed with the teachers and a time for questions and
discussion was provided.
Grade-Level Data Meetings. The school-based data team held grade-level data
meetings following each benchmark assessment. The intern school psychologist printed Excel
worksheet handouts that included individual student MAP RIT scores, OAA scores, and DIBLES
or AIMSweb data. MAP generated report handouts of student data by grade-level and subject
area were also printed by the Intern School Psychologist. The intern school psychologist
provided this information to teachers at grade-level data meetings. The intern school
psychologist worked closely with one sixth grade team in analyzing their student data and
making decisions for intervention support placement based on the data.
Adherence data
Adherence checks for MAP assessment implementation were completed informally by
accessing classroom MAP reports on the NWEA website. It could be viewed which teachers had
created class rosters and began assessments. Classroom reports generated after the assessment
period confirmed that teachers had implemented the assessments during the assessment window
as expected.
SYSTEMS CHANGE 16
Social validity
Social validity information was gathered informally during grade-level data meetings.
Teachers were asked to share problems that they had encountered implementing the assessments
and accessing student scores. Teachers reported that the ease of implementation increased from
the fall to the winter implementation. Teachers were also asked whether they liked the math (Do
the Math) and reading (Corrective Reading) intervention materials. Teachers expressed that they
were pleased that the “Do the Math Series” aligned well with common core math standards set
by ODE.
Results
Two bells during the school day had been created for students to either receive support
through and “Enrichment” or “Intervention” bell. During the summer, prior to the start of
school, the school psychologist and school counselor reviewed OAA and AIMSweb or DIBELS
scores to inform placement decisions into either an “Enrichment” or “Intervention” bell. For
students who earned passing OAA scores and had met benchmark norms on AIMSweb or
DIBELS assessments were placed into “Enrichment” classes. “Enrichment” classes provided
students with opportunities to explore higher-level math or reading material to continue to grow
their scores on measures or reading or math. “Intervention” classes provided students who had
not passed the OAA and/or had scores significantly below AIMSweb or DIBELS norms an
opportunity for additional direct instruction, smaller group setting, and additional practice
opportunities with math or reading. Math intervention materials included the “Do the Math
Series” and reading intervention materials included Corrective Reading. Corrective Reading is a
scripted, explicit instruction reading program to remediate reading deficits (McDaniel, Houchins,
& Terry, 2011). Students receiving intervention support through an “Intervention” bell were
progress monitored weekly. Students participating in “Enrichment” classes were not progress
SYSTEMS CHANGE 17
monitored and universal screening using DIBELS and AIMSweb data was only collected at the
fall benchmark.
MAP assessments were administered by all sixth and seventh grade teachers at the fall,
winter, and spring benchmarks. Following the fall and winter administrations of the
assessments, the intern school psychologist and school psychologist used Excel spreadsheets to
sort the MAP assessment data. Figure 1 represents a sample from the sixth grade fall math
spreadsheet. Figure 2 represents a sample from the sixth grade fall winter spreadsheet.
Figure 1: Sixth Grade Fall Math Spreadsheet
Last
Name
First
Name ID TEST
Teacher
Name
Math
OAA
F Overall
RIT
196239 6 401 160
195274 6 370 170
203959 6 365 179
195003 6 387 195
197133 6 401 199
6 394 199
195425 6 394 204
196052 6 397 208
194994 6 397 209
195977 6 401 209
195075 6 401 210
195066 6 394 212
195484 6 404 212
198044 6 410 218
197255 6 401 219
195510 6 404 223
SYSTEMS CHANGE 18
Figure 2: Sixth Grade Winter Math Spreadsheet
Excel data sheets were created including student last name, first name, grade, teacher
name, the previous year’s OAA score, Fall Overall MAP RIT, and Winter Overall RIT.
AIMsweb and DIBELS data was also available via Excel Spreadseets. As data were collected at
each benchmark assessment, it was added to an Excel document with the previous benchmark
data. The intern school psychologist first sorted data by the previous year’s OAA score, color-
coating the scores by red, yellow, and green. Scores indicated in green were passing scores,
scores in yellow were those that fell within one standard deviation of the mean, and scores in red
were students whose scores fell below one standard deviation of the mean. The data were then
sorted by fall MAP score with the students with the lowest scores at the top of the sheet. Upon
receiving winter data, it was then added to the spreadsheet and sorted by lowest to highest score.
During the grade-level data meetings, data were shared with grade-level teams and
discussions began surrounding students who should be moved into or out of “Intervention” or
Last
Name
First
Name ID TEST
Teacher
Name
Math
OAA
F Overall
RIT
W
Overall
RIT
196052 6 397 208 199
197255 6 401 219 205
195003 6 387 195 207
195274 6 370 170 209
194994 6 397 209 210
195425 6 394 204 212
195075 6 401 210 213
195484 6 404 212 217
195977 6 401 209 220
195066 6 394 212 221
203959 6 365 179 222
6 394 199 223
198044 6 410 218 223
195510 6 404 223 226
196239 6 401 160
197133 6 401 199
SYSTEMS CHANGE 19
“Enrichment” skills groups. The team worked to identify students in the lowest 20% of each
grade-level in order to provide them with Tier II intervention support through the use of the
“Intervention” bell. To guide the discussion regarding student placement into intervention or
enrichment skills groups, a decision rule worksheet was provided (Appendix B). The worksheet
served as the teams as they recommended student placement changes. Changes in placement
between “Intervention” and “Enrichment” bells were only granted by the school-level data team
when teaching teams provided data supporting student through math or reading intervention.
Discussion
A need was recognized by a rural school in southeastern Ohio, to use more
comprehensive universal screening measures in order to inform decisions regarding students
receiving reading and math intervention support. Although no formal needs assessment form
was created and completed by administration and staff, the school-based data team determined a
need for additional universal screening measures and systematic decision rules for providing the
most appropriate supports to students. Newly created “Intervention” and “Enrichment” bells
provided the time for students showing a need, through data, for more intensive intervention
support in reading or math to receive additional support. Similarly, for students meeting state
criteria and benchmark norms on universal screening data were provided with the opportunity to
receive more challenging materials in reading or math to continue to grow their knowledge.
Reading and math universal screening and progress monitoring and OAA assessments had
occurred in previous academic years. However, the addition of math and reading “Intervention”
or Enrichment” bells and the use of MAP assessments were new to the middle school during the
represented school year. Due to concerns with the previous year’s OAA scores, the intern school
psychologist and other members of the school-level data team developed procedures for smooth
implementation of the MAP assessments and decision rules for how the data from the
SYSTEMS CHANGE 20
assessments, in combination with other universal screening data and OAA data, would be used to
inform decisions about students receiving additional reading or math support.
The intern school psychologist was responsible for creating materials and gathering
materials for teacher training on the MAP assessments, assisting with MAP assessment
implementation, creating and sorting Excel spreadsheets of student universal screening data, and
sharing this information with sixth and seventh grade teachers at grade-level data meetings.
Procedures for preparing for the MAP assessments, using the MAP assessments, and utilizing the
MAP data to inform decision-making were well implemented and were modified and completed
with higher levels of fluency and accuracy following each benchmark. The first year of
implementation showed success in that all teachers met the goal of implementing the
assessments. Upcoming Spring OAA data will provide more data to the school on the
effectiveness of using MAP as a projected indicator of OAA score and as its use to inform
decision-making regarding reading and math intervention. The Systems-Wide universal
screening MAP measure implemented for the first time in the middle school was an opportunity
for the school to link data to intervention. The interventions utilized were research-based and
provided by the district.
Limitations of this consultation include the use lack of formal social-validity measures by
teachers. More comprehensive data could have been gathered regarding teacher’s feelings about
the MAP assessments and the intervention materials if a structured social-validity questionnaire
or survey would have been provided. It would also have been interesting to have provided a
survey following each implementation regarding the ease of implementation of the MAP
assessments to better determine if procedures were implemented more fluently and with fewer
difficulties than during the fall implementation.
SYSTEMS CHANGE 21
An additional limitation included the district-mandated intervention materials. The
materials were research-based interventions; however, the school psychology intern did not gain
experience in intervention planning as the interventions were directed from administration rather
than decided upon at the building-level.
The lack of formal adherence checks is another limitation of this consultation.
Procedures for implementation of the MAP assessments were provided to teachers; however, no
formal adherence data was collected to ensure that procedures were followed as necessary.
This consultation provided the intern school psychologist the opportunity to serve as a
child advocate. Best practice is to utilize data-based decision making when intensifying or
removing supports from a child’s services. By utilizing data-based decision making when
making decisions on individual children, I was able to be an advocate for the child’s needs.
Also, because collaborative relationships are essential to effective service delivery, working in
collaboration with educators allowed me to assist in providing positive intervention support to
students (Kaniuka, 2009).
This consultation allowed me to share knowledge and educate school personnel on
evidence-based instruction and interventions through the teacher training and teacher
collaboration that occurred. This is a goal of my personal model of practice. Doing so allowed
me to take on a leadership/initiative role and use collaborative approaches as I worked with
others. As best practice calls for problem-solving teams to use data-based decision making
during RtI service delivery to increase child outcome measures, this consultation allowed me to
utilize my skills in problem-solving with a diverse team of individuals (Burns et al., 2010). The
use of the scientist-practitioner model allowed me to work with the team in order to work toward
the development of better reading and math intervention plans. This systems-level consultation
SYSTEMS CHANGE 22
allowed me to practice and improve existing skills as well as learn new skills that I will use as an
independent practitioner in the years to come.
SYSTEMS CHANGE 23
References
Curtis, M.J., Castillo, J.M., & Cohen, R.M. (2010). Best practices in systems-level change. In A.
Thomas & J. Grimes (Eds.), Best Practices in School Psychology III (pp. 887-901).
Bethesda, MD: National Association of School Psychologists.
Elliot, S.N., Huai, N., & Roach, A.T., (2007). Universal and early screening for educational
difficulties: Current and future approaches. Journal of School Psychology, 45, 137-161.
Kaniuka, M. (2009). Blueprint III: is the third time the charm?. Journal of educational and
psychological consultation, 19(3), 224-235. doi: 10.1080/10474410903106430
McDaniel, S.C., Houchins, D.E., & Terry, N.P. (2011). Corrective reading as a supplementary
curriculum for students with emotional and behavioral disorders. Journal of Emotional
and Behavioral Disorders, 21 (4), 240-249.
NWEA MAP website: http://www.nwea.org/node/98
Ohio Department of Education (2012). Student assessments for teacher and principal evaluation
service provider publicly-available service summary. Retrieved from:
http://education.ohio.gov/getattachment/Topics/Teaching/Educator-Evaluation-System/Student-
Growth-Measures/Approved-List-of-Assessments/NWEA-s-MAP-2-12-OH-
Application.pdf.aspx
SYSTEMS CHANGE 24
Appendix A
March
Monday Tuesday Wednesday Thursday Friday
17
6& 7th Reading
MAP in
Language Arts
Core Classes
18
6th & 7th-
Reading MAP in
Language Arts
Core Classes
19
6th & 7th Math
MAP in Math
Core Classes
20
6th & 7th-Math
MAP in Math
Core Classes
21
6th&7th-Makeups
in Math and
Language Arts
Classes
24
6th&7th-MAP Test
Makeups
25
6th&7th-MAP Test
Makeups
26
6th&7th-MAP Test
Makeups
27
6th&7th-MAP Test
Makeups
28 6th&7th-MAP Test
Makeups
1. Reading MAP will be given in Language Arts bells by the LA teacher. Math MAP will be given in Math bells by the math teacher. Teachers will use their team cart that day. 2. Special education teachers should meet with their team to determine best testing conditions for their students. 3. Barring technical issues, EIGHTH grade should be able to keep their computer carts. 4. Main Office has instructed us that we do not have to give the Language MAP this spring.
5. Please be mindful that Student Services will not be available all day to provide
troubleshooting. Please see your MAP team member before contacting the office and review your proctoring cheat sheets.
Assessment Schedule
Spring MAP
SYSTEMS CHANGE 25
Appendix B
Sixth Grade Math Goals
Fall Winter Spring
Math OAA
>400 >400 >400
MAP Math Overall Average RIT
220 223 226
AIMSweb Progress Monitoring
Computation: 20 Concepts & Applications:14
Computation: 28 Concepts & Applications:18
Computation: 34 Concepts & Applications: 20
Sixth Grade Reading Goals
Fall Winter Spring
Reading OAA
>400 >400 >400
MAP Reading Overall RIT
212 214 216
MAP Language Overall Average RIT
212 214 216
Fall Benchmark DORF 107 Words Correct 97% Accuracy
109 Words Correct 97% Accuracy
120 Words Correct 98% Accuracy
Fall Benchmark DAZE
18 19 21
SYSTEMS CHANGE 26
Appendix C
SYSTEMS CHANGE 27
Appendix D
MAP Administration 411
Entering the Site for Proctors
1. Open browser and enter https://westcler-admin.mapnwea.org/admin
2. Enter your User Name and Password; click “Submit”. You were sent your username
and password via email. The email is from NWEA. Check your junk mail if you do
not see it in your inbox.
3. If this is your first login, you may be asked to create a new password.
4. Write down your username and new password for reference.
Setting up a Test Session/Roster
Complete the following steps 1-5 prior to the testing day.
1. After logging in, click the “Manage Test Sessions” tab on the left.
2. Click “Find Students to Test.”
a. In the dropdown menu titled “School” choose “XXXX”
b. To find students, enter last name in the correct field, then click “Search.”
c. When screen pops up, make sure the correct student is checked, then click “Add
Student.”
i. If multiple student names pop up, make sure only the student that you
want is clicked.
3. To assign a test to all listed students, make sure that the button to the left of “Last Name”
is checked.
a. Once students are selected (either individually or as a class), click on “Assign
Test.”
b. You will see multiple test options. Do not choose the “Survey” option.
4. Choose the appropriate test from drop down menu and click “Assign.”
5. Click “Save Session,” give the session a unique name, then click “Save and Exit.”
To Start Testing- For Proctors
1. When ready for all students to take the test, log back in (see above) and click on “Manage
Test Sessions” tab.
2. Under “Saved Test Sessions” make sure that the appropriate test is checked, then click
“Test Now.” Write the session name and password for students on the board.
3. Follow Student Login procedures below.
Student Login
1. Students have to login to the chrome books using THIS login. It is specific to MAP. a. Username: [email protected]
b. Password: ammap2013
2. This login will direct the students to the MAP website and lockdown the browser,
preventing them from looking at other sites.
3. Once students have opened the secure browser, they will input the session name and
password. This is listed on the proctor’s screen after you clicked “Test Now.”
SYSTEMS CHANGE 28
After Student Login- For Proctors
4. After students log in, the student names will show up on the proctor’s screen as “To Be
Confirmed.” Click “Confirm Now” once all students are ready to be confirmed.
a. Students then click “Start Test” to begin.
5. To pause or suspend a student’s test (for bathroom break, etc.) click the drop down menu
next to “Select Status” on your screen.
a. Choose “Pause” if student will resume test in less than 30 minutes or “Suspend” if
student will be resuming test in more than 30 minutes.
b. When in doubt choose “Suspend”.
6. Click the box next to the student needing paused or suspended, then click “Go.”
7. To resume testing, change status to “Resume” for paused tests or “Test Again” for
suspended tests.
a. For suspended tests, students will be able to choose whether to continue with next
question or start over.
8. The test will automatically stop when needed for each student.
Relevant Links and Tabs Overview
Online Training—Click here to access information and training on MAP assessments.
-MAP Introduction—Five minute video on MAP assessments and how they can be
beneficial for your students.
-System Management—Use this link to access training on adding students and setting up
a testing window.
-Testing—Use this link for videos showing how to proctor a MAP assessment and how to
pause and resume student tests.
-Using Test Results—Use this link for training on interpreting MAP reports.
Quick References—Click here for printable resources, including student directions for MAP, and
a summary of available reports.
MAP Testing Warm Up—Click here for a practice test. Includes a video to watch with the whole
class and a link to complete the Warm Up together (students can also take the Warm Up
individually).
Manage Students—Click here to add students or view student profiles.
Manage Test Sessions—Click here to create a test session and choose students for that session.
See below for more information about test sessions.
View Reports & Instructional Resources—Click here to find reports and results of MAP
assessments.
-MAP Reports—Use this link to find results by student or class. You can also find goal
setting reports for individual students.
-Reports Reference (Under Information Center)—This document includes a
description of each available report.
-Student Quick Search—Quickly find results and reports for specific students.
TIER I ACADEMIC 29
A Tier I Academic Classwide Peer Tutoring Intervention for Increasing Oral Reading
Fluency in a Second Grade Classroom
An intervention was implemented to increase the reading fluency of 20 students in a
second grade classroom. Following the winter benchmark screening of the DIBELS DORF
assessment, a second grade teacher requested consultation to supplement the Tier I core
instruction in the classroom. The school utilized the “Response to Intervention” (RtI) model to
providing intervention support to students. The RTI framework would suggest that
approximately 80% of students should respond to the Tier I core curriculum and meet research
based benchmark goals, placing them in a “low risk” category. Approximately 15% of students
will require strategic Tier II support, placing them into a “some risk” category. Approximately
5% of students, scoring in the “at risk” category do not respond to Tier I and Tier II intervention,
requiring the need for Tier III intensive intervention. Nine of the students in the target classroom
scored in the “at risk” category on DORF assessment, three students scored in the “some risk”
category, and eight students scored in the “low risk” category. The student scores on the DORF
assessment indicated a need for additional intervention to provide fluency practice. The
intervention chosen to provide the additional fluency practice was the Classwide Peer Tutoring
(CWPT) intervention. The DIBELS DORF assessment was used to collect baseline and progress
monitoring data.
The CWPT intervention (Appendix 1) was implemented in a general education second
grade classroom, in a rural elementary school. All students in the class participated in the
intervention four days per week for fifteen to twenty minutes per session. Progress monitoring
data was collected on the fifth day. Points were collected through the intervention and
documented using the Classwide Peer Tutoring Point System (See Appendix B). The
intervention was conducted by the general education classroom teacher.
TIER I ACADEMIC 30
Data showed that showed overall increases in ORF scores. ORF levels during the
intervention phase did not however, reach the goal of meeting or exceeding the Benchmark 3
DORF goal of 87 words correct per minute.
Method
Participants and roles
A second grade general education teacher requested consultation for adding intervention
support for the students in her classroom. Universal screening data were collected in the
elementary school three times per year (fall, winter, spring) through DIBELS assessment.
DIBELS were created to identify students who have not yet mastered the acquisition of basic
early literacy skills (Kaminski, Cummings, Powell-Smith, & Good, 2011). Following the winter
implementation of DIBELS DORF universal screening measures and a grade-level data meeting
where student assessment scores were shared, the teacher saw a need for additional fluency
practice for the students. 47% of the students in the classroom had scored in the “need for
intensive support” range, 18% scored in the “need for strategic support” range, and 35% scored
in the “core-curriculum support” range. The data indicated a need for fluency practice to
increase the number of correct words per minute that students could read.
The consultation team consisted of the building principal, school psychologist, intern
school psychologist, and the participating second grade teacher. In collaboration with the
consultation team, the intern school psychologist determined that a CWPT intervention would
provide supplemental reading fluency practice for the students in the targeted classroom.
Permission was gathered from the classroom teacher and supervising school psychologist to
report the intervention results in the form of classwide oral reading fluency data.
TIER I ACADEMIC 31
Setting
The intervention was implemented in one- second grade general education classroom in a
rural elementary school. The intervention procedures were followed by the participating teacher
and students at their desks within the classroom. Students were asked to sit in pairs at desks.
Every Friday, progress monitoring data was collected by the teacher and entered into the online
data-storage website, VPort. The school psychology intern had access to the school VPort
account in order to view classroom data.
Target Variable
Reading Fluency. The target variable for this intervention was the participating
students’ oral reading fluency score (words correct per minute), as measured by the DIBELS
ORF assessment. Oral reading fluency is a necessary prerequisite skill to the ultimate goal of
reading, which is comprehension (Hofstadter-Duke & Daly, 2011). Research has shown that oral
reading fluency scores are predictive of reading comprehension standardized tests, especially in
primary grade levels (Shinn & Walker, 2010). Students with significant academic deficits often
face additional difficulties in later years including grade retention, school dropout, discipline
concerns, and placement into special education; all barriers which hider students’ access to
college and access to higher paying jobs (Maheady & Gard, 2010). Additionally, whereas
reading fluency activities alone have been found to have great effects on the oral reading fluency
scores of younger students, older students require more comprehensive intervention support
encompassing word study and comprehension (Shinn & Walker, 2010). This highlights the
importance of early intervention for struggling readers.
Inter-Scorer Agreement
Co-observations were completed by the school psychology intern while the classroom
teacher collected ORF data. Inter-scorer agreement was calculated for 20% of the ORF progress
TIER I ACADEMIC 32
monitoring probes completed during the baseline and for 33% of the ORF progress monitoring
probes conducted during the intervention phases. Inter-scorer agreement was calculated by
dividing the total number of agreements by the number of agreements plus disagreements.
Baseline inter-scorer agreement was taken on 3/05/14 and intervention inter-scorer agreement
was taken on 3/12/14. See Table 1 for inter-scorer agreement data.
Table 1
Inter-scorer Agreement Data
Date Inter-Scorer Agreement
3/05/14 100%
3/12/14 100%
Goals and Decision Rules
Goals and decision rules were determined in collaboration with the consultation team. It
was decided that the DIBELS DORF Benchmark 3 goal for the end of the year, 87, would be
used as the intervention goal. According to the second grade DIBELS benchmark goals,
between the fall administration to the spring administration, students were expected to gain 35
words correct per minute and score 87 words correct per minute or higher. The consultation team
decided that the decision rule for fading the intervention would be when the classroom average
was above the goal line on the DORF for three consecutive progress monitoring assessments. It
was also determined that the team would modify the intervention when the class scored three
consecutive data points below the aimline.
Functional Hypothesis
Teacher interviews and the review of universal screening data indicated that the below
average skills in ORF were due to skill deficits. It was hypothesized that by providing frequent
TIER I ACADEMIC 33
practice opportunities, feedback, and reinforcement through the intervention, an increase in
grade-level ORF, as indicated by universal screening assessments, would result.
Accountability Plan
An AB research design was used for this case. The baseline condition (A) included five,
averaged, ORF data points for a class of twenty-second grade students. The Classwide Peer
Tutoring intervention phase (B) consisted of peer tutoring sessions where students sat in pairs
and took turned reading a passage with fluency and accuracy as they received feedback and
verbal reinforcement from their partner. Internal validity threats can limit the use of an AB
design. The AB design however, is often used in practical settings because of the ethical
concerns that can arise when withdrawing an effective intervention (O’Neill et al., 2011).
Intervention Procedures
Baseline. The baseline data collected for this classwide intervention consisted of five
DORF data points. Each data point represented the class average DORF score for that week’s
DIBELS DORF assessment. The first two baseline points represented the fall Benchmark 1
assessment average student score and the class average winter Benchmark 2 assessment score,
respectively. The final three baseline data points consisted of the class average DORF scores for
the three weeks prior to the start of the intervention.
CWPT. Intervention can be described as modifications of the environment that are
planned in order to alter behavior as desired (Upah, 2010). The most beneficial interventions for
students are ones which provide practice opportunities, prompting, error correction, and
reinforcement for the targeted skill (Hofstadter-Duke & Daly, 2011). Additionally, peer-
mediated interventions have been found effective for increasing reading fluency scores
(Hofstadter-Duke & Daly, 2011). Reading failure can be remediated through the use of
collaborative learning groups such as CWPT (Marr, Algozzine, Nicholson, & Dugan, 2011).
TIER I ACADEMIC 34
Peer-mediated interventions such as CWPT are efficient for classroom teachers because peers
serve as tutors, freeing the teacher to monitor all students, and allowing more students to access
intervention (Hofstadter-Duke & Daly, 2011). Peer tutors can not only provide the intervention
to target a specific academic skill, but they also provide reinforcement in the form of peer
attention, as praise and encouragement are given for their partner’s effort (Hofstadter-Duke &
Daly, 2011).
In collaboration with the team, the intern school psychologist created a CWPT script for
the teacher to follow while implementing the intervention. The script provided necessary steps
to complete prior to the start of the intervention, during the first implementation of the
intervention, and for each day of implementation following the first. In this consultation, three
data points were collected during the intervention phase.
The CWPT intervention required the teacher to assign students into pairs and divide the
pairs into two teams each week. On each day of implementation, students moved into teams at
desks in the classroom. They each received a reading passage, which was placed inside of a see-
through plastic sleeve in a folder, as well as a dry-erase marker. Students chose who would
begin reading first and upon teacher prompting and the start of the timer, the first student would
begin reading orally the passage in front of them. While their partner read the passage, the
second student marked any errors onto the plastic sleeve. When the timer rang signifying the
end of the minute, partners totaled the number of errors and recorded the number of correct lines
that they read. Roles were then switched. Following both partnered reads, the teacher called on
each pair to report the number of combined points that they had earned for the day. The number
of points earned for each of the teams was then calculated and the daily winner announced.
TIER I ACADEMIC 35
Every Friday, weekly total points were calculated and the teacher provided verbal praise as well
as tangible items to the winning team.
The twenty students in the selected second grade classroom participated in the
intervention four days per week for fifteen to twenty minutes per session. Every Friday, progress
monitoring data was collected by the classroom teacher and recorded on VPort where it could be
accessed by the intern school psychologist.
Adherence data
The intern school psychologist collected adherence data during 16% of the intervention
implementation periods using the intervention script. See table 2 for adherence data.
Table 2
Intervention Adherence Data
Date CWPT
3/10/14 100%
3/24/14 100%
Social validity
All aspects of the intervention including target variable selection, goals and decision rules
selection, intervention selection, were decided upon in collaboration with the consultation team.
The consultation team met several times both before and after intervention implementation to
discuss intervention details and student progress. The teacher was also asked frequently, if any
modifications needed to be made to the intervention. Social validity was continually addressed
as frequent check-ins allowed for the intern school psychologist to get information from the
teacher as to the ease of use of the intervention, the amount of buy-in to the intervention that the
TIER I ACADEMIC 36
teacher held, and if the teacher would use the intervention again in the future. Social validity
information for the students was also informally collected through observations of the class
while implementing the intervention. When the points were read aloud following both partner
reads, student pairs provided praise to each other for high scores earned. When team winters
were announced, students cheered aloud and appeared to enjoy the competitive nature of the
CWPT intervention.
Results
Baseline and progress monitoring for the ORF score of the second grade class is
displayed in Figure 1 below. As the data in the figure depicts, overall positive trends were seen
in ORF data during the intervention phase.
Reading Fluency
Figure 1 displays the classwide average DORF scores collected from students in the
second grade classroom every Friday as well as comparison data from another second grade
classroom in the building. Baseline data showed an upward trend of increasing ORF scores for
the second grade classroom until the final baseline data point was collected. ORF levels during
the intervention phase further increased toward the end of the year goal and data showed overall
increases in ORF scores. ORF levels during the intervention phase did not however, reach the
goal of meeting or exceeding the Benchmark 3 DORF goal of 87 words correct per minute. It
can be seen that the comparison second grade class made gains above the aimline across all but
one DORF assessment. The comparison classroom had met the goal of 87 wcpm, two
consecutive times when this data was compiled. Due to the decision rule to meet to modify the
intervention if the students in the target classroom scored three consecutive data points below the
aimline, a meeting was scheduled with the intervention team to discuss next steps for modifying
the intervention.
TIER I ACADEMIC 37
Figure 1: Classwide Average DORF Scores
Summary statistics were calculated to support visual analysis. Summary statistics can be
viewed in Tables 3, 4, and 5 below. The percentage of non-overlapping data (PND) for the ORF
variable was determined by identifying how many data points did not overlap with the highest
baseline data point. The effect size for the ORF variable was calculated by dividing the
difference of the baseline mean and the last intervention phase mean by the standard deviation of
the baseline. A large effect size for ORF (1.20) was found with the percentage of non-
overlapping data being 100%. ORF scores were somewhat variable during the intervention
phase (SD=13.91). Although the class had not met the final goal of three data points above the
goal line, positive increases were being made as the intervention continued.
Overall, improvements were seen in the ORF scores of the second grade students in the
participating class. The participating teacher reported that she believed that the intervention was
TIER I ACADEMIC 38
helpful for increasing the number of words correct per minute that students could read and she
enjoyed the ease of intervention implementation. Using goal attainment scaling measures (where
‘0’ indicates no progress made toward the goal, ‘1’ indicates progress made toward the goal, and
‘2’ indicates the goal was met), the Tier I received a ‘1’ for the ORF variable. A follow-up ORF
data point will be collected during a maintenance phase to determine if the effects of the
intervention hold over time.
Table 3
Means and Standard Deviations of Graphed Variables (Baseline Phase)
Variable (Behavior) Average of ORF progress monitoring scores (Baseline Phase)
Number of baseline
data points
Mean Standard Deviation
ORF 5 55.82 13.91
Table 4
Means and Standard Deviations of Variables (Intervention Phase)
Variable (Behavior) Average of ORF progress monitoring scores (Intervention Phase)
Number of intervention
data points
Mean Standard Deviation
ORF 3 72.46 0.99
TIER I ACADEMIC 39
Table 5
Percent of Non-overlapping Data (PND), Goal Attainment Scaling (GAS), and Effect Size for
Graphed Variables
Student PND GAS Effect Size
ORF 100% 1 1.20
Discussion
Improving the ORF scores of the students in a second grade classroom, was an important
goal for this consultation. The classroom teacher requested consultation to add additional
intervention for her class of twenty students to increase their scores on ORF assessments. The
intervention was implemented with fidelity and overall improvements were seen in the classroom
ORF average.
Limitations did exist within this consultation. One limitation was the lack of stable
baseline data. Data showed that baseline measures of ORF were following an increasing trend.
This makes determining if positive increases seen in the ORF data are due to the effects of the
intervention or if they would have occurred without the intervention in place. Improvements
could also be made with the timeframe in which the intervention was implemented. With the
class average ORF score at levels below the aimline since the start of the school year, the CWPT
intervention could have begun at an earlier time to increase the gains made across the school year
for the students.
This consultation allowed for use of the scientist-practitioner model for intervention
planning and implementation. Using the data to guide intervention decisions, a target variable
was determined and defined, an intervention was chosen, and progress monitoring data were
TIER I ACADEMIC 40
collected. The consultation team used a problem-solving approach to develop the intervention
and measure growth toward the goal.
The consultation also allowed for collaboration between the intern school psychologist
and the participating teacher. Several meetings were held where the intern school psychologist
and teacher problem-solved together and discussed pros and cons of potential interventions. In
collaboration, the CWPT intervention was decided upon for use. The collaborative relationship
between the intern school psychologist and the teacher during the problem-solving process,
facilitated teacher buy-in toward the intervention. The use of the scientist-practitioner model and
use of collaboration are key components for my personal model of practice during consultation.
This case provided an excellent opportunity to increase skills to aide in my future practice as a
school psychologist.
TIER I ACADEMIC 41
References
Hofstadter-Duke, K.L. & Daly, E.J. (2011). Improving oral reading fluency with a peer-mediated
intervention. Journal of Applied Behavior Analysis, 44 (3), 641-646.
Kaminski, R., Cummings, K.D., Powell-Smith, K.A., & Good, R.H. (2010). Best practices in
using dynamic indicators of basic early literacy skills for formative assessment and
evaluation. In A.Thomas & J. Grimes (Eds.), Best Practices in School Psychology IV (pp.
1181-1204). Bethesda, MD: National Association of School Psychologists.
Maheady, L. & Gard, J (2010). Classwide peer tutoring: Practice, theory, research, and personal
narrative. Intervention in School and Clinic, 46 (2), 71-78.
Marr, M.B., Algozzine, B., Nicholson, K., & Dugan, K.K., (2011). Building oral reading
fluency with peer coaching. Remedial and Special Education, 32 (3), 256-264.
Shinn, M.R., & Walker, H.M. (Eds.). (2010). Interventions for achievement and behavior
problems in a three-tier model including RTI. Bethesda, MD: National Association of
School Psychologists.
Upah, K.R.F. (2010) Best practices in designing, implementing, and evaluating quality
interventions. In A.Thomas & J. Grimes (Eds.), Best Practices in School Psychology II
(pp. 209-223). Bethesda, MD: National Association of School Psychologists.
TIER I ACADEMIC 42
Appendix 1
Classwide Peer Tutoring (CWPT) Script
CWPT Combines:
1. Feedback and reinforcement for improved performance
2. Opportunities to respond
3. Reading fluency practice
Frequency/Duration:
4 days/week, on the 5th
day ORF progress monitoring is collected.
Before Peer Tutoring Session Begins:
Step 1: Use ORF data to determine students’ highest instructional level
Step 2: Divide class into two teams each week
Step 3: Within each team assign students to dyads based on CBM results; students in each dyad
should be reading from the same material
During First Peer Tutoring Session:
Step 1: Introduce Class wide Peer Tutoring to the class
Teach students about the purpose of the model (e.g. “to help each other learn”)
Explain that the students will be working with a partner, taking turns being the “tutor”
and “tutee”
Explain that the students will be divided into “teams” and that each week the team with
the most points will be declared the winter
Step 2: Conduct training sessions: Demonstrate how peer tutoring will look by going through a
lesson with another teacher or student as the tutee and “modeling” appropriate tutor and
tutee behavior
Demonstrate getting into pairs quickly and quietly
Demonstrate the feedback/error correction procedure
o Tutee reads aloud
o If tutee reads all words correctly:
Tutor adds a checkmark to the end of each sentence/ line read correctly
Tutor switches roles and becomes the tutee
o If tutee reads a word incorrectly:
Tutor corrects errors (ie. Correctly pronouncing the misread word)
Demonstrate the reward system
o As mentioned above, tutor awards points for correct sentences read
o Teacher can award points for following tutoring procedure and working
cooperatively
o At the end of the tutoring session, students tally the number of points they earned
Demonstrate system to request help
o Example: Tutor and tutee are to raise their hands if they have a question
TIER I ACADEMIC 43
Step 3: Have children role-play with other children in the class as the teacher did
o Provide feedback to children as they practice
Every Monday:
Step 1: Pair tutoring partners
Example: Rank students by achievement on ORF measures (high to low), divide the list
in two, and pair the first student in List 1 with the first student in List 2, and so on
Step 2: Divide students into two competing teams
Step 3: Create partner chart and post in a visible area of the classroom
Each Day That the Intervention is Implemented
Step 1: Tell/remind students to check the partner chart for their partner assignment
Step 2: Instruct students to move to their partners
Step 3: Give students materials to use.
Example: Reading passage
Step 4: Set timer for half the length of the tutoring session
Example: If tutoring session is 30 minutes, set timer for 15 minutes
Step 5: Monitor students while they tutor each other
Provide feedback to students
Award bonus points to pairs that are working cooperatively, implementing procedures
accurately, and reading words that the tutees are unable to decode.
Answer students’ questions
Step 6: When timer goes off, tell students to switch roles and continue until the end of the
tutoring period
Step 7: At the end of the session, ask students to record their total points for the day on their
partner chart. Teacher should then announce which team is in the lead for the day.
Every Friday:
Step 1: Announce the winning team for the week
Provide praise
TIER I ACADEMIC 44
Appendix B
Classwide Peer Tutoring Point System
Team 1
Monday Tuesday Wednesday Thursday Friday Weekly Total
Sara & Kelly 5 6 5 4 3 23
Matt & Braden 2 2 3 1 5 13
Team 1 Total Points
36
Team 2
Monday Tuesday Wednesday Thursday Friday Weekly Total
Team 1 Total Points
TIER I BEHAVIOR 45
A Tier I Behavior Intervention: Educating Ninth Grade Teachers on Positive Behavior
Practices to Decrease Classroom Discipline Referrals
The assistant principal of a rural high school sought consultation from the school
psychologist and intern school psychologist for concerns for the high number of office discipline
referrals in the high school. After reviewing the school’s discipline data for the previous months,
it was noted that the number of office discipline referrals were at a high rate. A team was
created, composed of the high school principal, assistant principal, school psychologist, and
intern school psychologist. The team met monthly to review discipline data and document trends
in the overall high school data as well within grade-levels. After collecting and analyzing
several months’ discipline data and establishing trends within the data, it was determined that the
highest number of referrals were seen in the ninth grade, and were made in the classroom. The
high number of referrals indicated a need for Tier I core behavior support in the classroom.
Informal observations conducted in several ninth grade classrooms and in the hallway
highlighted the lack of consistent, universal procedures for explicitly teaching school rules or
expectations for what the rules looked like within each room of the school. Further, there was
also a lack of reinforcement provided for students to perform the appropriate behaviors expected
of them.
A “High School Behavior Flow Chart” (See Appendix A) document was created by the
intern school psychologist as a reference for teachers to determine the student behaviors that
should be managed in the classroom versus office-managed. A second document, a “High
School Refocus Form” (See Appendix B), was created by the intern school psychologist to be
used by teachers when correcting teacher-managed behaviors. A professional development
forum was then created and presented to ninth grade teachers. Recent months’ ninth grade
TIER I BEHAVIOR 46
discipline data was shared as well as Positive Behavior Support (PBS) strategies, the “High
School Behavior Flow Chart”, and “High School Refocus Form” documents.
The intern school psychologist conducted the discipline data forum, with support from
the assistant principal. Pre and post tests were given to ninth grade teachers present at the forum
to assess their knowledge of effective PBS strategies, their confidence with using PBS strategies,
knowledge of the high school’s use of discipline data, knowledge of the high school’s PBS plan,
as well as their opinion on the helpfulness of the forum. Teachers indicated that following the
discipline data forum, they had a stronger understanding of effective PBS strategies, better
understood how discipline data was collected and analyzed in the high school, could list more of
the current school rules, felt more confident in determining “teacher-managed” vs. “office-
managed” student behaviors, felt an increased level of collaboration with other teachers in
developing consistent discipline procedures across the grade-level, and learned new information
that they could use in the classroom setting. Target variables for this consultation included
teacher knowledge and use of appropriate school discipline procedures and effective PBS
strategies and ninth grade classroom discipline referrals reported. It was hypothesized that
teacher education of PBS strategies and high school discipline procedures, coupled with the use
of the “Behavior Flow Chart” and “Refocus Form”, would contribute to a decrease in classroom
discipline referrals in the ninth grade. Results showed an increase in teacher knowledge of PBS
strategies and high school discipline procedures and a decrease in office referrals, indicating that
the goals of this Tier I consultation were met. This consultation served only as a Tier I case
versus systems-wide as only the ninth grade teachers were consulted with at this time regarding
the full implementation of PBS strategies. In the future, the school is interested in expanding
PBS strategies into full School Wide PBS (SWPBS).
TIER I BEHAVIOR 47
Method
Participants and roles
The assistant principal contacted the school psychologist requesting consultation for
analyzing the collected high school discipline data. A discipline data team was created in
September 2013 and included the high school principal, assistant principal, school psychologist,
and intern school psychologist. The assistant principal was serving her first year in the building.
The principal, for the first time in several years, was solely in the high school building; in
previous years he had split time between the middle and high schools. Upon analyzing the
discipline data collected, it was determined that the majority of the office discipline referrals for
problem behavior were being made by ninth grade teachers from their classrooms. The
discipline data team decided to consult with the ninth grade teachers to educate them on
research-based PBS practices and share their grade-level discipline data in order to decrease the
number of students being referred for administrator-level disciplinary action. Sixteen teachers,
the school counselor, and assistant principal attended the “9th
Grade Discipline Data Forum” led
by the intern school psychologist on a professional development day in early February. The
sixteen teachers in attendance consisted of mathematics, English, history, science, and Spanish
teachers who taught ninth graders. The intern school psychologist explained the recent trends
seen in the high school discipline data, shared documents relating to discipline referrals, shared
PBS strategies to implement in the classroom, and facilitated discussion on ideas for
improvements, areas of strength, and current teacher practices that were shared. Following the
“Ninth Grade Discipline Data Forum”, ninth grade teachers began utilizing the “High School
Refocus Form” and the “High School Behavior Flow Chart”. Special education teachers for the
ninth grade were invited to attend the forum but were required to attend a special education-
focused professional development session that was occurring at the same time. The two ninth
TIER I BEHAVIOR 48
grade special education teachers were consulted with individually following the forum and were
delivered the information discussed and shared at the forum. The ninth grade consisted of
approximately 120 students.
Setting
The “9th
Grade Discipline Forum” took place in the classroom of a ninth grade teacher.
The session took place for one hour and fifteen minutes on a Friday afternoon during a
professional development training day. The use of the “High School Refocus Form” took place
in the classrooms of ninth grade teachers in a private area. Teachers could choose to speak with
students regarding student problem behavior at the teacher’s desk or in the hallway.
Target Variables and measurement
Target variables for this consultation included teacher knowledge and use of appropriate
school discipline procedures and effective PBS strategies and ninth grade classroom discipline
referrals reported.
Teacher knowledge and use of school discipline procedures and PBS Strategies. This variable
was chosen in collaboration with the discipline data team as it was important to administration to
measure how much teachers knew about their school’s discipline procedures. Additionally, it
was important for the team to see increases in teacher knowledge following the professional
development session on the topic. The implementation of PBS strategies require that direct
teaching of school rules and positive reinforcement for students for following the school rules
occur (Caldarella, Shatzer, Gray, Young, & Young, 2011). Following the direct teaching of
school rules and positive reinforcement of those rules, teachers must have knowledge of the
school discipline procedures for taking next steps towards correcting problem behaviors when
rules are not followed. PBS research indicates that the occurrence of problem behavior should
be attended to quickly and in a consistent manner (McKevitt & Braaksma, 2011). A high
TIER I BEHAVIOR 49
number of discipline referrals were being made to administration that were minor in nature and
expected to be managed by teachers in the classroom. It was desired for teachers to increase
their knowledge on what school policy stated that constituted a teacher vs office-managed
student behavior in order to begin following the procedures for managing more behaviors within
their classroom.
School-wide positive behavior support (SWPBS) is a framework for providing
intervention practices which increase the social culture, educational environment, and student
behavior (Fallon, McCarthy, & Sanetti, 2014). When SWPBS is implemented in schools,
improvements in the behavior and academic achievement of students and positive increases in
staff perceptions of school safety is seen (Fallon, McCarthy, & Sanetti, 2014). Due to the
importance of teacher implementation of PBS strategies, teacher knowledge of effective PBS
strategies and their current use of those strategies in their classroom was an important target for
the team. One key feature of PBS is the focus on changing the environment within the school
versus changing the people in the school (Horner, 2000). For Positive Behavior Supports to be
effective within a school, teacher buy-in is a necessary element. When teachers feel invested in
the PBS strategies, they are more likely to implement the strategies them with fidelity
contributing to a change in environment and thus lower levels of student problem behavior.
Classroom Discipline Referrals (9th
Grade). The high number of office discipline
referrals being made from ninth grade teachers could be due to number possible explanations.
One explanation was that the ninth grade students displayed a higher number of problem
behaviors in the classroom due to lack of reinforcement for displaying appropriate behaviors. A
second explanation for increased problem behaviors in the ninth grade could have been a lack of
explicit teaching of school rules and expectations across settings within the school. Another
TIER I BEHAVIOR 50
explanation of the high number of ninth grade student referrals could have been due to a lack of
teacher clarification and knowledge on what constitutes off-managed versus teacher-managed
student behavior and the lack of an effective behavior management system in the classroom,
causing an inflated number of referrals in the grade level. The discipline data team discussed the
listed possible explanations for the increased problem behaviors displayed by ninth grade
students and determined that the increased office discipline referrals were likely a combination
of the above-listed explanations. Office discipline referrals served as a meaningful target
behavior because of the strong correlation between behavior in the classroom and academic
achievement (Elliot, Huai, & Roach, 2007). Research has shown that when PBS strategies are
implemented, less time is used for addressing problem behaviors and additional time is allowed
for instruction and administration (McKevitt & Braaksma, 2011).
Pre and Post Survey Rating Scale. Teachers that participated in the Ninth Grade
Discipline Data Forum completed a pre and post survey rating scale to indicate changes in
knowledge following the forum (See Appendix C and D). The intern school psychologist
created an evaluation survey containing six questions to measure the target variable “Teacher
knowledge and use of school discipline procedures and PBS Strategies”. The evaluation survey
contained a rating scale from 1-5. A rating of one represented “Strongly Agree”, a rating of two
represented “Agree”, a rating of three represented “Not Sure”, a rating of four represented
“Disagree”, and a rating of five represented “Strongly Disagree”. A lower rating from pre-forum
to post-forum was desired it indicated a stronger agreement toward the positively stated
comments.
Excel Worksheet Recording System. The school utilized a district-created Excel
workbook to input all school discipline referrals into. Data were sorted by grade level and
TIER I BEHAVIOR 51
discipline codes and were created to categorize specific measurable, observable behaviors. Data
were further sorted by location and time. The ninth grade classroom discipline referrals variable
was measured through collecting the number of classroom discipline referrals per month for each
grade and inputting them into the Excel Worksheet. Data were accessed and graphed by the
intern school psychologist.
Goals and Decision Rules
Based on the baseline ninth grade classroom discipline referral data collected, the team
collaboratively determined that the target concern was the number of office discipline referrals
made by ninth grade teachers. The goal was set to reduce the number of office discipline
referrals made by teachers to less than one classroom discipline referral per day as this goal was
socially valid and meaningful for the high school administrators and teachers. Additionally, the
goal of increasing teacher knowledge as displayed through pre and post survey data was created.
Functional Hypothesis
Teacher interview information suggested that students were displaying a high number of
problem behaviors due to multiple functions. It was proposed that some students were
displaying problem behavior due to skill deficits because they had not been explicitly taught the
expected school behaviors and expectations. Observations in the ninth grade classrooms
suggested that few teachers implemented a reinforcement system in their classroom for
performing expected academic tasks or appropriate behaviors. It was hypothesized that the
function of other students’ behavior was to gain access to attention or to escape a task. The
discipline data team noted that when students acted out, they often were called into the hallway
to speak one on one where they received adult attention and escape from in-class tasks. Students
also received peer attention as the problem behavior elicited peer attention. Students were often
TIER I BEHAVIOR 52
sent to speak with administrators following problem behavior; therefore, they were receiving
additional attention and escape.
Intervention procedures
Baseline. Baseline discipline data on ninth grade office discipline referrals from the
classroom consisted of the average number of classroom discipline referrals per day during the
months of September, October, November, December, and January. Pre-forum survey data were
also used as baseline data to compare to post-forum survey data.
Discipline Data Forum. The Ninth Grade Discipline Data Forum was presented to
sixteen ninth grade teachers during a professional development day in early February.
Administrators were concerned with the increased number of classroom referrals seen in the
ninth grade compared to other grades within the rural high school. A discipline data team was
created in order to begin analyzing data and determining how best to begin implementing PBS
strategies within the school. The implantation of PBS strategies requires a leadership team
which analyzes school data supporting staff as they implement the strategies (Flannery, Frank,
Kato, & Fenning, 2013). The forum was provided in order to educate teachers on effective PBS
strategies, educate them on the current school rules, inform them of current discipline reporting
procedures in the high school, share grade-wide discipline data, and provide Refocus Form and
Behavior Flow Chart documents. It has been noted that school psychologists are well trained to
assist in building and maintaining the capacity of systems to improve the behavioral, emotional,
and academic outcomes of students (McKevitt & Braaksma, 2011). In this way, the intern
school psychologist created and presented the Ninth Grade Discipline Forum in order to assist in
improving the systems for behavioral outcomes for students, in turn affecting the academic
outcomes.
TIER I BEHAVIOR 53
The intern school psychologist created a Prezi presentation which was displayed on the
whiteboard. See Appendix E for a sample of the presentation. First, a quick review of PBS
strategies was completed. The group then discussed current discipline procedures within the
school including the school rules. The intern school psychologist then reviewed the year’s
graphed ninth grade discipline data. The group discussed the patterns seen in the data, why the
team believed that the patterns existed, and ways to implement PBS strategies to reverse the
patterns. A Behavior Flow Chart Form and Refocus form were then discussed.
Behavior Flow Chart. A behavior flow chart was created by the intern school
psychologist, in collaboration with administrators, which guided teachers in the school’s
expectation for providing discipline for problem behavior. Administration saw a need to define
for teachers what constituted a “teacher-managed” versus and “office/administration-managed”
behavior. The classification of these behaviors was socially valid as teachers first sorted
behaviors into the categories which they believed were best. The categories were then reviewed
to the administrator who made further edits. Best practice in PBS requires defined, explicit steps
to follow when implementing PBS strategies (McKevitt & Braacksma, 2011). Further, it
requires that staff buy-in be created in order attain the highest level of implementation (McKevitt
& Braacksma, 2011). For these reasons, the Behavior Flow Chart was created to define what
constitutes a “teacher-managed” versus and “office-managed” behavior and was reviewed by
teachers for agreement for the categorization of these behaviors. See Appendix A for Behavior
Flow Chart.
Refocus Form. A “Refocus” form was created as a means of meeting with a student
following a behavioral incident and discussing the behavior and changes that could be made the
next time before a student displayed a problem behavior. Due to the high number of minor
TIER I BEHAVIOR 54
classroom behavioral incidents being reported as discipline referrals to administration, the
refocus form served as means of teacher management of minor problem behavior. The form was
to be given to a student following a teacher-managed behavioral incident (as indicated on the
Behavior Flow Chart) to complete. Upon student completion, the teacher was to discuss the
student responses on the form and follow the actions stated on the Behavior Flow Chart. If it
were a first offense, the teacher would provide a verbal warning to a student and mail a copy of
the refocus form home to the parent. A second offence would be documented using a refocus
form then a phone call and mailed refocus form to the parent. A third offense would be mailed
home to the parent then documented as a discipline referral to be handled by administrators. All
forms completed were copied and saved to serve as documentation of teacher contact with
parents and as documentation of following the expected procedures for making an office referral.
See Appendix B for the Refocus Form. It was hypothesized that through teacher use of the
Behavior Flow Chart and Refocus form, and through teacher education on PBS strategies and the
high school’s discipline procedures, that a decrease would be seen in ninth grade classroom
discipline referrals due to increased teacher-management and following proper procedures for
documenting problem behavior.
Adherence data
A sample of 20% of the participating teachers were asked in the week following the
discipline data forum, whether or not they were utilizing the “Refocus Form” when problem
behaviors occurred. Of the four teachers asked, all four stated that they had used the “Refocus
Form” for managing problem behavior.
Social validity
Social validity was assessed through five questions presented on the Ninth Grade
Discipline Forum Post-Test. Teacher ratings were averaged and rounded to correlate with the
TIER I BEHAVIOR 55
rating scale of “Strongly Agree” to “Strongly Disagree”. All questions, when averaged and
rounded, indicated a rating of “Agree”. Agreement with the five positively stated questions
indicated that the forum was socially valid to the teachers. Additionally, a question about
session length was added. 50% of the participating teachers indicated that the session length was
“Just Right” and 50% of the participating teachers indicated that they session length was “Too
Short”. The belief that the forum was too short was seen as a positive statement indicating that
the teachers desired to discuss the topic of PBS further and learn more about strategies to
increase the implementation of PBS in the school. PBS research suggests that a minimum of
80%staff-buy in be obtained before implementing PBS procedures (McKevitt & Braaksma,
2011). Although it would have been more desirable to use a measurement system that provided
an outcome for staff buy-in, the questions asked on the post-survey allowed the team to get an
informal assessment of staff buy in of PBS procedures. See Table 1 below.
Table 1
Average response data for social validity questions
Strongly Agree
Agree Not Sure Disagree Strongly Disagree
Average Scoring
7. I learned something new today
1 2 3 4 5 2.3
8. I will use the information that I learned today
1 2 3 4 5 1.8
9. It was easy to talk with the group to ask questions
1 2 3 4 5 1.7
10. The handouts were helpful
1 2 3 4 5 2
11. This session was… Too Short
Too Long Just
Right
50% TS 50% JR
TIER I BEHAVIOR 56
Results
The Ninth Grade Discipline Forum has 16 teachers in attendance. Teacher engagement
during the forum was high. The teachers presented many questions to the intern school
psychologist regarding the forms given and their use. The teachers also presented many
concerns and questions to the assistant principal who was in attendance.
Results of the Ninth Grade Discipline Data Forum can be determined through group pre-
survey data and post-survey data as well as through office referrals written from the classroom.
Results are displayed in Figure 1 and 2. To support visual analysis, summary statistics were
calculated and are displayed in Tables 2, 3, and 4.
The baseline phase means and standard deviations for the “Average Classroom Discipline
Referrals per day” and “Teacher knowledge and use of school discipline procedures and PBS
strategies” variables are displayed in Table 2. The intervention phase means and standard
deviations of the variables are displayed in Table 3. The percent of non-overlapping data, goal
attainment scaling, and effect size for the target variables can be viewed in Table 4. The
percentage of non-overlapping data (PND) for each variable was calculated by identifying how
many intervention data points did not overlap with the highest baseline data point (the proportion
of non-overlapping intervention data points to the total number of intervention data points). The
effect size was calculated by dividing the difference of the baseline and intervention means by
the standard deviation of the baseline. Effect sizes which were calculated as being negative were
indicated as positive as the goal was to obtain a lower score rather than a higher score. Goal
attainment scaling was used to indicate (“0” corresponds with no progress towards the goal, “1”
corresponds with progress towards the goal, and “2” indicates that the goal was fully met)
growth toward the final goal. Goal Attainment Scale scores for the variables are displayed in
Table 4.
TIER I BEHAVIOR 57
The pre and post forum survey required the teachers to rate their beliefs and use of PBS
strategies and of school discipline data procedures. Teacher rating scales were given before and
after the forum. Teacher ratings were converted to a 1 to 5 scale with a rating of 1 indicating
“Strongly Agree” and a rating of 2 being “Strongly Disagree”. Figure 1 shows the results of the
pre and post-forum surveys and are displayed as an average rating across questions. Teacher
ratings decreased for all questions, which indicate that they more strongly agree with the
positively stated questions. Teachers indicated that they have a stronger understanding of
effective PBS strategies, better understand how discipline data is collected and analyzed in the
high school, and feel more confident in their knowledge of classroom behavioral management
strategies. On the pre-forum survey the teachers rated themselves as “agreeing” or being “not
sure” on the measures on beliefs and use of PBS strategies and school discipline procedures with
a mean of 2.42 (SD=0.82). The average teacher rating on measures of beliefs and use of PBS
strategies and school discipline procedures was 1.82 (SD=0.48) on post-forum survey data. The
intervention of the “Ninth Grade Discipline Data Forum” to increase knowledge and use of PBS
strategies and of school discipline data procedures showed a moderately large effect size. The
goal of increasing teacher knowledge and use of PBS strategies and high school discipline
procedures was met. Visual analysis shows a stronger agreement from pre-survey post-survey
data.
TIER I BEHAVIOR 58
Figure 1: Ninth grade pre-forum survey and post-forum survey data
Figure 2 displays the results of average number of classroom referrals per day for the
ninth grade and for a comparison grade, tenth grade. Pre-forum discipline data showed average
classroom referrals per day in the ninth grade as 1.52 (SD=0.63). The average for classroom
referrals per day in the ninth grade was 0.76 on the post-forum survey. The intervention of the
“Ninth Grade Discipline Forum” showed a large effect size. The goal of maintaining less than
one classroom referral per day in the ninth grade was met one time following the discipline
forum. Further collection of monthly data will determine if the goal is met three consecutive
times.
TIER I BEHAVIOR 59
Figure 2: Ninth Grade vs. Tenth Grade Classroom Referrals per Day
Table 2
Means and Standard Deviations of Graphed Variables (Baseline Phase)
Variable (Behavior) Percentage of observed intervals displaying the behavior (Baseline Phase)
Number of baseline
data points
Mean Standard Deviation
Average Classroom
Discipline Referrals
per day
5 1.52 .63
Teacher knowledge
and use of school
discipline procedures
and PBS strategies
1 2.42 0.82
TIER I BEHAVIOR 60
Table 3
Means and Standard Deviations of Graphed Variables (Intervention Phase)
Variable (Behavior) Percentage of observed intervals displaying the behavior (Intervention Phase)
Number of intervention
data points
Mean Standard Deviation
Average Classroom
Discipline Referrals
per day
1 0.76 N/A
Teacher knowledge
and use of school
discipline procedures
and PBS strategies
1 1.82 0.48
Table 4
Percent of Non-overlapping Data (PND), Goal Attainment Scaling (GAS), and Effect Size for
Graphed Variables
Variables PND GAS Effect Size
Average Classroom
Discipline Referrals
per day
100 +1 1.21
Teacher knowledge
and use of school
discipline procedures
and PBS strategies
N/A +2 .73
Discussion
Interestingly, the most commonly discussed topic during the forum discussion time was
the desire for the high school to return to using teaming. This system was used in the past;
however, due a decrease in staff because of budget cuts, the number of teachers was not even for
teaming to work best. Best practice states that it is recommended for teams to monitor the
implementation of PBS because teams are able to collaborate and problem solve in order to make
the best decision (McKevitt, B.C. & Braaksma, A.D., 2011).
TIER I BEHAVIOR 61
Limitations of this consultation include the small number of intervention/post-forum
discipline data collected. The data had not yet been run by the administrator for months
following the month displayed. Data from March and April would strengthen the argument that
the Ninth Grade Discipline Forum contributed to changes in overall discipline referral numbers
due to better use of PBS strategies and use of the Refocus form and Discipline Flow Chart.
A second limitation was the lack of adherence data collected following the discipline
forum. Teachers were told at the forum by the assistant principal that they must begin using the
Refocus Forms immediately in their classrooms when dealing with a problem behavior. The
teachers were emailed an electronic copy of the Refocus Form and told by the principal that its
use should occur immediately. Formal adherence data however, was not collected on teacher use
of the form. Informal adherence was collected by asking 20% of the teachers if they were using
the forms. In the future, I will create a formal plan with the team for adherence data to be
collected in order to insure that implementation of the discussed strategies is occurring.
Another limitation of this consultation was that only the ninth grade was included in the
professional development forum. Administrators desired to “pilot” the forum with ninth grade
due to data showing the highest level of need there before including tenth, eleventh, and twelfth
grades. With all grade levels participating, overall discipline data could be viewed to determine
overall changes in data. Despite the limitations of this consultation, positive results were seen
both in post-forum survey data and in ninth grade classroom discipline data.
This consultation provided the intern school psychologist an excellent opportunity for
collaboration with administrators and teachers in creating documentation for increasing PBS
strategies in the classroom in order to decrease overall ninth grade classroom discipline referrals.
The opportunity to work with a grade-level versus the entire school allowed for use of making a
TIER I BEHAVIOR 62
small change in one area of a school and then making additional changes once that change is in
place. As stated in my model of professional practice I wish to use problem solving in my
practice as a school psychologist. Early in the school year, the team saw a problem with the high
numbers of classroom discipline referrals coming from the ninth grade. The team utilized
problem solving to determine the best next steps for ameliorating the problem. The intern school
psychologist also utilized data based decision making with the implementation of strategies
based on data showing high numbers of discipline referrals in the ninth grade classroom. A link
was created between data and intervention which is best practice. This consultation was a
learning opportunity for Tier I strategies and plans into action for creating overall change in a
grade level.
TIER I BEHAVIOR 63
References
Caldarella, P., Shatzer, R.H., Gray, K.M., Young, K.R., & Young, E.L. (2011). The effects of
school-wide positive behavior support on middle school climate and student outcomes.
RMLE Online, 35 (4), 1-14.
Elliot, S.N., Huai, N., & Roach, A.T., (2007). Universal and early screening for educational
difficulties: Current and future approaches. Journal of School Psychology, 45, 137-161.
Fallon, L.M., McCarthy, S.R., & Hagermoser Sanetti, L.M., (2014). School-wide positive
behavior support (SWPBS) in the classroom: Assessing perceived challenges to
consistent implementation in Connecticut schools. Education and Treatment of Children,
37 (1), 1-24.
Flannery, K.B., Frank, J.L., Kato, M.M., Doren, B., & Fenning, P., (2013). Implementing
schoolwide positive behavior support in high school settings: Analysis of eight high
schools. The High School Journal, 96 (4), 267-282. Doi: 10.1353/hsj.2013.0015
Horner, R.H, (2000). Positive behavior supports. Focus on Autism and Other Developmental
Disabilities, 15 (2), 97-105. doi: 10.1177/108835760001500205
McKevitt, B.C. & Braaksma, A.D. (2011). Best practices in developing a positive behavior
support system at the school level. In A. Thomas & J. Grimes (Eds.), Best practices in
school psychology III (pp. 735-748). Bethesda, MD: National Association of School
Psychologists.
TIER I BEHAVIOR 64
Appendix A
TIER I BEHAVIOR 65
Appendix B
TIER I BEHAVIOR 66
Appendix C
Professional Development
Pre-Forum Evaluation
Presenter: Katie Newman Date: February 2014
Topic: 9th
Grade Discipline Forum
Indicate Department:
English
Math
Science
Social Studies
Special Education
Other:
Instructions: Please circle the number that represents your feelings regarding each question
Strongly
Agree Agree Not Sure Disagree
Strongly
Disagree
1. I have a strong understanding of
effective PBS strategies 1 2 3 4 5
2. I know and understand how
discipline data is collected and
analyzed at AHS 1 2 3 4 5
3. I feel confident in my knowledge
of classroom behavioral
management strategies 1 2 3 4 5
4. What are the current AHS school
rules?
1.
2.
3.
4.
5. I feel confident in my
understanding of what constitutes a
“teacher- managed” vs. an “office-
managed” behavior
1 2 3 4 5
6. I feel that my 9th
grade colleagues
and I have worked as a team to
develop and implement consistent,
behavioral expectations and
discipline procedures.
1 2 3 4 5
TIER I BEHAVIOR 67
Appendix D
Post-Forum Evaluation
Presenter: Katie Newman Date: February 2014
Topic: 9th
Grade Discipline Forum
Indicate Department:
English
Math
Science
Social Studies
Special Education
Other:
Instructions: Please circle the number that represents your feelings regarding each question having
participated in the 9th
Grade Discipline Forum Strongly
Agree
Agree Not Sure Disagree Strongly
Disagree
1. I have a strong
understanding of effective
PBS strategies 1 2 3 4 5
2. I know and understand
how discipline data is
collected and analyzed at
AHS
1 2 3 4 5
3. I feel confident in my
knowledge of classroom
behavioral management
strategies
1 2 3 4 5
4. What are the current AHS
school rules?
1.
2.
3.
4.
5. I feel confident in my
understanding of what
constitutes a “teacher-
managed” vs. an “office-
managed” behavior
6. I feel that my colleagues
and I have worked as a
team to develop and
implement consistent,
behavioral expectations
and discipline procedures.
I learned something new
today 1 2 3 4 5
I will use the information
that I learned today 1 2 3 4 5
TIER I BEHAVIOR 68
It was easy to talk with the
group to ask questions 1 2 3 4 5
The handouts were helpful 1 2 3 4 5
This session was… Too Short Too Long Just Right
Additional Comments:
TIER I BEHAVIOR 69
Appendix E
TIER I BEHAVIOR 70
TIER I BEHAVIOR 71
TIER II ACADEMIC 72
A Tier II Academic Intervention for Increasing MAZE Comprehension Scores of Six Ninth
Grade Students
An intervention was created to increase the reading comprehension of six ninth grade
students. Selection criteria for the group included not passing the eighth grade Ohio
Achievement Assessment (OAA) coupled with non-passing first quarter English class grades.
Four of the six students benchmarked on fluency measures in the spring of their eighth grade
year, however did not benchmark on DIBELS DAZE comprehension measures, which led to the
primary focus of the proposed intervention to be reading comprehension. The remaining two
students did not benchmark on fluency measures in the spring of their eighth grade year,
enforcing the need for an intervention with a secondary target of fluency. The intervention
chosen to primarily target reading comprehension and secondarily target reading fluency was the
Reread-Adapt and Answer-Comprehend (RAAC) intervention. The DIBELS Daze assessments
were used to collect baseline data as well as to progress monitor. Secondary baseline and
progress monitoring data was collected using the RAAC Built-in fluency measures, DIBELS
Oral Reading Fluency (ORF), as well as English grades.
The RAAC intervention (Appendix A) was implemented in a small group setting in the
cafeteria in a rural school. Students participating in the intervention were provided with an
introduction to the intervention and training on the steps completed. The intervention took place
on Mondays, Wednesdays, and Fridays for twenty to twenty five minutes per session. The
intervention was conducted by a parent volunteer to the school.
Monthly, the University of Cincinnati school psychology student would assess each
student using a DIBELS Daze probe to monitor the students’ progress toward the reading
comprehension goal. Built-in, RAAC fluency data were collected weekly through the
intervention by the interventionist. Additionally, grades in students’ English class were recorded
TIER II ACADEMIC 73
weekly. The data for grades were summarized in graphical form. Student grades were reported
by displaying the median grade per quarter. Data showed that three students met the goals for
comprehension and fluency and could graduate from the intervention but did not increase
English grades. One student had not met the goals three consecutive times however it was
determined that the data collected was not an accurate depiction of his skills, as his behavior
impeded his performance. For the above-mentioned four students, the intern school psychologist
would meet with them weekly to review grades and missing assignments to target work
completion and motivation. Data revealed that two students had not yet met the goals of the
intervention and would remain as participants in the RAAC intervention.
Methods
Participants and roles
The assistant principal at the high school contacted the school psychology intern after
reviewing first quarter English grades for all ninth grade students. Upon reviewing the first
quarter English grades of ninth grade students, it was determined that a small group of six
students received a failing grade of 70 or below; not including students in special education. The
assistant principal recommended that these students be targeted for a small group reading
intervention. Upon reviewing spring eighth grade DIBELS progress monitoring for the selected
students, it was determined that four of the six students benchmarked in DIBLES Oral Reading
Fluency (ORF); however, did not benchmark in DIBELS DAZE, indicating a primary need for
comprehension instruction. The remaining two students did not benchmark on either ORF or
DAZE measures at the end of their eighth grade year indicating a need for both fluency and
comprehension instruction.
The school psychologist and school psychology intern met with the assistant principal
and further narrowed the group selection criteria to students who did not receive a passing first
TIER II ACADEMIC 74
quarter grade in English and also did not pass the eighth grade Reading OAA. The University of
Cincinnati school psychology intern determined, in collaboration with the school psychologist
and assistant principal, that the Reread-Adapt and Answer-Comprehend (RAAC) intervention,
targeting both reading fluency and comprehension, would be an appropriate intervention
according to student need. Permission to collect baseline, implement the intervention, and
progress monitor was collected from parents of participating students.
Setting
The intervention was implemented at large tables in the high school cafeteria during a
non-meal time. At the conclusion of their second bell classes, the participating students reported
to the cafeteria. Students participated in the intervention three times per week (Mondays,
Wednesdays, and Fridays) for twenty to twenty five minutes per session. The participating
students participated in the intervention either at the start of either a non-core class (physical
education or computers) or a history class.
Target variables
As determined through universal screening data, the primary target variable for the Tier II
intervention was reading comprehension. Reading comprehension is defined as “the active
extraction and construction of meaning from all kinds of text” (Vorstius, Radach, Mayer, &
Lonigan, 2013, p.191). This variable was measured using the DIBLES DAZE assessment. To
develop reading comprehension skills, one must have mastered prerequisite skills such as
reading fluency, phonological skills, inference generation, vocabulary, prior knowledge, and
verbal ability (Bellinger & DiPerna, 2011). When students are unable to read text fluently,
comprehension of the text lacks because excess cognitive resources are being used to decode
words, leaving little capacity to understand the text (Roberts, Good, & Corcoran, 2012). The
most common curriculum-based measure of reading competence and reading comprehension
TIER II ACADEMIC 75
used is oral reading fluency (Roberts et al., 2012). Progress monitoring of the target variable
occurred monthly and was taken through DAZE probes. One Friday per month at 9:00 am the
students were given a DAZE assessment and progress was tracked over time. The DAZE probe
was given for the DIBELS recommended three minutes.
The secondary target variable was reading fluency. Reading fluency is defined as “the
ability to read with speed, accuracy, and proper expression” (Therrien, Kirk, &Woods-Groves,
2011). Oral reading fluency has been found to better predict reading comprehension than direct
comprehension measures such as questioning and retell (Fuchs, Fuch, & Hosp, 2001). This
variable was primarily measured through the number of words correct per minute that the student
could read using the RAAC. Progress monitoring occurred weekly through the built-in RAAC
fluency measures as well as quarterly through DIBELS ORF probes. One Friday per quarter at
9:00 am the students were given a ORF assessment and progress was tracked over time. The
ORF probe was given for the DIBELS recommended one minute.
The tertiary variable targeted was student grades in English class. Grades were chosen as
a tertiary variable due to the link between oral reading fluency and overall reading achievement,
as indicated by grades in English (Rasinski, Samuels, Hiebert, Petscher, & Feller, 2011).
Research indicates that reading fluency instruction improves word recognition, reading fluency,
reading comprehension, and reading achievement (Rasinski et al., 2011). Although much
fluency instruction and intervention occurs at the primary grade levels, research suggests that
there is also a need at the high school level for such instruction (Rasinski et al., 2011). Vorstius
et al., (2013) reported that nearly 25% of eighth grade students read below the basic level of
competence. For the reported Tier II intervention, it was desired for increases in the number of
TIER II ACADEMIC 76
words correct per minute read, as indicated by the RAAC fluency measures and the ORF
assessment, and increases in DAZE measures, to be reflected in increased English grades.
Inter-scorer agreement
Inter-scorer agreement was calculated for 20% of the DAZE progress monitoring probes
given as well as 20% of the ORF progress monitoring probes given. Inter-scorer agreement was
also calculated for 21% of the RAAC Fluency progress monitoring probes given. Inter-scorer
agreement was calculated by dividing the total number of agreements by the number of
agreements plus disagreements. Inter-scorer agreement percentages are indicated in Table 1
below.
Table 1
Inter-scorer Agreement Data
Date DAZE IOA ORF IOA RAAC Fluency IOA
12/11/13 100%
1/24/14 100% 100% 100%
2/28/14
3/05/14
100% 100%
0100 100%
3/21/14 100% 100% 100%
3/8/13 100% 100%
Goals and decision rules
Goals and decision rules were established jointly between team members. The primary
variable, reading comprehension, was measured using sixth grade DIBELS DAZE progress
monitoring assessments. Sixth grade assessments were used, as baseline data from both the 8th
grade year and pre-intervention baseline taken in the ninth grade year, indicated that not all
students had met the spring benchmark norm for the sixth grade year. The sixth grade spring
benchmark goal for DAZE is an adjusted score of 21.
TIER II ACADEMIC 77
The primary goal for reading fluency was 140 words correct per minute which was the 9th
grade local norm established by the school district. Reading fluency was primarily assessed
using the RAAC passages, as words correct per minute was recorded as well as total time to read
the passage. DIBELS ORF sixth grade benchmark norms were also considered as the ORF
assessment was given to students quarterly as a secondary, standardized measure of reading
fluency. Sixth grade materials were used as baseline data indicated that not all students had met
the end of the year sixth grade benchmark of 120 words correct per minute (wcpm).
The RAAC intervention included progress monitoring of words correct per minute and
this score was recorded and used to monitor progress. Students were assessed weekly on words
per minute. Student grades in English class were also considered, and grades were monitored
and tracked weekly. The goals are indicated in Table 2 below.
Decision rules for the measures were also determined jointly by the team. It was
determined that a student was prepared to graduate from the intervention when they scored three
consecutive data points above the goal line on the DAZE and ORF assessments. Similarly, the
decision rule was to make a change in the intervention when a student scored three consecutive
data points below the aimline. See Table 2 for assessment goal information.
Table 2
Assessment Goals
Spring Universal Screening Goal
DAZE (6th Grade Level) 21
ORF (6th Grade Level) 120
WCPM (9th Grade Level) 140
English Grade 70%
TIER II ACADEMIC 78
Functional hypothesis
Based on administrator and teacher interviews, it was determined that the function of the
below average DAZE scores was a skill deficit due to a lack of practice opportunities and
corrective feedback on reading comprehension activities. The team concluded that the students
were not making adequate progress in the area of oral reading fluency due to the lack of practice
on repeated reading activities and feedback on errors.
Accountability plan
An ABC research design was used for this Tier II intervention. The baseline condition
(A) included three DAZE data points for 2 of the students and one DAZE data points for four of
the students. Baseline for ORF included three ORF data points for 2 of the students and one
ORF data points for four of the students. Baseline for the RAAC measure of words correct per
minute included one data point for each of the six participating students. The RAAC intervention
phase (B) consisted of sessions where the students listened to the assigned text read aloud by the
interventionist, listened to peers read the text aloud, read the text aloud themselves, and
answered comprehension questions derived from the text. The RAAC intervention phase plus
review of progress monitoring data and grades phase consisted of sessions where the students
continued to receive support through the RAAC intervention, however, one time per week they
were also shown their progress monitoring data and current English grades. The ABC design
type is used in practical settings because of the ethical concerns with withdrawing an
intervention that is showing positive results; however, there are limits to the use of an ABC
design due to threats to internal validity (O’Neill, McDonnell, Billingsley, & Jenson, 2011).
Intervention procedures
Baseline. Three baseline data points were taken for two of the students for the DAZE
and ORF measures. The remaining four students were given one baseline measure for the DAZE
TIER II ACADEMIC 79
and ORF assessments. Two to four baseline data points were taken for all students on the RAAC
wcpm measure. The baseline measurement of grades consisted of the group median grade in
English for quarter one (fall).
The DIBELS DAZE screening measure can be administered to a whole class at the same
time, to a small group of students, or individually. Students are given a passage where
approximately every seventh word has been replaced by a box containing the correct word and
two distractor words. Using standardized directions, students are asked to read the passage
silently and circle their word choices. The student receives credit for selecting the word that best
fits the omitted word in the reading passage. The scores that are recorded are the number of
correct and incorrect responses. An adjusted score, which compensates for guessing, is
calculated based on the number of correct and incorrect responses. Daze Adjusted Score =
number of correct responses – (number of incorrect responses ÷ 2).
The DIBELS ORF screening measure is an individually administered, standardized
assessment of fluency with connected text. Passages are leveled by grade and a goal is derived
for each grade level text. Student performance is measured through the number of correct words
that the student can read aloud in one minute and is determined to be the students’ oral reading
fluency rate. Words that are omitted, substituted, and hesitated upon for more than 3 seconds are
counted as errors. Self-corrected words (within 3 seconds) are counted as words correct.
http://dibels.uoregon.edu
The RAAC measurement of words correct per minute was an individually administered,
assessment of fluency. RAAC passages were provided and students were prompted to read the
passage. Although students were instructed to read the entire passage, a bracket was placed after
TIER II ACADEMIC 80
the last word read at the one minute time mark as students continued to read through the end of
the passage.
Grade information was viewed online from an online grade recording system, Progress
Book. Classroom grades as well as report card information could be viewed at any time by
students, parents, and school personnel.
Reread-Adapt and Answer-Comprehend (RAAC). Following the identification of six
students in need of both reading fluency and comprehension intervention, the school
psychologist and intern school psychologist met to discuss intervention options. Of the six
students participating in the Tier II group, all students displayed a need for comprehension
intervention and two of the students displayed the additional need for reading fluency
intervention. The intern school psychology researched the school’s intervention resources and
determined the research-based RAAC intervention to be an appropriate Tier II intervention
matched to the target needs of the six participating students.
The RAAC intervention targets reading fluency through repeated reading at the students’
instructional level to an interventionist who corrects decoding errors (Youjia et al., 2012).
Repeated reading is a reading strategy which allows students to re-read a leveled reading passage
until they obtain a goal-level of fluency (Therrien, Kirk, & Woods-Groves, 2012). Repeated
reading has much research support as an effective intervention for improving the reading fluency
of students (Therrien, Kirk, & Woods-Groves, 2012). Comprehension is addressed through the
RAAC intervention as students read the passages with the purpose of answering questions; then
are asked to provide answers to question generation prompts (Therrien, Kirk, & Woods-Groves,
2012). Research indicates that the RAAC program has been effective in increasing the reading
TIER II ACADEMIC 81
fluency scores and reading comprehension of participating students (Therrien, Kirk, & Woods-
Groves, 2012).
RAAC+ Review of Progress Monitoring and Grades
Research has suggested that behavioral or motivational components added to the regular
RAAC intervention procedures will strengthen the intervention promote the highest student
growth. Following the start of intervention, it was noted by the interventionist and the intern
school psychologist that students did not feel investment in their progress monitoring scores,
were continuing to struggle with English class work completion, and were not motivated to
attend group on time. When assessing students using the progress monitoring materials, students
looked off, not oriented toward the assessment, quickly circled answers without fully reading the
materials, and talked out during the assessments. To address these concerns, an additional
component of reviewing progress monitoring as well as grades was added to the RAAC
intervention one day per week. Weekly, the intern school psychologist printed students’ grades
in English class and reviewed any missing assignments. Prior to each ORF and DAZE progress
monitoring assessment, the intern school psychologist reviewed updated graphs of the students’
data with them, providing a reminder of the goal as well as prompting to do the best work
possible on the assessment. Students were provided with tangible reinforcement (candy) when
they met, or made progress toward the goal.
Adherence data
Adherence checks were conducted using the RAAC intervention script. The intern
school psychologist observed 20% of the intervention sessions, recording adherence to the
intervention script. The parent volunteer interventionist had implemented the RAAC
intervention in the past and therefore was familiar with the script and provided the intervention
fluently. See Table 3 for intervention adherence.
TIER II ACADEMIC 82
Table 3
Intervention Adherence Data
Date RAAC
12/11/13 100%
1/24/14 100%
2/28/14 100%
3/05/14 100%
3/08/14 100%
3/21/14 100%
Social validity
All target variables, intervention components, decision rules, and goals were determined
with the team in a collaborative manner. Frequent meetings occurred between the intern school
psychologist, school psychologist, and assistant principal to discuss the progress of the
participating students in the classroom, both in classroom grades and in progress monitoring
scores for reading fluency and comprehension. Social validity was also informally assessed by
asking the English teacher if changes were seen in student performance in the classroom. The
teacher noted that behavior in the classroom continued to be a concern for several of the students
participating in the group, therefore he did not feel that students’ true ability was being displayed
on in-class tasks.
Results
The baseline phase means and standard deviations for the DAZE and ORF variables are
displayed in Tables 4 and 5, respectively. The intervention phase means and standard deviations
of the DAZE and ORF variables are displayed in Tables 6 and 7, respectively. The percent of
non-overlapping data, goal attainment scaling, and effect size for the target variables can be
viewed in Tables 8 and 9. The percentage of non-overlapping data (PND) for each variable was
calculated by identifying how many intervention data points did not overlap with the highest
TIER II ACADEMIC 83
baseline data point (the proportion of non-overlapping intervention data points to the total
number of intervention data points). The effect size was calculated by dividing the difference of
the baseline and intervention means by the standard deviation of the baseline. Effect sizes could
not be calculated in instances where only one baseline data point could be collected. Goal
attainment scaling was used to indicate (“0” corresponds with no progress towards the goal, “1”
corresponds with progress towards the goal, and “2” indicates that the goal was fully met)
growth toward the final DAZE goal. Goal Attainment Scale scores for the DAZE and words
correct per minute variable are displayed in Tables 8 and 9.
Table 4
Means and Standard Deviations of DAZE Variable (Baseline Phase)
Observed DAZE score (Baseline Phase)
Student Number of baseline
data points
Mean Standard Deviation
Student 1 3 11.0 1.73
Student 2 1 40.0 N/A
Student 3 1 24.0 N/A
Student 4 3 21.0 2.65
Student 5 1 21.0 N/A
Student 6 1 23.0 N/A
Table 5
Means and Standard Deviations of WCPM Variable (Baseline Phase)
Observed RAAC wcpm score (Baseline Phase)
Student Number of baseline
data points
Mean Standard Deviation
Student 1 3 114.8 7.5
Student 2 3 233.0 42.5
Student 3 3 165.0 23.3
Student 4 2 173.0 5.7
Student 5 2 134.5 19.1
Student 6 3 132.7 19.4
TIER II ACADEMIC 84
Table 6
Means and Standard Deviations of DAZE Variable (Intervention Phase)
Observed DAZE score (Intervention Phase)
Student Number of intervention
data points
Mean Standard Deviation
Student 1 3 14.7 8.7
Student 2 3 34.0 7.5
Student 3 3 28.0 11.5
Student 4 3 28.3 2.3
Student 5 3 14.0 3.0
Student 6 3 22.0 4.4
Table 7
Means and Standard Deviations of WCPM Variable (Intervention Phase)
Observed RAAC wcpm score (Intervention Phase)
Student Number of intervention
data points
Mean Standard Deviation
Student 1 11 97.2 17.2
Student 2 9 221.2 17.9
Student 3 7 172.6 28.1
Student 4 9 162.1 13.1
Student 5 10 147.9 18.1
Student 6 14 142.5 14.8
Table 8
Percent of Non-overlapping Data (PND), Goal Attainment Scaling (GAS), and Effect Size for DAZE
variable
Student PND GAS Effect Size
Student 1 66.7% +1 2.1
Student 2 0% +2 Cannot be calculated
Student 3 100% +1 Cannot be calculated
Student 4 33.3% +1 2.8
Student 5 33.3% 0 Cannot be calculated
Student 6 66.7% +1 Cannot be calculated
TIER II ACADEMIC 85
Table 9
Percent of Non-overlapping Data (PND), Goal Attainment Scaling (GAS), and Effect Size for words
correct per minute variable
Student PND GAS Effect Size
Student 1 0% +0 -2.35
Student 2 0% +2 -.28
Student 3 29.6% +2 0.33
Student 4 22.2% +2 -1.9
Student 5 30.0% +1 0.7
Student 6 28.6% +1 0.51
Figure 1 displays the DAZE comprehension data for Student 1. The student did reach the
6th
grade DIBELS DAZE goal of 21, however did not yet meet the intervention goal of scoring a
21 or higher, three consecutive times. Progress was made toward the goal. Data show the
student’s baseline points as below the goal line. It was determined that the student should
continue to receive intervention support through the RAAC intervention until reaching the
intervention goal.
TIER II ACADEMIC 86
Figure 1: Student 1 DAZE
Figure 2 displays the oral reading fluency data for Student 1. Baseline measures of
fluency using both the RAAC passages, and DIELS ORF measures were below the goaline of
140 words correct per minute. Following intervention, the student’s scores fell below the
aimline and an additional compondent was added to the intervention package. Despite adding
the review of progress monitoring data and grades, the student’s scores remained below the
aimline. The team decided that the student would continue to participate in the RAAC
intervention and a meeting was scheduled to discuss modifying the intervention to better address
the student’s fluency concerns, as the oral reading fluency score usign the RAAC measure had
been below the aimline three consecutive times since the prior modification to the intervention.
TIER II ACADEMIC 87
Figure 2: Student 1 ORF
Figure 3 displays the DAZE comprehension data for Student 2. The student did reach the
6th
grade DIBELS DAZE goal of 21, three consecutive times. Data show the student’s baseline
point as being above the goal line; however, spring benchmark scores from the 8th
grade year had
displayed below goal scores and a failing OAA score in Reading suggested that the student
participate in the intervention. Following the intervention implementation, the student reached
the DAZE goal and it was demined that the student could discontinue the support of the RAAC
intervention. It was decided for the student to meet one time per week with the intern school
psychologist to review current grades and missing assignments. The student would be asked to
create an action plan for completing and turning in missing assignments.
TIER II ACADEMIC 88
Figure 3: Student 2 DAZE
Figure 4 displays the oral reading fluency data for Student 2. Baseline measures of
fluency using both the RAAC passages, and DIELS ORF measures were avove the goaline of
140 words correct per minute. Following intervention, the student’s scores remained above the
goal line. Student scores did not significatnly increase in the data point collected directly
following the additional component of review of progress monioting and grades. Student 2’s
oral reading fluency scores which displayed consistency above the goal line throughout
intervention implementation, further confirmed that the student no longer required Tier II
support in the areas of reading comprehension and reading fluency.
TIER II ACADEMIC 89
Figure 4: Student 2 ORF
Figure 5 displays the DAZE comprehension data for Student 3. The student did reach the
6th
grade DIBELS DAZE goal of 21, two consecutive times; however, had not met the goal of
scoring a 21, three consecutive times. Data show the student’s baseline point as being above the
goal line; however, spring benchmark scores from the 8th
grade year had displayed below goal
scores and failing OAA scores in Reading suggested that the student participate in the
intervention. After intervention implementation, the student fell below the goal line. Following
the addition of the review of progress monitoring data and grades to the RAAC intervention, the
student then reached the DAZE goal two consecutive times. Although the student had not yet
reached the goal a third time, above goal oral reading fluency scores, which can be an indicator
TIER II ACADEMIC 90
of reading comprehension, supported the team decision the student could discontinue the
support of the Tier II RAAC intervention. It was decided for the student to meet one time per
week with the intern school psychologist to review current grades and missing assignments. The
student would be asked to create an action plan for completing and turning in missing
assignments.
Figure 5: Student 3 DAZE
Figure 6 displays the oral reading fluency data for Student 3. Baseline measures of
fluency using the RAAC passages, were above the goaline of 140 words correct per minute.
Following intervention, the student’s scores remained above the goal line. Student scores were
variable in the data collected following the additional component of review of progress
monitoring and grades. Oral reading fluency scores for Student 3, which displayed consistency
TIER II ACADEMIC 91
above the goal line throughout intervention implementation, further confirmed that the student
no longer required Tier II support in the areas of reading comprehension and reading fluency.
Figure 6: Student 3 ORF
Figure 7 displays the DAZE comprehension data for Student 4. The student did reach the
6th
grade DIBELS DAZE goal of 21, three consecutive times. Data show the student’s baseline
points as being variable above and below the goal line. Following the intervention
implementation, the student’s scores maintained above the DAZE goal and it was demined that
the student could discontinue the Tier II support of the RAAC intervention. It was decided for
the student to meet one time per week with the intern school psychologist to review current
TIER II ACADEMIC 92
grades and missing assignments. The student would be asked to create an action plan for
completing and turning in missing assignments.
Figure 7: Student 4 DAZE
Figure 8 displays the oral reading fluency data for Student 4. Baseline measures of
fluency using the RAAC passages, were above the goaline of 140 words correct per minute.
Measures using the DIBELS ORF assessment, indicated scores above and slightly below the
goal. Following intervention, the student’s scores on the RAAC measure of fluency remained
above the goal line. Student scores indicated no positive increase on the assessment directly
following the additional component of review of progress monitoring and grades. Oral reading
fluency scores for Student 4, which displayed consistency above the goal line throughout
TIER II ACADEMIC 93
intervention implementation, further confirmed that the student no longer required Tier II
support in the areas of reading comprehension and reading fluency.
Figure 8: Student 4 ORF
Figure 9 displays the DAZE comprehension data for Student 5. The student did not reach
the 6th
grade DIBELS DAZE goal of 21. Data show the student’s baseline point as being at the
goal line, with spring benchmark scores from the 8th
grade year displaying below goal scores. In
addition, teacher recommendations suggested that the student participate in the intervention.
Following the intervention implementation, the student’s DAZE score fell below the goal line.
The student’s DAZE score remained below the goal line for three consecutive points. It was
determined that the student should continue to receive the support of the RAAC intervention. A
TIER II ACADEMIC 94
meeting was scheduled to determine modifications that could be made to the intervention, as per
the decision rule guidelines.
Figure 9: Student 5 DAZE
Figure 10 displays the oral reading fluency data for Student 5. Baseline measures of
fluency using the RAAC passages and DIBELS ORF assessments, were variable, displaying
scores both above and below the goaline of 140 words correct per minute. Following
intervention implementation, the student’s scores on the RAAC measure of fluency remained at
or slightly below the goal line. Student scores indicated no immediate positive increase on the
assessments directly following the additional component of review of progress monitoring and
grades. The student’s last two RAAC passage oral reading fluency assessments showed scores
TIER II ACADEMIC 95
above the goal line. DIBELS ORF assessment scores however, were shown to be below the goal
line. Due to continued concerns in reading comprehension, as confirmed by the DAZE
assessment scores, coupled with not having consistently met the goal for oral reading fluency, it
was decided by the team that the student would continue to benefit from the support of Tier II
RAAC intervention supports.
Figure 10: Student 5 ORF
Figure 11 displays the DAZE comprehension data for Student 6. The student did reach
the 6th
grade DIBELS DAZE goal of 21, however did not meet the goal three consecutive times.
Data show the student’s baseline point as being above the goal line. Following the RAAC
TIER II ACADEMIC 96
intervention implementation, the student’s DAZE score remained above the goal line. The
student’s DAZE score fell below the goal line following the addition of the review of progress
monitoring data and grades to the intervention package. The last collected data point displayed a
score above the goal. Student 6 displayed significant behavior concerns and had received several
discipline referrals throughout the school year. In having conversations with the assistant
principal and the student’s teachers, the team determined that the student’s primary concerns
were work completion and behavior. Scores of reading fluency and comprehension were not
reflecting the student’s true abilities due to his off-task behaviors during the assessments. It was
decided for the student to meet one time per week with the intern school psychologist to review
current grades and missing assignments. The student would be asked to create an action plan for
TIER II ACADEMIC 97
completing and turning in missing assignments.
Figure 11: Student 6 DAZE
Figure 12 displays the oral reading fluency data for Student 6. Baseline measures of
fluency using the RAAC passages, show scored both below and above the goaline of 140 words
correct per minute. Measures using the DIBELS ORF assessment, indicates a score below the
goal. Following intervention implentation, the student’s first two scores on the RAAC measure
of fluency were below the goal line, followed by one score slightly above the goal line. The
student’s scores again fell below the goal line for two progress monitoring points. The final
progress monitoring point indicated a score above the goal line. Student scores indicated no
TIER II ACADEMIC 98
positive increase on the assessment directly following the additional component of review of
progress monitoring and grades. Oral reading fluency scores for Student 6, which displayed
inconsisty throughout the intervention, did not meet the goal of three consistent scores above the
goal. Despite not meeting the oral reading fluency goal three consecutive times, it was decided
by the team that assessments were not displaying the student’s full ability, as his behavior was
impeding the accuracy of the assessments. The team determined that a one time per week
meeting to review current grades as well as missing assignments would be beneficial for the
student.
Figure 12: Student 7 ORF
TIER II ACADEMIC 99
The median grades of the six participating students for each quarter were also graphed
and reviewed. Visual analysis shows that no positive increases were seen in student grades
following participating in the intervention. It was seen that students 2,3,5, and 6, although
meeting DAZE and words correct per minutes goals, were continuing to display failing English
grades. To address this, those students were to transition out of the RAAC intervention group
and begin to meet with the intern school psychologist weekly for a “Check-In” to review current
grades and missing assignments.
TIER II ACADEMIC 100
Discussion
Progress monitoring data indicated that three students reached both the MAZE and words
correct per minute goal, two students were making progress toward the goals, and one student
was not making progress towards the goal. Some limitations were present throughout the
duration of this case. One limitation was that although students were making progress and
meeting the MAZE comprehension and words correct per minute end of year goal, the gains
were not reflected in student English grades. It was hypothesized that student motivation was a
factor in not seeing the expected increases in grades. Although students may have increased
their skills on reading comprehension and fluency, lack of work completion may have decreased
students’ overall English grade, masking any positive effects that may have been reflected on
tests/quizzes. In the future I will collect data on student quiz and test averages as well as overall
English grade in order to determine if student skills are being displayed on exams although
overall English grade is low due to missing assignments.
Another limitation of this consultation was that students participated in the intervention
across 29 sessions. The RAAC intervention is intended to be implemented for at least 50 sessions
as reading fluency intervention is most effective in making a change in students’ overall reading
achievement when provided over an extended period of time (Therrien, Kirk, & Woods-Groves,
2012). Two of the students will continue to participate in the intervention; however, the timing
of summer break prohibits the total length of intervention implementation. In the future when
considering a similar reading comprehension and fluency intervention, I will review data and
collect progress monitoring earlier in the school year in order to begin intervention as soon as
possible to provide support over a longer period of time.
TIER II ACADEMIC 101
It is possible that other measurement options could have been explored to serve as
additional measures of comprehension. For example, the number of comprehension questions
answered correctly on each RAAC worksheet could have been tracked over time to see if the
number of comprehension questions increased.
Another limitation of this group was its size. The RAAC intervention is best utilized in a
one-on-one consultation or with two or three students. The intervention time was lengthened
with less individual time for each student due to group size. In the future I will better match my
population size to the type of intervention implemented. Additionally, more baseline data for the
DAZE measure would have allowed for a more accurate measure of gains for the variable.
Effect size could not be calculated for students with only one baseline data point which
contributed to a limitation for this case.
This case served as an excellent example of the importance of the use of multiple sources
of data to inform decision making when utilizing Multi-Tiered Systems of Support. Universal
screening data from the Spring of the previous year, combined with state assessment data, and
current progress in the classroom combined to provide evidence that the participating students
would benefit from Tier II supports for reading comprehension and fluency.
This case was also a great example of the importance of data-based decision making. For
two students who did not meet the goals of the intervention, the data provided evidence that they
should remain as participants in the intervention. For the third student who did not meet the
goals of the intervention, an opportunity for child advocacy was provided. Although the
student’s scores indicated that his skills were below goal levels, the team determined that the
student would be best supported with a more behavioral intervention to aide with work
TIER II ACADEMIC 102
completion. For all other students, visual analysis of the data allowed the team to make data-
based decisions in graduating students from the intervention.
This consultation provided a positive experience for the intern school psychologist.
Collaboration skills with teachers, parent volunteer, and other stakeholders were practiced and
the intern school psychologist was provided the opportunity to discuss student data and progress
towards meeting the comprehension and reading fluency goals with the collaborating team
members. Data-based decision making skills were also expanded, as student data was frequently
measured and analyzed. The skills discussed are critical components of the intern school
psychologist’s personal model of practice. This consultation was extremely valuable in
providing an opportunity to use data to inform decision-making for adding Tier II supports to
increase student growth in the areas of reading comprehension and fluency.
TIER II ACADEMIC 103
References
Bellinger, J.M. & DiPerna, J.C., (2011). Is fluency-based story retell a good indicator of reading
comprehension? Psychology in the Schools, 48 (4), 416-426.
Fuchs, L.S., Fuchs, D., & Hosp, M.S., (2001). Oral reading fluency as an indicator of reading
competence: A theoretical, empirical, and historical analysis. Scientific Studies of
Reading, 5, 239-256.
O’Neill, R.E., McDonnell, J.J., Billingsley, F.F., & Jenson, W.R. (2011). Single Case Research
Designs in Educational and Community Settings. Upper Saddle River, NJ:
Pearson Education, Inc.
Rasinski, T., Samuels, S.J., Hiebert, E., Petscher, Y., & Feller, K., (2011). Reading Psychology,
32, 75-97.
Roberts, G., Good, R., & Corcoran, S., (2005). Story retell: A fluency-based indicator of reading
comprehension. School Psychology Quarterly, 20 (3), 304-317.
Therrien, W.J., Kirk, J.F., & Woods-Groves, S., (2011). Comparison of a reading fluency
intervention with and without passage repetition on reading achievement. Remedial
Special Education, 33 (5), 309-319.
Vorstius, C., Radach, R., Mayer, M.B., & Lonigan, C.J. (2013). Monitoring local
comprehension monitoring in sentence reading. School Psychology Review, 42(2), 191-
206.
Youjia, H., Therrien, W.J., Hendirckson, J.M., Woods-Groves, S., Ries, P.S., & Shaw, J.W.,
2012, Education and Training in Autism and Developmental Disabilities, 47 (1), 72-83.
TIER II ACADEMIC 104
Appendix A
Modified RAAC Intervention Script
Sessions will last between 15-30 minutes.
Day 1:
1. Prompt the students with the following statement “Open your binders to the page marked
with a Post-It note. I am going to read the story aloud one time and then you will take
turns reading the story two more times. Everyone should follow along with the passage
as others are reading. Pay attention to what is being read as you will need to answer these
questions.” (Point to the cue card questions)
2. Read the passage one time aloud. When you have completed the story say, “This time
you will take turns reading the story”.
3. Prompt one student to begin reading the first sentence of the story. Continue to assign
each remaining sentence to different students.
4. Repeat step 3 again, having the students read the story for the 2nd
time.
5. Present a cue card containing the generic story structure questions and prompt one
student to read the first question aloud. Call on one student to answer the first question.
Repeat step 5 with each remaining question.
Day 2:
1. Prompt the students to open their binders to the page marked with a Post It note.
2. Explain to the students that one student at a time will read the passage to you while the
other students are to reread the story on their own.
3. Pull one student at a time and ask the student to reread the passage aloud to you until s/he
reaches the pre-established goal. Each passage is read a minimum of 2 times or a
maximum of 4 times.
***Place a bracket behind the last word read in 1 minute and mark a slash through any
words read incorrectly. Mark the student’s total time for each read on the indicated
line as well as the number of errors for each reading.
4. Provide instructional feedback on word errors.
a. If student hesitates on a word for 3 seconds or omits a word or words, error
correction is to be provided immediately.
b. Otherwise, error correction is to be provided after the passage has been read but
before it is reread.
c. Error correction includes providing the word or words and asking the student to
repeat the error word back.
Day 3:
1. Read the passage one time aloud to the group.
2. Ask the students to silently answer the factual/inferential questions that follow the story,
on their worksheet.
3. When all students have completed the questions on their worksheets, call on students
individually to read the questions and their responses. Correct any incorrect responses
and refer to where the answer is implied in the story.
TIER II BEHAVIOR 105
A Tier II Behavior Small Group Intervention to Increase Positive Peer and Positive Adult
Interactions in Eighth Grade Students
A social skills group was designed for a small group of seven eighth grade students in a
rural middle school in Ohio. The seven students participating in the group were referred by
eighth grade teachers based on concerns with the students’ negative peer and negative adult
interactions. Additionally, input for referrals was also given by two middle school counselors
and the building principal. The target variables for the social skills intervention group were
positive peer and positive adult interactions. A self-rating survey was given to students pre-
intervention and post intervention, to measure student beliefs regarding their own proneness to
bullying, positive peer interactions, positive adult interactions, and anger management
(Appendix A). A survey was also given to teachers assessing their ratings on the participating
students’ social skills pre-participation and post-participation in the social skills intervention
group (Appendix B).
One University of Cincinnati school psychology intern student and the school’s Positive
Behavior Support (PBS) Specialist collaborated to develop session materials, rating scales, and
served as facilitators of the small group social skills intervention. Group sessions were
conducted in the cafeteria on Mondays from 8:30am-9:10am. The small group social skills
group met weekly over eight sessions. The average length per session was 35 minutes. Average
attendance for the seven participating students was 85.7%.
Parental permissions were created by the PBS specialist and were mailed home to the
parents of the seven referred students (Appendix C). All participating students were Caucasian
males and females from lower-middle class socio-economic status families attending a rural
middle school in Ohio.
TIER II BEHAVIOR 106
The function of the negative adult and peer interactions seen from the participating
students appeared to be due to problem solving skill deficits, lack of practice opportunities for
displaying positive social skills, as well as attention seeking. The students participating in the
intervention group accessed little reinforcement for appropriate social interactions within the
school setting by teachers or other school staff. Intervention sessions followed a set format using
the Second Step Violence Prevention Curriculum (Beland, 1997). Each session included the
following phases, “Energizing Phase”, “Advanced Organizer Phase”, “Work Phase”, “Processing
Phase”, and “Closing Phase”. Sessions were designed using an agenda which group facilitators
utilized to remain on topic and on script. Edible reinforcement was kept by group facilitators
and was frequently provided to students who participated by answering questions or who
displayed appropriate behaviors during group sessions. Additionally, students received
reinforcement for returning any “homework” assignments given for the group.
Results of the student self-survey did not show average positive changes in Positive Peer
Interactions (PPI) or Positive Teacher Interactions (PTI). Results of the teacher survey however,
did show average positive changes in both Positive Peer Interactions (PPI) and Positive Teacher
Interactions (PTI). Students indicated that they enjoyed participating in the small group
intervention. Collected teacher social validity indicated that for some students, positive changes
in peer and adult interactions were noted; however, for other students, no positive changes were
seen.
Based on data gathered from the pre and post surveys, it was determined that a follow-up
session should be conducted to determine if maintenance of skills had occurred. Additionally,
four of the seven students would continue one-on-one PBS consultation support to continue to
teach and practice peer and adult interaction skills.
TIER II BEHAVIOR 107
Limitations of this small group behavioral intervention included difficulty building and
maintaining a routine for the group meeting due to inclement weather with school delays and
closings. Also, measurement bias is a concern due to student self-reported data for measuring
the results of the intervention. Further, students selected for the group were reported by teaching
teams rather than through the use of universal screening data. Due to the lack of universal
screening, it was not shown that the Tier I, core behavior management system of the school was
effective making it difficult to support movement from Tier I to Tier II behavior intervention, as
the moves were not data-based, rather referral-based.
Methods
Participants, Roles, and Setting
The intervention took place in a rural middle school that serves approximately 950
students. Of those students, 94.2% were Caucasian; 45.4% were economically disadvantaged,
1.2% had limited English proficiency, and 13.6% were identified as having and educational
disability. The school was identified as having a medium-low poverty status. The school
implemented school-wide positive behavior supports; however, implementation was not
consistent. The PBS specialist serving the middle school saw a need within the eighth grade, for
additional teaching and practice with positive peer and positive adult interactions. The PBS
specialist asked the intern school psychologist to co-lead a social skills group. The three
teaching-teams within the eighth grade were asked for student referrals for students who needed
additional support with increasing positive peer and positive adult interactions.
Teaching-teams referred a total of seven students. Four of the seven students referred,
were receiving PBS support services through the PBS specialist. Informal interviews were
conducted with teachers and the PBS specialist to confirm that the included students most needed
TIER II BEHAVIOR 108
support with increasing positive peer interactions and positive adult interactions. Permission
forms were mailed home to parents and all seven permission forms were returned, consenting for
student participation. All participants were Caucasian, from lower-middle class families, were in
the eighth grade, and were between the ages of 14 and 15.
The intern school psychologist informally conduced needs assessments through teacher
consultation regarding specific referred students, prepared group session materials, and served as
a co-leader for the social skills group. The PBS specialist and intern school psychologist
collaborated to create group agendas. The social skills group was conducted at tables in the
cafeteria and occurred across eight sessions on Mondays from 8:30am to 9:10am. School
cancellations and school delays caused schedule changes with the intervention on several
occasions.
Target Behaviors and Measurement
Target behaviors were selected through consultation with the eighth grade teachers,
school counselors, PBS Specialist and the principal. The selected students were referred due to
high levels of negative peer and teacher interactions in the classroom setting.
Positive Peer Interactions. The students referred for the social skills group were noted
by teachers as displaying lower levels of positive peer interactions than same-age peers. One
goal of the social skills group was therefore to increase positive peer interactions, as measured by
student self-reporting and teacher reporting. Positive Peer Interactions was defined as
“associations with equals that are cooperative, reciprocal, and mutual” (Fabes et al., 2012,
pg.570). Positive Peer Interaction is a meaningful target variable because research has shown
that student behavior is affected by peer influence (Fabes, Hanish, Martin, Moss, & Reesing,
2012 ). Students in the adolescent phase of development reinforce negative communication and
patterns of interaction when interacting with aggressive peers (Fabes et al., 2012). When these
TIER II BEHAVIOR 109
negative interactions occur, negative communication and interaction patterns are reinforced and
delinquent behaviors are increased (Fabes et al., 2012). It was therefore important, through this
social skills group, to decrease negative peer interactions while increasing the replacement
behavior of positive peer interactions.
Positive Adult Interactions. Positive adult interactions were defined as associations
with adults that were cooperative and reciprocal. It has been found that students with chronic
problem behavior have an increased chance of developing negative teacher relationships
(Sutherland, Conroy, Vo, Abrams, & Ogston, 2013). Additionally, students who have negative
teacher relationships during the early school years are at an increased risk for later academic and
behavioral problems (Sutherland et al., 2013). Based on teacher recommendation, student need,
and research support, the intern school psychologist and PBS specialist determined that Positive
Adult Interactions was an important variable to include as a target variable for the social skills
group intervention.
Teacher rating scale. The teacher rating scale survey was based on the “Child Behavior
Scale” (Ladd & Profilet, 1996). The intern school psychologist utilized 22 items to create two
scales: Positive Peer Interactions (17 questions) and Positive Adult Interactions (5 questions).
The rating scale was given to the teachers prior to the start of intervention and following the
intervention. Ladd and Profilet (1996) found a strong correlation between the scales and direct
observations of aggressive and prosocial behaviors. To measure growth using the scale, scores
of 0,1, or 2 were given for each question. Several questions were reverse-scored. Scores were
totaled, with higher scores indicating higher levels of Positive Peer Interactions and Positive
Adult Interactions.
TIER II BEHAVIOR 110
Student rating scale. The student rating scale survey was modified from the “Child Behavior
Scale” (Ladd & Profilet, 1996). The survey included 19 questions: 14 addressing Positive Peer
Interactions and 5 addressing Positive Adult Interactions. The survey was given to students at
the beginning of the first social skills group and at the end of the last social skills group. To
measure growth using the scale, scores of 0,1, or 2 were given for each question. Several
questions were reverse-scored. Scores were totaled, with higher scores indicating higher levels
of Positive Peer Interactions and Positive Adult Interactions.
Progress monitoring. Progress monitoring was informally conducted by meeting with
teachers and asking if they noticed students making progress with increasing Positive Peer
Interactions or Positive Adult Interactions. Based on teacher comments and suggestions,
additional discussion, practice, and/or role playing opportunities were provided for certain topics.
Goals and decision rules
Goals for the intervention included increasing Positive Peer Interactions and increasing
Positive Adult Interactions. The survey allowed for a score of 0,1, or 2 for each questions
provided. It was determined that the goal for each target variable would be the score
representing an 80% of the total points possible. For instance, on the Student Self-Rating of
Positive Peer Interactions, the highest number of points that could be earned is 28; 80% of 28 is
22.4. The goal for Student Self-Rating of PPI was therefore set for 22.4. The specific points
associated with scores of 80% are indicated by a goal line on each graph. See figure 1-4 for goal
information.
Functional hypothesis
Based on interviews with teachers, the principal, the PBS specialist, and the school
psychologist, it was determined that the function of the students’ negative behaviors was due to
skill deficits, lack of practice opportunities, and attention seeking. The students received little
TIER II BEHAVIOR 111
reinforcement for displaying appropriate positive peer and positive adult interactions and often
acted out resulting in both peer and adult attention. The students participate in the small group
intervention did not have the interaction skills of same-age peers. The students’ lack of skills to
maintain positive peer and adult interactions and the desire to gain teacher and peer attention
guided the type of intervention chosen. The intervention consisted of weekly lessons which
would teach a skill, model the skill, and allowed for practice opportunities of the skill.
Accountability plan
A within-subjects design with two measurement occasions (pre-assessment and post-
intervention) and two informants (student self-report and teacher) was used for this
consultation. The pre-assessment included the collection of teacher and student survey data prior
to the start of the intervention. The intervention consisted of weekly lessons using the “Second
Step” curriculum. The post-intervention consisted of collecting teacher and survey data upon the
conclusion of the intervention.
Intervention procedures
Pre-survey data. Data were gathered from students participating in the social skills
group as well as from their corresponding teaching teams. Students were asked to complete a
survey that addressed their feelings and understanding of Positive Peer Interactions and Positive
Teacher Interactions. Teachers were also provided with a similar survey which assessed
students’ knowledge and growth of positive adult interactions.
Weekly Social Skills Group Sessions. Research indicates that teachers and other school
staff who have contact with students have the ability to support students ways which increase
tolerance and positive peer interactions among students (Osher et al., 2008). Students can be
taught social and emotional skills (Frey, Hirschstein, & Guzzo, 2000). Research indicates that
TIER II BEHAVIOR 112
safer schools are ones which provide social-emotional support for students. (Osher et al., 2008).
When studies increase their repertoire of social and emotional skills, it has been found that levels
of aggression decrease (Frey et al., 2000).
The Second Step curriculum targeted general competencies and domain-specific skills,
both of which have been shown to effectively lower student aggression (Frey et al., 2000). Each
counseling session consisted of a portion of the Second Step curriculum. Each session followed
5 phases. The first, the “Energizer Phase” included an “icebreaker” warm up activity. The
second phase, “Advanced Organizer”, included a review of the previous week, a review of the
group rules, and an introduction to the week’s topic. The third phase, “Work Phase”, included a
lesson from the Second Step curriculum. The fourth phase, “Processing Phase”, included a recap
of the material learned that day as well as how it related to the previous lessons, and a discussion
of how the day’s topic could be applied in the students’ daily lives. The fifth and final phase
“Closure” included a discussion of the weekly homework activity.
Each lesson in the Second Step curriculum copied directly into an agenda. Sample
agendas can be seen below in Appendix D. The Second Step curriculum included opportunities
for explicit teaching of problem-solving skills, empathy, and anger management. Practice
opportunities through use of role play were also included. The social skills group target
variables, positive peer interactions and positive adult interactions, were addressed using the
Second Step curriculum as the program addressed the building of skills for using empathy and
problem-solving with others. Student problem solving is increased when interventions
specifically target problem solving (Frey et al., 2000).
Social validity
Social validity data was assessed through a question on the post-survey asking how
helpful that they felt the social skills group was. Social validity was also assessed informally by
TIER II BEHAVIOR 113
asking teachers if they noticed positive changes in peer and adult interactions. Teachers
indicated that for some students, positive changes were seen; however, for other students, no
positive changes were seen.
Results
Pre-survey and post-survey data are provided by individual student as well as by a group
average for Students’ Self-Reported Positive Peer Interactions, Students’ Self-Reported Positive
Adult Interactions, Teacher Reported Positive Peer Interactions, and Teacher Reported Positive
Adult Interactions. All data are displayed in Figures 1,2,3, and 4. To support visual analysis,
summary statistics were calculated and are displayed in Table 1. Standard deviation, mean and
effect size were calculated for the group.
The student self-reported survey provided questions regarding specific positive or
negative interactions and their likelihood of performing those interactions as well as questions
regarding beliefs surrounding positive peer or adult interactions. The rating scale provided,
contained three possible scores: 0 (Not True), 1 (Sometimes True), and 2(Often True). Teacher
rating surveys were also provided before and after intervention implementation. Teacher surveys
contained the same rating score as students’.
Figure 1 shows the results of the individual and group average Student Self-Reported
Positive Peer Interactions. Students’ ratings about their use and knowledge of Positive Peer
Interactions showed an average decrease from pre-survey to post-survey. This indicated that
students did not feel that they made positive increases in positive peer interaction items such as
getting along with peers and being concerned for the welfare of peers. On the pre-intervention
survey the students indicated a mean score of 20.29 (SD= 4.68). Student post-intervention self-
TIER II BEHAVIOR 114
reported survey data indicated a score of 19.29 (SD=5.47). The goal of a rating 22.4 or higher
was not met by any individual student or by combined student average data.
Figure 1: Students’ Self-Reported PPI
Figure 2 shows the results of the individual and group average for the Teacher Reported
Positive Peer Interactions variable. Teacher reported Positive Peer Interactions show increases
from pre to post-intervention for six of the seven participating students and one student showed
scores that stayed the same across surveys. Average pre-survey data show a score of 14.14 (SD=
4.68) and post survey data show an average score of 15.8 (SD= 5.47). The goal for Teacher
TIER II BEHAVIOR 115
Reported PPI is noted by the line in Figure 2. Although the goal was not met, overall positive
increases were made from pre to post-intervention.
Figure 2: Teacher Reported PPI
Figure 3 indicates the results of the individual and group average Student Self-Reported
Positive Adult Interactions. Students’ ratings about their use and knowledge of Positive Adult
Interactions showed an average decrease from pre-survey to post-survey. This indicated that
students did not feel that they made positive increases in positive adult interaction items such as
listening to and getting along with teachers or complying with teacher direction. On the pre-
intervention survey the students indicated a mean score of 6.43 (SD= 2.23). Student post-
intervention self-reported survey data indicated a score of 6.14(SD=2.41). The goal of a rating
TIER II BEHAVIOR 116
and 8.0 or higher was met by two students, both on pre and post-survey data. The combined
student average data did not meet the goal and showed a decline from pre to post-survey data.
Figure 3: Students’ Self-Reported PAI
Figure 4 shows the results of the individual and group average for the Teacher Reported
Positive Adult Interactions variable. Teacher reported Positive Peer Interactions show increases
from pre to post-intervention for six of the seven participating students with the remaining
student maintaining their score from pre to post survey. Average pre-survey data show a score
of 4.43 (SD=1.51) and post survey data show an average score of 5.57 (SD=1.4). The goal for
TIER II BEHAVIOR 117
Teacher Reported PAI is noted by the line in Figure 2. Although the goal of 8 for Teacher
Reported PAI was not met, overall positive increases were made from pre to post-intervention.
Figure 4: Teacher Reported PAI
Table 2 shows summary statistics regarding student and teacher ratings on Positive Peer
Interactions and Positive Teacher Interactions. The summary statistics support the visual analysis
by showing small effect sizes for the student ratings of PPI and PAI and moderately-large effect
sizes for the teacher ratings of PPI and PAI. Students’ self-ratings about each variable and
teacher ratings about each variable were averaged together. The data reveal a decrease in Student
Positive Peer Interaction and Student Positive Adult Interaction from the pre-survey to the post-
survey data for student self-ratings, indicating the post-intervention goal was not met. The data
TIER II BEHAVIOR 118
reveal an increase in Teacher ratings of Positive Peer Interaction and Teacher ratings of Positive
Adult Interaction from the pre-survey to the post-survey data.
Table 2
Summary Statistics
Pre-Assessment Post-Intervention Effect
Size
Mean SD Mean SD
Student ratings of PPI skills
20.29 4.68 19.29 5.47 -0.21
Student ratings of PAI skills
6.43 2.23 6.14 2.41 -0.13
Teacher ratings of PPI skills
14.14 5.76 17.71 5.94 0.62
Teacher ratings of PAI
4.43 1.51 5.57 1.40 0.75
Discussion
One limitation of this small group behavioral intervention was the difficulty building and
maintaining a routine for the group meeting due to inclement weather with school delays and
closings. The multiple school closings due to poor weather conditions changed the social skills
group schedule when the school closing occurred on a Monday. Additionally, due to the social
skills group occurring in the early morning, school delays also affected the social skills group
schedule. Three of the students in the group had been medically diagnosed as children with
Autism Spectrum Disorder. The change in schedule when delays or closing occurred changed
the participating students’ routine. Several of the children displayed anger and/or anxiety when
the schedule was changed, and often, their behavior was not typical the next time that the group
met, due to their anger or anxiety.
Another limitation of the small group social skills intervention was the possibility of
measurement bias due to student self-reported data and teacher rating data for measuring the
results of the intervention. Students may have been reluctant to relay accurate information about
TIER II BEHAVIOR 119
themselves on the surveys for fear of the information reflecting badly upon them or eliciting
disciplinary action by the school. Interestingly, teacher data showed positive changes from pre-
intervention to post intervention for both PPI and PAI; changes that were not reflected in student
data. It is a positive finding that teacher data did indicate change.
Also, despite research-based surveys that the pre and post teacher and student surveys
were taken from, the surveys were modified by the intern school psychologist, possibly reducing
the reliability and validity of the measures. In the future, I will use direct methods of
measurement to determine student behavior change. Additionally, this consultation did not
include a formal means of progress monitoring, as pre and post surveys were given, but not
“during-intervention” surveys. In the future I will address progress monitoring by utilizing direct
measures, conducted throughout the intervention, to measure student change in order to inform
modifications made to the intervention during intervention implementation.
Further, students selected for the group were reported by teaching teams rather than
through the use of universal screening data. Lack of universal screening prohibited the intern
school psychologist from providing data to support whether or not the Tier I, core behavior
management system of the school was effective. This increased the difficulty in supporting
student movement from Tier I to Tier II behavior intervention, as the moves were referral-based
rather than data-based.
Formal social validity assessments for teachers, parents, and students would have
strengthened this consultation. Social validity was informally assessed by asking students during
the final session how much they enjoyed participating in the group and if they learned skills for
positively interacting with peers and adults. Social validity was assessed from teachers by
asking on the post-survey if they felt that the intervention was helpful to students. However,
TIER II BEHAVIOR 120
more extensive and systematic questioning regarding teacher beliefs about the intervention
would have provided stronger support for its effectiveness. Additionally, it would have been
ideal to include parent social validity data to determine if generalization of skills, into the home,
had occurred.
Despite the limitations and lack of positive effects displayed through student-reported
self-data, the consultation provided the intern school psychologist with an excellent opportunity
for collaboration in planning for and implementing a small group intervention. Meeting weekly
with a small group of students allowed for the intern school psychologist to form relationships
with students and provide targeted intervention support through the use of the Second Step
curriculum. Collaboration and problem solving were both essential components of my Personal
Model of Practice which were skills which I was able to utilize and improve on through this
consultation.
TIER II BEHAVIOR 121
References
Beland, K. (1997). Second step: a violence prevention curriculum: middle school/junior
high. Seattle, WA: Seattle Institute for Child Advocacy.
Fabes, R.A., Hanish, L.D., Martin, C.L., Moss, A., & Reesing, A.R., (2012). The effects of
young children’s affiliations with prosocial peers on subsequent emotionality in peer
interactions. British Journal of Developmental Psychology, 30, 569-585.
Frey, K.S., Hirschstein, M.K., & Guzzo, B.A. (2000). Second step: Preventing aggression by
promoting social competence. Journal of Emotional and Behavioral Disorders, 8(2),
102-112.
Kidron, Y. & Fleischman, S. (2006). Promoting adolescents' prosocial behavior. Educational
Leadership, 63(7), 90-91.
Ladd, G.W. & Profilet, S.M. (1996). The child behavior scale: A teacher-reported measure of
young children's aggressive, withdrawn, and prosocial behaviors. Developmental
Psychology, 32(6), 1008-1024.
Osher, D., Sprague, J., Weissberg, R.P., Axelrod, K, Keenan, S., Kendziora, K., & Zins, K.E.
(2008). A comprehensive approach to promoting social, emotional, and academic growth
in contemporary schools. In A. Thomas & J. Grimes (Eds.), Best practices in school
psychology V (pp. 1263-1278). Bethesda, MD: National Association of School
Psychologists.
Sutherland, K.S., Conroy, M.A., Vo, A., Abrams, L., & Ogston, P., (2013). An initial evaluation
of the teacher-child interaction direct observation system: measuring teacher- child
interaction behaviors in classroom settings.
TIER II BEHAVIOR 122
Appendix A
Student Behavior Survey: Self-Report
Student: ____________ Date: _____________ Team: _______________________
Instructions: Please answer these questions as honestly as possible. Your answers will be
confidential and will not hurt your relationship with the school or teachers. Your participation in
the survey is voluntary. You may discontinue this survey at any time.
Please select the answer you feel most appropriately represents your behavior during the past month.
Not True=0 Sometimes True=1
Often True=2
I am concerned when something is bothering another classmate
It is never okay to be aggressive toward another student
If you are angry at someone it is okay to tease them
It is wrong to threaten other people
I get along with other students
I listen to what other students say without interrupting
If I have a conflict with a classmate I will talk with them to reach a compromise so that both of us are happy
I enjoy working in groups with classmates
I control my temper when I get angry
I argue with classmates
Classmates say that I annoy or irritate them
If I have something to say, it is okay to disrupt a classmate’s activity
When a classmates is upset, I will help them
It is okay to keep bothering a classmates even when they are clearly upset
Behavior toward Teachers
I listen to what my teachers say without interrupting
When I do not agree with a comment, I will argue with a teacher
I like my teachers
I listen to my teachers’ requests
If I have a problem in a class, I calmly use my words to talk to the other student and/or teacher to find a solution to the problem.
TIER II BEHAVIOR 123
Appendix B
Child Behavior Survey: Teacher Report
Student: ____________________________________________ Date: _______________________
Teachers Completing Survey: ____________________________ Team: _______________________
Instructions: Please circle the answers to the following questions about your student. The results of this checklist will help monitor the progress of your student as they participate in a small social skills group. Please select the answer you feel most appropriately represents your student’s behavior during the past month. Not True=0 Sometimes
True=1 Often True=2
Tends to react to classmates’ distress by teasing them or making things worse
Seems concerned when classmates are distressed
Is an aggressive child
Taunts and teases classmates
Threatens classmates
Is kind toward classmates
Listens to classmates
Compromises in conflicts with classmates
Is cooperative with classmates
Loses temper easily in conflicts with classmates
Argues with classmates
Is friendly toward classmates
Annoys or irritates classmates
Disrupts classmates’ activities
Shows concern for moral issues (e.g. fairness, welfare of others)
Offers help or comfort when classmates are upset
Will continue to bother or hurt classmates even when they are clearly upset
Behavior toward Teachers
Is kind toward teachers
Argues with teachers
Is friendly toward teachers
Is cooperative with teachers
Generates good-quality solutions to interpersonal problems
TIER II BEHAVIOR 124
Appendix C
Dear Parent/Guardian, Your child has been invited to participate in a social skills group at school. This group
has been created to provide students with extra time to learn social skills with a small
group of peers. Students are nominated to participate in social skills groups in a variety of
ways, but are often referred by their Teacher and/or School Counselor. The social skills group
will use a social skills curriculum that teaches a variety of social skills that include topics like
bullying, social media, anger management, conflict resolution, and friendly communication. The
topics covered will ultimately promote students’ skills in interacting positively with peers and with
adults.
During the group, social skills are taught using modeling, role-playing, and discussion. In
addition to the positive impact of learning social skills, students who participate in small groups
often develop a stronger sense of belonging to a social group and a more positive attitude about
school. Your child’s group will meet during school hours for approximately 6 weeks. It will be
taught by XXXXX, PBSS (Positive Behavior Support Specialist) and Katie Newman, Intern
School Psychologist at Amelia Middle School. The group will meet one time each week and the
group will last approximately 30-40 minutes. Your permission is requested for your student to
participate in the social skills group. Please sign and return this form to the front office if your
child has permission to participate in this activity. If you have any questions, please do not
hesitate to contact Ms. Newman or Ms. XXXXX at XXXXX Middle School, XXX-XXX-XXXX.
Sincerely, XXXXX
PBSS Katie Newman
Intern School Psychologist
My child, ______________________________has my permission to participate in a social skills group. Parent/Guardian Signature:________________________ Date:___________
TIER II BEHAVIOR 125
Appendix D
Social Skills Group Agenda
Week 1- Introduction to the Group & Lesson 1
Energizing Phase (5 mins)
Candy activity Assign get to know you questions to each color M&M. Hand out M&M's or skittles to the group, but don't let them eat the candy. Blue/Purple- Wild Card (share anyone that you chose) Red- Favorite Hobbies Yellow- Favorite Movies Green- Favorite Foods Orange: Favorite Places to Travel Brown: Most memorable or embarrassing moment
Advanced Organizer (9 min) Create group rules together Confidentiality
-Explain that we do not talk about anything that went on in this group outside of this time. If you mention harming yourself or others confidentiality will have to be broken. We can talk about what we do, but not what others do
Explanation/ Purpose of this group o Explain to them we are here to develop stronger
relationships and to become leaders at the school. Right now there is a problem with forming and maintaining positive relationships with peers and adults.
What we will talk about over the coming weeks o Review skills they will learn in this group: friendship skills,
empathy skills, anger management, problem solving. Work Phase (12 mins) PreTest
Processing Phase (3 mins) Question & Answer time
Closure (3 mins) Review that we will be having “homework” assignments in the coming weeks
TIER II BEHAVIOR 126
Social Skills Group Agenda
Week 2- Lesson 1 (Understanding the Problem)
Energizing Phase (5 mins)
I’m Unique” Ask each person to share one thing that makes him or her unique.
Advanced Organizer (9 min)
Getting To Know You Jenga Introduction to “Understanding the Problem”
Work Phase (12 mins)
Quiz on key terms (Interpersonal conflict, Interpersonal violence) Lesson 1:
o What does the word interpersonal conflict mean? o What are some different types of conflict people your age
might have? -Ex. Conflicts with teachers, conflicts with
friends, conflicts with peers o What are different ways people respond to interpersonal
conflict? o What is Interpersonal violence? o What is an acquaintance? o What are some examples of acquaintances?
Processing Phase (3 mins)
Question & Answer time
Closure (3 mins)
Television Log…We will review your responses next week. Preview of next week: Empathy
TIER III ACADEMIC 127
A Tier III Intervention to Improve the Early Reading Skills of a Fourth Grade Student
Sarah was a fourth grade student in a rural Ohio elementary school who had received
intensive Tier III intervention in kindergarten, third, and fourth grade. It was documented that
Sarah had a history of Tier II intervention support starting in kindergarten in the areas of
phonemic awareness and alphabetic principle as well as in early mathematics skills. In the
current year, Sarah was receiving Tier III support in early mathematics skills two to three times
per week for fifteen to twenty minutes per session using the Strategic Math Series intervention,
in addition to automatic Tier II intervention in the area of mathematics computation and
concepts/applications, and Tier I core mathematics curriculum. She was also receiving
automatic Tier II intervention in the area of reading consisting of 30 minutes daily of Corrective
Reading intervention, as well as Tier I core instruction in reading.
It was recommended that supports for Sarah in the area of reading be intensified further
due to lack of gains made on oral reading fluency (ORF) and comprehension (DAZE) progress
monitoring assessments. The team determined that due to specific skill deficits in reading
fluency and comprehension, the Reread-Adapt and Answer-Comprehend (RAAC) intervention
(Appendix A) would provide additional repeated reading practice as well as comprehension
questions to build comprehension skills for Sarah. The RAAC intervention took place three
times per week for 15-20 minutes per session in the classroom of the reading specialists’. The
reading specialist implementing the intervention provided Sarah with a reading passage during
each session. Sarah was asked to read the passage aloud and answer generic questions about the
story following the first read. Sarah was then asked to reread the passage a minimum of two
times and a maximum of four times, until she reached the pre-established goal indicated on the
page. The teacher provided error correction for words that Sarah did not know or pronounced
incorrectly. The student was then asked to answer the cue card questions. Scores on the reading
TIER III ACADEMIC 128
fluency portion of the intervention as well as number of comprehension questions answered
correctly were recorded for each session.
Results of the intervention showed that Sarah failed to make appropriate gains in reading.
Additionally, she was not making appropriate gains in math or in writing. A special education
referral was completed by the classroom teacher to initiate an evaluation to determine if there
was a suspected disability present. Through the evaluation process, it was determined that a
suspected disability was present and Sarah began to receive special education services as a child
with a Specific Learning Disability.
Method
Participants and roles
Following the review of the fourth grade Tier II reading intervention progress monitoring
data, the school psychologist identified students who were not making progress on DIBELS Oral
Reading Fluency (DORF) or DAZE comprehension measures. Sarah, a fourth grade student
began receiving automatic Tier II support using the Corrective Reading intervention, due to
inadequate progress made across her third grade year on ORF and DAZE benchmarks. Due to
scores in the “Need of Intensive Intervention”, on the Fall of fourth grade benchmark in ORF
and DAZE, it was determined by the team that Sarah receive Tier III intervention support, in
addition to the Tier II supports in place.
Sarah was a fourth grade, nine year old Caucasian female attending a rural public
elementary school in Ohio. She had attended the current elementary school since November of
her kindergarten year. Sarah had been receiving Tier II and Tier III interventions since the
winter of her kindergarten year in both mathematics and reading. The school psychologist
contacted the student’s parent to request permission for continued support for the student.
TIER III ACADEMIC 129
The consultation team consisted of the principal, school psychologist, school psychology
intern, fourth grade teacher, and two reading specialists. In collaboration with the consultation
team, it was determined that the Tier III intervention support in reading for Sarah would include
the RAAC intervention. Permission was gathered to implement the intervention and progress
monitor the target student from the supervising school psychologist and the student’s parent.
Setting
The Tier II and III interventions were implemented in the reading specialists’ classroom.
Sarah was receiving Tier II reading intervention in a small group setting with six, fourth grade
peers. She was also receiving Tier II and Tier III mathematics intervention support which took
place in a small group and individualized basis, respectively, in the general education classroom,
at the time that the team met to discuss the implementation of Tier III reading supports. The Tier
III reading support began in the fall, following the DIBELS fall universal screening. Sarah
participated in the individualized intervention during non-core class time three times per week
for fifteen to twenty minutes per session in the reading specialists’ classroom.
Target Variables
The target variables selected were determined in collaboration with the team following
the fall universal reading screenings. The universal screening measures utilized were the
DIBELS Oral Reading Fluency assessment (DORF) and the DAZE measure of reading
comprehension. DIBELS assessments provide data for two purposes; to determine those
students who are “at risk” for failure in reading, and to evaluate intervention effectiveness
through progress monitoring (Elliot, Huai, & Roach, 2007). Universal screening data indicated
that Sarah’s DORF score was significantly below levels of peers and below the fall benchmark.
Sarah’s DAZE score was also below the fall benchmark. Teacher interview information and
TIER III ACADEMIC 130
parent concerns confirmed that reading fluency and comprehension were prioritized areas of
concern. The DORF measure served as the weekly progress monitoring tool.
Reading Fluency. Reading fluency is defined as the ability to “read text with speed,
accuracy, and proper expression” (NICHHD, 2000, p. 3-1). The National Reading Panel
identifies reading fluency as one of the five necessary components for reading skills (NICHHD,
2000). Fluent reading is a necessary skill that students must possess for academic success as
well as for success after K-12 education (Lo, Cooke, & Starling, 2011). Reading fluency is
comprised of several components including word attack skills, decoding skills, and word
identification (Lo, Cooke, & Starling, 2011). To measure the student’s progress in reading
fluency, the DIBELS DORF assessment was administered weekly and served as the primary
target variable for the determined intervention.
Reading Comprehension. A secondary target variable for the intervention chosen was
reading comprehension. Research has shown that good readers monitor their comprehension as
they read, as opposed to poor readers (Therrien, Gormley, Kubina, 2006). Without monitoring
comprehension, often poor readers do not comprehend the text being read (Therrien, Gormley,
Kubina, 2006). A strong correlation between reading fluency and reading comprehension has
been demonstrated through research (Swain, Leader-Janssen, & Conley, 2013). Although
evidence exists that the rate or oral reading is in indicator of students’ comprehension level, it is
still suggested that reading comprehension assessments be used for universal screening purposes
(Shapiro, 2011) thus ORF was used as a measure of reading comprehension for weekly progress
monitoring purposes but quarterly universal screening benchmark DAZE assessments served as
periodic review of the student’s progress in reading comprehension.
Inter-Scorer Agreement
TIER III ACADEMIC 131
The intern school psychologist observed a progress monitoring DORF assessment with
the student. The progress monitoring was being taken by the classroom teacher. Inter-scorer
agreement was calculated by dividing the total number of agreements by the number of
agreements plus disagreements. Inter-scorer agreement for baseline was taken on 09/20/2013.
Inter-scorer agreement for intervention was taken on 2/24/14. Inter-scorer agreement was
calculated during the single ORF progress monitoring probe completed during the baseline.
Inter-scorer agreement was data was collected for one of the ORF progress monitoring probes
conducted during the intervention phase, which accounted for 4% of the total weekly ORF
assessments completed. See Table 1 for inter-scorer agreement data.
Table 1
Inter-Scorer Agreement Data
Date Inter-scorer agreement
09/20/2013
2/24/2014
2/24/2014 100 100%
100%
Goals and Decision Rules
The goals decided upon for this consultation by the team were determined by Sarah’s
performance on baseline assessments and by the grade level benchmark norms for the fourth
grade. The ORF goal of reading 115 words correct per minute by the spring of the fourth grade
year was determined. The progress monitoring data for Sarah was graphed and an aimline was
created to assist with visual analysis of the data. It was determined that if Sarah had scored
three data points below the aimline, the team would meet to discuss changes to the intervention.
Functional Hypothesis
TIER III ACADEMIC 132
The progress monitoring data collected in oral reading fluency suggested that the function
of Sarah’s poor performance in reading fluency was a skill deficit. Teacher interview
information confirmed that Sarah had not yet mastered the skill of reading fluency. Sarah was
unable to read connected text in a fluency measure and therefore experienced difficulty in
extracting meaning from the text read (reading comprehension).
Accountability Plan
An AB design was utilized to monitor Sarah’s progress while receiving Tier III support
through the RAAC intervention. The baseline condition (A) consisted of 1 data point collected
at the fall DIBELS benchmark period. It was determined by the team that due to Sarah’s
intensive need for additional support, as supported by third grade progress monitoring and
benchmark data that Tier III intervention should begin immediately. The intervention phase (B)
consisted of the use of the Tier III intervention support using the RAAC intervention.
Intervention Procedures
Baseline. The fall universal screening of DIBELS served as the baseline data for this
consultation. Sarah was administered the DORF and DAZE fourth grade assessments. The
DORF assessment was one minute long. Three DORF assessments were given and the median
score was recorded. The DAZE assessment was three minutes in length. The DIBELS
assessment directions and steps for administration were scripted, aiding with standardization
when assessors varied.
Reread-Adapt and Answer-Comprehend (RAAC). Reading practice is the basis for
building reading fluency (NICHHD, 2000). The RAAC intervention is one which combines
repeated reading and question generation techniques and allow for extensive reading practice
(Therrien, Gormlet, & Kubina, 2006). Repeated reading is a process that has strong research
support for increasing reading fluency (Vadasy & Sanders, 2008). The National Reading Panel
TIER III ACADEMIC 133
has reported that repeated reading is the only well-supported reading fluency intervention for
improving fluency (Vadasy & Sanders, 2008). Three essential components of repeated reading to
be effective are that the student should orally read the passage to an adult, the adult should
provide error correction, and the student should reread the passage until he or she has met the
predetermined level of fluency desired (Therrien, Gormlet, & Kubina, 2006). Question
generation is a strategy for improving comprehension in which the student generates and answers
questions while reading (Therrien, Gormlet, & Kubina, 2006). Research in the area of question-
generation has found that when students are provided with prompts such as the “story-structure
questions” provided in the RAAC, to cue the generation of questions, outcomes were greater
than for those without such prompts (Therrien, Gormlet, & Kubina, 2006).
It is recommended that the RAAC intervention be used with students whose instructional
level in reading is between the first and third grade (Therrien, Gormlet, & Kubina, 2006).
Although Sarah was a fourth grade student, her instructional reading level was lower than her
actual grade level, making the RAAC intervention a good match with her current skills and
deficits. Passages at the third grade level were provided to Sarah as she completed the RAAC
intervention. A sample RAAC student passage sheet can be seen in Appendix B.
Adherence data. The paraprofessional kept a log of intervention adherence data and would
mark an “I” on an intervention log after each completed session adhering to the intervention
script.
Table 2
Intervention Adherence Data
Date Corrective Reading (TII) RAAC (TIII)
10/23/13 100% 100%
TIER III ACADEMIC 134
12/9/13 100% 100%
1/13/14 100% 100%
3/07/14 100% 100%
Social validity
Social validity was informally assessed through interviews with the consultation team
members. The teacher, paraprofessional, parent, and student were all asked if they liked the
intervention and if they felt that it was effective. All agreed that the intervention was somewhat
helpful for improving reading fluency and comprehension; however they still had concerns about
Sarah’s progress. The paraprofessional noted that the intervention was easy to implement due to
the scripted nature of the materials. Although progress monitoring did show improvements
overall in reading fluency following intervention, the team continued to have concerns with
Sarah’s progress. Although previous scores had been variable, in January, Sarah’s score on ORF
measures fell below the aimline, and remained below.
Results
Baseline and progress monitoring for Sarah’s ORF and DAZE assessment scores are
depicted in Figures 1 and 2 below. As demonstrated in the figures, data show overall
improvements in DORF measures, however recent data showed variable progress below the
aimline. Data showed little improvement on the DAZE measure from the Fall Benchmark 1 to
the Winter Benchmark 2 assessments.
Reading Fluency
Figure 1 displays the oral reading fluency scores for Sarah during baseline and during the
intervention phase. Baseline ORF assessments showed that Sarah could read an average of 19
correct words per minute. The Fall DIBELS Benchmark goal was 90 correct words per minute.
As seen in the graph below, for Sarah to reach the goal of 115 words correct per minute and
TIER III ACADEMIC 135
close the gap between her baseline score and the benchmark goal, her scores on the DORF
assessments should follow the aimline displayed by the small-dashed line. Visual analyses of the
data show that for the large-dashed trend line of Sarah’s scores, her reading fluency scores were
not following the expected trajectory. Although increases in oral reading fluency were seen
overall, scores were variable and recent data showed scores significantly below the aimline.
Figure 1: Student DORF Progress Monitoring Data
Reading Comprehension
Figure 2 displays the student’s scores on the DAZE benchmark assessments. Baseline
data consisted of one data point taken as the fall universal screening benchmark assessment. The
DAZE data collected during the intervention phase was taken during the winter universal
TIER III ACADEMIC 136
screening benchmark assessment. The small-dashed line represents the aimline in which Sarah’s
scores should follow in order to close the gap between her baseline score and Spring DIBELS
DAZE benchmark goal. The large-dashed line represents Sarah’s actual pattern of scoring
(trendline). Visual analysis of Sarah’s scores shows her progress as significantly below the
aimline and below the Spring Benchmark goal. Additionally, between the fall and winter
benchmarks, Sarah made little progress in comprehension.
Figure 2: Student DAZE Universal Screening Benchmark Data
In support of visual analysis, summary statistics were calculated. Summary statistics can
be viewed in Tables 3, 4, and 5 below. The percentage of non-overlapping data (PND) for each
variable was determined by identifying how many data points did not overlap with the highest
baseline data point. The effect size for each variable was calculated by dividing the difference of
TIER III ACADEMIC 137
the baseline mean and the last intervention phase mean by the standard deviation of the baseline.
For ORF, a large effect size of 7.43 was found with the percentage of non-overlapping data being
100%.
For DAZE, the summary statistics could not be calculated due to only one baseline point.
However, for ORF, three baseline data points were gathered by having Sarah read three passages
during the Fall Universal screening.
Overall improvements were seen in Sarah’s ORF assessment scores, although not enough
improvement was seen to close the gap between Sarah’s current scores and the Spring
Benchmark DIBELS goal for ORF. For the DAZE assessment, Sarah’s scores made a minimal
increase from the fall to the winter benchmarks. Reports from the reading specialist indicated
satisfaction with the intervention. Using goal attainment scaling measures (where ‘0’ indicates
no progress made toward the goal, ‘1’ indicates progress made toward the goal, and ‘2’ indicates
the goal was met), Sarah received a ‘1’ in both variables.
Table 3
Means and Standard Deviations of Graphed Variables (Baseline Phase)
Variable (Behavior) Average of ORF progress monitoring scores (Baseline Phase)
Number of baseline
data points
Mean Standard Deviation
ORF 3 19.3 5.69
DAZE 1 11 N/A
TIER III ACADEMIC 138
Table 4
Means and Standard Deviations of Variables (Intervention Phase)
Variable (Behavior) Average of ORF progress monitoring scores (Intervention Phase)
Number of intervention
data points
Mean Standard Deviation
ORF 15 61.6 13.52
DAZE 1 12 N/A
Table 5
Percent of Non-overlapping Data (PND), Goal Attainment Scaling (GAS), and Effect Size for
Graphed Variables
Student PND GAS Effect Size
ORF 100% 1 7.43
DAZE 100% 1 N/A
Discussion
There were several limitations for this consultation. One limitation was the lack of Inter-
Observer Agreement data collected across intervention implementation. The intern school
psychologist collected agreement data for 4% of the assessments, as DIBELS DORF progress
monitoring data was collected weekly on a day that the intern was assigned to work in another
school building. In ideal conditions, inter-observer agreement should be collected during at least
20% of observation.
A second limitation for this consultation was the accountability design that was used.
Accountability designs (A-B) limit internal validity arguments and conclusions that the
TIER III ACADEMIC 139
independent variable was the cause for the change in behavior (O’Neill et al., 2011); however,
the accountability design can be used for decision making in educational settings.
Another limitation was that although Sarah was receiving intervention in reading and
mathematics across the three tiers, and making some progress for increasing ORF score from
baseline, the teacher continued to have significant concerns for Sarah’s reading fluency, reading
comprehension, mathematics calculation, and mathematics problem solving. Due to the
student’s discrepant achievement, inadequate progress, and educational need based on extensive
progress monitoring and assessment data, despite the intensive, comprehensive intervention
services that Sarah was receiving, the team determined that they suspected a disability in the
category of Specific Learning Disability. An evaluation was conducted and the student was
found to qualify for special education services. As the evaluation was conducted, the team
reviewed the data from intervention, assessments, parent, and teacher information in order to
answer guiding questions for the four components needed to make a determination if a child
qualifies as having a Specific Learning Disability (See attached Evaluation Team Report)
Establishing Discrepant Achievement
To determine whether Sarah showed discrepant achievement from same aged peers,
Sarah’s scores on the 3rd grade OAA were analyzed. Sarah scored below the state grade-level
standards for both the reading OAA and math OAA. Sarah receives scientifically-based Tier 1
core instruction in reading by a highly qualified teacher using the Reading Street curriculum
which aligns with the state common core standards. Similarly, Sarah also receives scientifically-
based Tier 2 core instruction in math by a highly qualified teacher using the Go Math curriculum
which aligns with the state common core standards. Sarah’s scores on universal screening
measures using the DIBELS DORF assessment on the winter benchmark screening was 68
TIER III ACADEMIC 140
words correct per minute compared to the average score of 108 words correct per minute for her
classmates. The DIBELS DORF winter benchmark goal was 103 words correct per minute
providing evidence that the core instruction was effective for most students. On the AIMSweb
Concepts and Applications assessment, Sarah scored a 6, when AIMSweb winter norms indicate
the goal of 17. Sarah’s score fell below the 10th
percentile. On the AIMSweb Computation
assessment, Sarah scored a 34, when the winter benchmark norm was 43. Sarah’s score fell
below the 50th
percentile. Peers in Sarah’s class were similar in demographics as the school was
composed of students from lower-middle class families in a rural area outside of Cincinnati. The
student displayed no attendance concerns. This information allowed the team to establish that
Sarah did display discrepant achievement in reading and mathematics compared to same-
age/grade level peers.
Establishing Inadequate Progress
The RAAC intervention, involving repeated reading and question generation
components, has research support as an evidence-based intervention for improving student
reading fluency and comprehension (Vadasy & Sanders, 2008). The Tier II and Tier III services
provided for Sarah in both reading and math did match the student need. Tier II intervention for
mathematics and reading occurred five days per week for 30 minutes per day in a small group
setting. Tier III intervention in mathematics and reading occurred 3 days per week for 20
minutes per session in a one-on-one setting. The interventions, as evidence by collected
adherence data, were implemented with fidelity. Research indicates that intervention length
should be a minimum of 4-6 weeks. All of the interventions in which Sarah participated met the
adequate length criteria. Systematic, scientifically-based progress monitoring did occur during
instruction in the area of Oral Reading Fluency. Progress monitoring data was collected weekly.
TIER III ACADEMIC 141
DIBELS DAZE comprehension assessments were conducted three times per year, as were
AIMSweb Concepts/Applications and Computation. These assessments provided baseline data
from the fall benchmark assessment as well as intervention data for the winter and spring
administrations of the assessment. Best practice would indicate that progress monitoring for
these assessments would occur more frequently than three times per year. Progress monitoring
data was used to create decision rules for changes made to intervention, depending on student
scores. Sarah began receiving Tier II intervention following fall universal screening in reading
and mathematics which indicated scores significantly below the benchmark goals. Sarah was
placed into Tier II small groups in reading and math to target specific skill needs. Data showed
little improvements in scores on DIBELS and AIMSweb assessments prompting an increase in
support to Tier III intervention. While receiving Tier III reading intervention support, Sarah
continued to make inadequate progress as evidenced by poor performance on weekly progress
monitoring assessments prompting the team to suspect that Sarah has a disability. Data from
assessments including DIBELS (DORF and DAZE), AIMsweb (Concepts and Applications and
Computation), NWEA MAP, and Ohio Achievement Assessments (OAA), were used to inform
decisions on intervention placement for Sarah. Sarah begin receiving targeted intervention
supports in Kindergarten and despite increases in intervention intensity, did not make adequate
progress, as evidenced grades and the above-mentioned assessments. Progress monitoring data
was collected frequently and data were shared with parents on quarterly report cards; further,
grade information can be viewed by parents at any time online. As evidenced by Sarah’s lack of
progress in reading and mathematics despite receiving intensive intervention support, the team
established that Sarah was making inadequate progress.
Establishing Educational Need
TIER III ACADEMIC 142
Despite the provided intervention, the degree to which Sarah’s scores on measures of
reading and mathematics were lower than that of other students her age/grade, was sizable. As
evidenced by ORF and DAZE assessment data; the trendline of Sarah’s reading scores were
significantly below the trajectory of the aimline, needed to close the achievement gap to meet
benchmark goals. Despite not closing the gap between her baseline score and the end of the year
benchmark goal, Sarah did make some improvements during the intervention phase over her
performance during the baseline condition. As evidenced by ORF data, displayed in Figure 1,
Sarah did make improvements in the number of words correct per minute that she could read.
The improvements however were not significant enough or at a rate sufficient enough to improve
to the goal within a reasonable period of time. As evidence by DAZE data, displayed in Figure
2, Sarah did not make significant improvements in the area of reading comprehension. As the
trendline of Sarah’s scores displays in Figure 2, it can be seen that the rate of improvement for
Sarah’s comprehension score was not at a level which would allow her to catch up to the end of
the year goal or to levels of peers. Interview information from the teacher and the parent
suggested that Sarah needed ongoing supports and services of sufficient intensity, in the form of
special education, in order to be successful in school. As evidenced by the information provided
regarding the severity of Sarah’s needs in reading and mathematics, the team determined that
Sarah displayed an educational need for special education services.
Putting it together for Entitlement Decision
All aspects of discrepant achievement were documented through frequent assessment
using DIBELS, AIMSweb, OAA, and NWEA MAP. Assessments choses were matched to
student areas of need and were chosen by the team as accurate measures of Sarah’s progress. All
aspects of instructional need were determined through collecting and analyzing various sources
of data including grades, teacher report, parent report, standardized assessments, progress
TIER III ACADEMIC 143
monitoring data, which informed decision making toward educational need. All aspects of
educational progress were documented through quarterly benchmark screening assessments,
weekly progress monitoring data, classroom performance, and grades. Data were graphed for
visual analysis. As evidenced by local norms and peer performance on the assessments
conducted, the student’s achievement differed significantly from other students’ with similar
demographic characteristics (see ETR for peer and local norms). Observational data were also
collected using the BOSS observation code and data indicated that Sarah maintained high
academic engagement in the classroom environment This would suggest that attention/off-task
behaviors are not impeding Sarah’s access to the general education curriculum. It was
determined by the consultation team that Sarah demonstrated discrepant achievement, inadequate
progress, and instructional need, as evidenced by her low performance despite extensive and
intensive research-based intervention support provided with fidelity, that demonstrated an
educational need for specially designed instruction. Following the evaluation for special
education, the team determined that Sarah did meet the criteria for qualifying for special
education as a child with a Specific Learning Disability.
This consultation provided experience with intervention planning and linking assessment
to intervention to provide intensive academic interventions to a student struggling in reading,
writing, and mathematics. The consultation provided an opportunity for the intern school
psychologist to be involved with increasing the intensity of services provided to a student
through the three Tiers of the Response to Intervention (RtI) process. The intern school
psychologist was provided the opportunity to use her personal model of practice to guide her
ethical decision making when decisions for possible student special education eligibility through
this consultation. Also, opportunity for parent collaboration and communication was present
TIER III ACADEMIC 144
within this case. Sarah’s father attended a Tier III review meeting where the intern school
psychologist shared Sarah’s progress and the team’s additional concerns with her father. The
father shared his concerns and together a plan for next steps was made. Sarah’s father failed to
attend the ETR meeting. The lack of parental attendance allowed the intern school psychologist
the opportunity to determine Ohio law for obtaining parent signatures on necessary documents so
as to remain within ETR compliance deadlines.
The intern school psychologist utilized problem solving skills when participating with the
team to determine Tier III reading intervention support for Sarah, guided by collected data and
demonstrated need. Problem solving skills were again utilized when determining next steps as
Sarah’s reading and math performance failed to make appropriate gains to levels seen in peers.
The use of problem solving to make accurate decisions in schools was an important goal in my
professional model of practice. This consultation provided a unique learning experience for the
intern school psychologist, and skills acquired and honed will be used in future practice.
TIER III ACADEMIC 145
References
Lo, Y., Cooke, N.L., & Starling, A.L. (2011). Using a repeated reading program to improve
generalization of oral reading fluency. Education and Treatment of Children, 34 (1),
115-140.
Musti-Rao, S., Hawkins, R.O., & Barkley, E.A. (2009). Effects of repeated readings on the oral
reading fluency of urban fourth-grade students: Implications for practice. Preventing
School Failure, 54 (1), 12-23.
National Institute of Child Health and Human Development (2000). Report of the National
Reading Panel. Teaching children to read: An evidence-based assessment of the scientific
research literature on reading and its implications for reading instruction (NIH
Publication No. 00-4769. Washington, DC: U.S. Government Printing Office.
Shapiro, E.S. (2011). Academic skills problems: direct assessment and intervention (4th
ed.).
New York: Guilford.
Swain, K.D., Leader-Janssen, E.M., & Conley, P (2013). Effects of repeated reading and
lestening passage preview on oral reading fluency. Reading Improvement, 50 (1), 12-18.
Therrien, W.J., Gormley, S., & Kubina, R.M. (2006). Boosting fluency and comprehension to
improve reading achievement. Teaching Exceptional Children, 31 (3), 22-26.
Vadasy, P.F. & Sanders, E.A. (2008). Benefits of repeated reading intervention for low-
achieving fourth- and fifth-grade students. Remedial and Special Education, 29 (4), 235-
249.
TIER III ACADEMIC 146
Appendix A
RAAC Intervention Script
Session shall last between 10-15 minutes.
1. Teacher cues the student with the following statement: “Read this story the best
you can and as quickly as you can. Pay attention to what you are reading as you
will need to answer these questions.” (Teacher points to the cue card questions)
2. Teacher presents a cue card containing the generic story structure questions and
teacher prompts the student to read the questions aloud.
3. The student rereads the passage aloud until s/he reaches the pre-established
goal.
a. Each passage is read a minimum of 2 times or a maximum of 4 times.
4. Teacher provides instructional feedback on word errors.
a. If student hesitates on a word for 3 seconds or omits a word or words, error correction
is to be provided immediately.
b. Otherwise, error correction is to be provided after the passage has been read but before
it is reread.
c. Error correction includes in both cases providing the word or words and asking the
student to repeat the error word back.
5. After the student meets the goal or after 4th reading, the teacher prompts the student to answer
the cue card questions orally.
a. If the student is unsure or incorrect on the 1st attempt, the teacher is to prompt the
student to look for the information in the passage is given.
b. If the student is unsure or incorrect a 2nd time, the teacher is to point to the sentence
where the correct answer can be found or inferred.
c. If the student is still unsure/incorrect, the teacher points to the sentence where the
answer can be found or inferred and provides the rationale needed to answer the question.
6. The teacher then asks the factual and inferential comprehension questions that follow the
passage.
Subsequent Sessions: Steps 1-6 are to be repeated for each session until all passages have are
read. Difficulty of the reading material can be adjusted as follows:
If the student is unable to reach the goal in 4 readings for 3 consecutive sessions, the reading
level to be used in subsequent sessions is to be lowered one grade level.
TIER III ACADEMIC 147
-If, for 3 sessions in a row, the student is able to reach the goal in 1 reading, the reading
material to be used in subsequent sessions is to be raised by one grade level.
Question Cue Card
Who is the main character?
Where and when did the story take place?
What did the main character do?
How did the story end?
How did the main character feel?
TIER III ACADEMIC 148
Appendix B
4.1
Tess and Leah were very excited for the sleepover this weekend.
All week long they talked about it at recess and during lunch. They
planned out the entire night and even decided what they would wear.
First, the girls would play with Leah’s older sister’s make-up.
They would pretend they were at the salon and do each other’s hair.
Tess even promised to bring all of her nail polish so they could paint
their toenails.
Leah’s mom said they could order pizza. Tess and Leah decided
that they would get cheese and mushrooms on the pizza. Leah’s mom
said she would take them to rent a movie from the store. They both
agreed to get a scary one.
The girls were so excited about the sleepover. They had been
waiting for this weekend for weeks and knew it was going to be so fun.
TIER III ACADEMIC 149
Tess and Leah were very excited for the sleepover this weekend.
All week long they talked about it at recess and during lunch. They
planned out the entire night and even decided what they would wear.
First, the girls would play with Leah’s older sister’s make-up.
They would pretend they were at the salon and do each other’s hair.
Tess even promised to bring all of her nail polish so they could paint
their toenails.
Leah’s mom said they could order pizza. Tess and Leah decided
that they would get cheese and mushrooms on the pizza. Leah’s mom
said she would take them to rent a movie from the store. They both
agreed to get a scary one.
The girls were so excited about the sleepover. They had been
waiting for this weekend for weeks and knew it was going to be so fun.
Goal: 1:13 2 or less errors Reading #: 1 Time: ___________
Errors: ___________
Reading #: 2
Time: ___________
Errors: ___________
Reading #: 3
Time: ___________
Errors: ___________
Reading #: 4
Time: ___________
Errors: ___________
TIER III ACADEMIC 150
Questions
Write down student’s response and place an X on questions answered incorrectly
(**If students say they do not know an answer to a question ask them the following:
Based on what the story said, what do you think the answer might be?)
1. (F) What were Tess and Leah excited about? (the sleepover this weekend)
2. (I) Whose house did they sleep at? (Tess slept over at Leah’s house)
3. (I) Did Tess’s mom say it was okay for her to sleep over? (yes, they planned it for weeks)
4. (F) What would the girls to first? (play with Leah’s older sister’s make-up)
5. (I) Was Tess allergic to cheese? (no, they planned on getting cheese on the pizza)
6. (F) What kind of movie did the girls agree to get? ( a scary one)
7. (I) Did Tess and Leah like spending time together? (yes)
8. (F) What were the girls going to do with the nail polish? (paint their toenails)
Number correct: ___________
TIER III BEHAVIOR 151
Tier III Behavior: Decreasing Off-Task Behaviors and Increasing Academic Engagement in
Mathematics for a Sixth Grade Student
A Daily Behavior Report Card (DBRC) intervention was designed to increase academic
engagement and decrease off-task behaviors for a sixth grade boy, Gavin, in a suburban middle
school in Ohio. Gavin was referred for intervention by his parents with concerns in the areas of
mathematics problem-solving and attention/focus. The concerns noted were unique to the target
student in the classroom, as the core mathematics curriculum and core behavior management
techniques were not meeting the student’s needs. The student was receiving Tier II intervention
support from the general education mathematics teacher in the area of behavior, with
modifications such as preferential seating and extended time. The student was also receiving
Tier II mathematics intervention through a “Math Intervention Bell” skills group using the “Do
the Math” curriculum. The student had a history of intervention support in the elementary
setting including Tier II support in the form of task-analysis in mathematics, task-analysis of
classroom expectations, check and connect system with use of a planner to facilitate home-
school communication, and use of an accordion folder to promote organization. Due to the
consistent demonstration of off-task behaviors during mathematics class, despite additional
supports inside of his general education classroom and through the Tier II math intervention bell,
it was determined by the team that Gavin required comprehensive Tier III support in
mathematics and behavior.
To address the low academic engagement and high off-task behaviors of the student, an
intervention package was created which included the use of Daily Behavior Report Cards
(DBRC) (Appendix A, B, C) and Mathematics Strategy Instruction (Appendix D,E). The target
variables for intervention included academic engagement and off-task behaviors. Mathematics
concerns were continuing to be addressed through Tier II intervention in the form of the
TIER III BEHAVIOR 152
mathematics intervention bell. The target behaviors were defined and prioritized through the use
of parent interview, teacher interview, and observation. Parental permission was signed by the
parent for participation. To collect baseline and weekly progress monitoring data of the target
variables, the school psychology intern collected observational data through the use of The
Behavioral Observation of Students in Schools (BOSS) behavior code.
The DBRC intervention took place daily in Mathematics class and in Math Intervention
class. The teacher rated Gavin’s behavior on four specific target expectations using a point
system. A reinforcer assessment (See Appendix F) was given to the student and based on the
results, a reward menu was created allowing for points earned through the DBRC to be tied to
receiving a designated number of “Baron Bucks” to use to shop for tangible items at the school
store. The point system allowed for Gavin to receive positive reinforcement for performing the
target behaviors listed on the report card. The report cards were taken home each day by the
student, signed by the parent, then returned the following day for reuse. Gavin’s parents
provided verbal reinforcement when he received adequate points for his positive behaviors.
The Mathematics Strategy Instruction intervention took place daily during Math
Intervention class. Gavin was provided with a Strategy Instruction sheet which included the
steps to mathematics problem-solving. The student was trained on how to perform the
highlighted mathematics problem-solving steps when faced with an unknown problem on in-
class assignments, homework, and on quizzes/exams. The student then independently accessed
the strategy instruction sheet when needed.
After the intervention was put into place, the parent provided documentation of a medical
diagnosis of Attention Deficit Hyperactivity Disorder (ADHD). Due to continued significant
concerns, the team decided to the evaluate Gavin’s current progress and significant needs
TIER III BEHAVIOR 153
through the 504 evaluation process. The school psychology intern served as the facilitator of
parent meetings and team meetings through the 504 evaluation. Through consultation and
through the district’s 504 evaluation process, it was determined that the student’s ADHD
substantially limited a major life activity which negatively impacted learning. A 504 Plan was
then developed and appropriate classroom accommodations were defined and put in place.
Method
Participants and Roles
The student’s parents requested consultation for adding individual supports to assist
Gavin, a sixth grade student with low academic engagement, high off-task behaviors, and low
math scores on assigned math work and math progress monitoring AIMSweb assessments.
Gavin’s off-task behaviors and academic engagement were significantly discrepant from levels
of peers thus warranting a Tier III intervention package. Gavin was diagnosed with ADHD and
was being supported through Tier II interventions both in mathematics and for behavior. During
the consultation an evaluation for 504 accommodations was being conducted by the intern school
psychologist. At the time of the current report, a 504 eligibility meeting was held and it was
determined by the team that Gavin qualified for a 504 plan due to significant behavioral concerns
associated with his ADHD that substantially limited a major life activity which negatively
impacted learning.
The consultation team consisted of the middle school principal, school psychologist,
intern school psychologist, parent, and the mathematics teacher. The intern school
psychologist, in collaboration with the consultation team, determined that the intervention
package would include the use of a Daily Behavior Report Card and Mathematics Strategy
Instruction. Permission was gathered to collect baseline data, implement the intervention, and
TIER III BEHAVIOR 154
progress monitor the target student from the school psychologist, the mathematics teacher, and
the parent of the involved student.
Setting
The intervention package was implemented in the student’s Math and Math Intervention
classes in a rural public middle school. Following the first period Math class, the student’s
behavior was rated on four criteria by the teacher using the DBRC. The student was then asked
to join the teacher at her desk, where the ratings were shared and points for the day were totaled.
Every Friday the student accessed the DBRC with the teacher at her desk and totaled the number
of points for the week. The student was then awarded the appropriate number of “Baron Bucks”
that he had earned for the week and was able to use the “Baron Bucks” to purchase desired
tangible items at the school store, located in the Student Services office. The Strategy
Instruction sheet was used by Gavin during both Math and Math Intervention class, which took
place in the Math classroom during two periods of the day.
Target Variables
Target behaviors were selected through consultation with the team as well as through
baseline observations. Teacher and parent interviews provided information on target concerns
and observation data provided confirmation for these concerns. Target variables selected were
academic engagement and off-task behaviors (motor, verbal, and passive). On-task passive and
on-task active behaviors were combined to inform the target behavior “Academic Engagement”.
The engagement and off-task behaviors of the student and peer comparisons were
measured using the BOSS observation code. The BOSS observation code allows for systematic
measurement of student academic engagement and off-task behavior (Shapiro, 2011). Gavin’s
behavior was monitored using 15-second intervals, with every fifth interval being a peer
TIER III BEHAVIOR 155
comparison observation. Observations took place for 15-20 minutes on average and occurred
one time per week.
Academic Engagement. Academic engagement can be defined as participating in
(actively engaged) or being oriented toward (passively engaged) an academic activity (Shapiro,
2011). Engagement is a meaningful target behavior as it is a predictor of academic success
(Gettinger & Ball, 2008). Increases in academic engagement can be seen by monitoring students
during daily school activities (Gettinger & Ball, 2008). The goal of intervention was for
increases to be made in academic engagement as decreases were made in off-task behaviors;
therefore, engagement served as the desired replacement behavior for off-task behaviors. Active
and passive engagement was measured using momentary time sampling (Shapiro, 2011).
Off-Task Behaviors. The off-task behaviors category consisted of off-task behaviors that
were motor, verbal, or passive in nature. Off-task motor behaviors were defined as non-
academic related motor activity (Shapiro, 2011). Off-task verbal behaviors were defined as non-
academic related vocalizations that violated classroom or school rules (Shapiro, 2011). Off-task
passive behaviors were defined as silently not attending to the current academic task at hand
(Shapiro, 2011). Student academic engagement is strongly correlated with student achievement,
thus high levels of off-task behaviors correlate with lower levels of academic achievement
(Shapiro, 2011). Off-task motor, verbal, and passive behaviors were measured using partial-
interval recording (Shapiro, 2011).
Inter-observer agreement
Co-observations were completed and inter-observer agreement (IOA) was calculated for
20% of the observations completed. IOA was calculated using interval by interval agreement on
the BOSS observation code. Percentage of IOA was calculated by dividing the total number of
TIER III BEHAVIOR 156
interval agreements by the total number of agreements plus disagreements. See Table 1 for IOA
scores.
Table 1
Interobserver Agreement Data
Date IOA
12/12/13 100%
2/6/14 100%
3/26/14 100%
Goals and decision rules
The goals and decision rules for this consultation were determined by the collaborating
consultation team. Due to the desire for Gavin to display behaviors more closely related to those
of peers, goals for total engagement and off-task behaviors were determined using peer
comparison data for that of male classmates. Goals were set for the student to be engaged for
80% of intervals observed, and display off-task behaviors during less than 10% of the intervals
observed. The set goals were agreed upon by the mathematics teacher.
Decision rules were also determined in collaboration with the team. It was determined
that if the student met or exceeded the goal for academic engagement across three consecutive
observation periods, the team would reconvene to discuss plans for fading the intervention.
Similarly, if the student did not meet the goal for academic engagement or off-task behaviors
across three consecutive observations, the team would meet to discuss intervention changes.
Reinforcer Preference Assessment. The school psychology intern conducted a reinforce
preference assessment using a modified version of the Cartwright-Cartwright forced-choice
reinforcement menu. The assessment was used to determine Gavin’s reinforcement preferences.
TIER III BEHAVIOR 157
The results of the assessment indicated that tangible items (ie. candy, pop) and adult attention
were the most reinforcing. See Appendix F for the modified version of the reinforcement menu.
Functional Hypothesis
It was hypothesized that Gavin was engaging in off-task behaviors in order to gain
attention from his teacher. Teacher and parent interview information, as well as BOSS
observation data helped to form this hypothesis. Gavin’s low academic engagement and high
off-task behaviors elicited teacher attention in the form of reprimands and redirections. The
intervention plan addressed the function of Gavin’s behavior by allowing Gavin to gain teacher
attention through the use of the DBRC without having to act out.
Accountability Plan
The accountability plan for this intervention was an AB design. The baseline condition
(A) consisted of two baseline observation points to serve as comparison data to the intervention
observation data. The intervention condition (B) consisted of the use of the DBRC and
Mathematics Strategy Instruction sheet. This design type is often utilized in school settings
because no withdraw of intervention is used; however, this type of design does pose threats to
internal validity (O’Neill, McDonnell, Billingsley, & Jenson, 2011).
Intervention procedures
Baseline. Two data points were collected as baseline data. The student’s math class took
place during the first bell of the school day; due to holiday break and school delays/cancellations,
only two baseline data points were collected prior to the start of intervention.
Daily Behavior Report Card. A Daily Behavior Report Card (DBRC) refers to rating
target behavior a minimum of one time daily, and sharing the information with someone besides
the rater (Chafouleas, Riley-Tillman, Sassu, 2006). DBRCs have four defining characteristics:
operationally defined target behavior, standardized observations for progress monitoring, use in a
TIER III BEHAVIOR 158
specific time and place with predetermined frequency, and consistent scoring of data (Riley-
Tillman, Chafouleas, & Briesch, 2007). Research has indicated that there is high teacher buy-in
for use of the DBRC intervention (Chafouleas, Riley-Tillman, Sassu, 2006). Research supports
the effectiveness of DBRCs as an intervention for behavior modification (Fabiano, et al., 2010).
Further, DBRC have been found to be an effective intervention for students with ADHD as it
entails targeting specific behaviors, meeting specific criteria, gaining immediate feedback, and
earning reinforcement. (Fabiano, Vujnovic, Naylor, Pariseau, & Robins, 2009).
In this consultation, eight data points were collected during the intervention phase. The
intern school psychologist, in collaboration with the team, created a rating scale with a three
point system. Three points were earned for each target expectation that Gavin followed
throughout 75% or more of the class period. Two points were earned for following the
expectation during most of the class period; or 50-75% of the time. One point was earned for
each target expectation that Gavin followed less than 50% of the time. A reward menu was also
created which allowed Gavin to earn reinforcers, depending on the number of points that he
accumulated that week using the DBRC system. Reward menu reinforcer items used were
“Baron Bucks”. “Baron Bucks” were bills that students could spend toward desired tangible
items (candy, food, small toys, backpacks, giftcards, books, etc.) at the school store.
Additionally, the DBRC was intended to be taken home each night to be signed by Gavin’s
parent, who agreed to deliver verbal reinforcement for a good day’s behavior. Taking the report
card home, also allowed for additional parent-teacher communication as the parent and teacher
could write notes on the back of the DBRC if they wished to communicate a message.
Mathematics Strategy Instruction. Problem-solving in mathematics is a complex
activity requiring both problem representation and problem execution (Montague, Enders, &
TIER III BEHAVIOR 159
Dietz, 2011). When students struggle with math problem-solving, best practice strategies
include encouraging students to “think aloud”; using metacognitive steps to solve a problem
(Furner, Yahya, & Duffy, 2005). Additionally, using the problem-solving process, allows for
students to read and understand the problem, develop a plan to solve the problem, use their plan
to compute the problem, and check their work (Furner, Yahya, & Duffy, 2005). Rewriting word
problems in one’s own terms is also an effective technique for problem-solving (Furner, Yahya,
& Duffy, 2005). The above mentioned strategies, shown through research to increase math
problem-solving in students were included in the Math Strategy Instruction worksheet for this
consultation. Mathematic Strategy Instruction was used to provide a visual model for
mathematics problem-solving for Gavin as he worked through multi-step problems. The strategy
instruction worksheet consisted of seven steps: Read, Paraphrase, Visualize, Hypothesize,
Estimate, Compute, and Check. Each step included statements for the student to “Say”, “Ask”,
and “Check” himself for understanding. The worksheet was copied and attached to the inside of
Gavin’s planner. Gavin was prompted to open his planner to the worksheet at the start of each
math and math intervention class and use it as a guide while answering math problems.
Adherence data
The school psychology intern collected intervention package adherence data during 20%
of the observations through the use of the intervention script. See table 2 for adherence data.
Table 2
Intervention Package Adherence Data
Date DBRC Strategy Instruction
1/27/14 100% 100%
2/27/14 100% 100%
TIER III BEHAVIOR 160
Social validity
The intervention package, target variables, goals, and decision rules were determined in
collaboration by the building principal, school psychologist, school psychology intern, parent,
and mathematics teacher. Meetings with the consultation team took place frequently in order to
share recent progress and determine if changes needed to be made. The team determined that the
intervention package should be discrete so as not to draw additional attention to the student or
take away from instructional time. The intervention package was designed to be simple in order
for ease of teacher implementation and ease for the student to access reinforcement. Prior to the
start of the intervention package, the school psychology intern met with Gavin to explain the
intervention and ask for his approval of the intervention plan. Communication between the
school psychology intern and teacher was frequent and any teacher concerns were addressed
immediately. Social validity was assessed in an ongoing manner as meetings including the
classroom teacher occurred prior to and during the intervention phase. Social validly
information was also gathered from Gavin as frequent conversations occurred between him and
the school psychology intern asking if he believed that the intervention was improving his
behavior and if he enjoyed earning “Baron Bucks”.
Results
Baseline and progress monitoring for Gavin’s engagement, off-task motor, of-task verbal, and
off-task passive behaviors are displayed in Figures 1 and 2 below. As displayed in the figures,
data showed overall increases in engagement and decreases in all off-task behaviors.
TIER III BEHAVIOR 161
Figure 1: Target student engagement with peer comparison data in mathematics class
Figure 1 displays the percentage of intervals in which Gavin was actively or passively
engaged in an academic activity. Baseline data show Gavin’s engagement to be between 40%
and 49% compared to peers, who showed levels of engagement near 90%. Low levels of
engagement seen from Gavin during baseline encouraged the use of academic engagement as a
target behavior of this consultation. Following intervention implementation, an immediate
increase was seen in academic engagement. Levels of engagement remained above the goal of
80% for two consecutive observations, followed by a decrease to 77%, an increase above the
goal for two consecutive observations, a decrease to 65% engagement, and an increase above the
goal for two consecutive observations.
TIER III BEHAVIOR 162
Data show that Gavin had not yet met the goal of three consecutive data points above the
goal of 80% academic engagement, although he was making progress towards the goal. Data
showed an overall increase in academic engagement from baseline to intervention. The
intervention will continue to be implemented until Gavin reaches the goal of three consecutive
data points above the goal. At that time, the team will meet to discuss fading the intervention.
The data support the DBRC intervention and use of mathematics strategy instruction as
successful in increasing Gavin’s academic engagement. Observation during DBRC and strategy
instruction use showed high adherence to the intervention scripts. The student was inconsistent
at times in taking the report card home nightly to be signed and viewed by his parent. This could
have contributed to lower-than-goal engagement during several observations.
TIER III BEHAVIOR 163
Figure 2: Target student Off-Task Passive, Off-Task Motor, and Off-Task Verbal behavior
Off-Task Behaviors
Figure 2 displays Gavin’s off-task behaviors. During baseline, levels of off-task behavior
ranged from 15-33%. The goal was set for off-task behaviors to remain below 10%. Following
intervention implementation, Gavin’s off-task behaviors decreased to levels below the goal.
Two times during the intervention phase, off-task passive behaviors increased to levels above
10%. Off-task motor and off-task verbal behaviors however, remained below 10% throughout
the intervention phase. For off-task passive behaviors, Gavin had maintained two data points
below the goal of less than 10% off-task behaviors. Gavin was making progress toward the goal
in off-task passive behaviors.
In support of visual analysis, summary statistics were calculated and can be seen below in
Tables 3, 4, and 5. The percentage of non-overlapping data (PND) for each variable was
determined by identifying how many data points did not overlap with the highest or lowest
baseline data point depending on the goal of the intervention. The effect size for each variable
was calculated by dividing the difference of the baseline mean and the last intervention phase
mean by the standard deviation of the baseline.
For engagement, a large effect size of 9.52 was found with the percentage of non-
overlapping data being 100%. For off-task passive behaviors, a large effect size of 1.13 was
found with the percentage of non-overlapping data being 75%. For off-task motor behaviors, a
large effect size of 4.28 was found with the percentage of non-overlapping data being 100%.
Off-task motor behaviors showed an effect size of 4.28 with 100% non-overlapping data. Off-
task verbal behaviors showed a large effect size of 10.41 with the percentage of non-overlapping
data being 100%. The standard deviation for off-task verbal behavior was 10.41
TIER III BEHAVIOR 164
Overall, improvements were seen in Gavin’s engagement and off-task behaviors although
Gavin had not yet reached the third data point for the goal of three consecutive data points above
80% for engagement, and below 10% for off-task passive behaviors. Gavin did reach the goal
for displaying less than 10% off-task motor and off-task verbal behaviors, three consecutive
times. Teacher reports from the classroom teacher indicated satisfaction with the interventions
chosen and the increased engagement and decreased off- task behaviors. Using goal attainment
scaling measures (where ‘0’ indicates no progress made toward the goal, ‘1’ indicates progress
made toward the goal, and ‘2’ indicates the goal was met), Gavin received a ‘1’ in the
engagement and off-task passive variables. He received a ‘2’ in the off-task motor and off-task
verbal variables. A follow-up observation will be conducted during a maintenance and
generalization phase to determine if the effects of the intervention package hold over time and
across settings.
Table 3
Means and Standard Deviations of Graphed Variables (Baseline Phase)
Variable (Behavior) Percentage of observed intervals displaying the behavior (Baseline Phase)
Number of baseline
data points
Mean Standard Deviation
Engagement 2 43.75% 4.79
Off-Task Passive 2 22.95% 11.81
Off-Task Motor 2 19.8% 4.38
Off-Task Verbal 2 31.25% 2.90
TIER III BEHAVIOR 165
Table 4
Means and Standard Deviations of Variables (Intervention Phase)
Variable (Behavior) Percentage of observed intervals displaying the behavior (Baseline
Phase)
Number of intervention
data points
Mean Standard Deviation
Engagement 8 89.34% 12.01
Off-Task Passive 8 9.64% 10.44
Off-Task Motor 8 1.05% 1.59
Off-Task Verbal 8 1.05 % 1.58
Table 5
Percent of Non-overlapping Data (PND), Goal Attainment Scaling (GAS), and Effect Size for
Graphed Variables
Student PND GAS Effect Size
Engagement 100% 1 9.52
Off-Task Passive 75% 1 1.13
Off-Task Motor 100% 2 4.28
Off-Task Verbal 100% 2 10.41
Discussion
This consultation provided Tier III behavioral intervention support for a sixth grade boy.
The consultation team, consisting of the school psychologist, intern school psychologist,
principal, mathematics teacher, and parent collaboratively developed and implemented the
chosen research-based interventions. The intervention package was implemented with fidelity
TIER III BEHAVIOR 166
and the target behaviors were progress monitored to determine the effectiveness of the
interventions. A daily behavior report card and mathematics problem solving intervention
package was implemented and teacher reports and collected data reflect an increase in the target
variable of academic engagement as well as decreases in all types of off-task behaviors.
This consultation has several limitations. One limitation was the lack of baseline data
points collected prior to intervention implementation. It is ideal to have a least three stable
baseline data points before intervention begins. In this consultation, due to the unusually high
numbers of calamity days (school cancellations and delays due to weather) and due to the
schedule of the intern school psychologist, only two observations could be conducted prior to the
start of the intervention. The intensity of Gavin’s behaviors and their impact on his and other
students’ learning, made it important to begin intervention prior to gathering a third baseline data
point. However, this limitation displays the real-world challenges that school psychologists face
in a school setting. School schedules can change and a school psychologist must have the ability
to adapt to the changes in order to create the best outcomes for students.
Another limitation of this case was the decreased consistency of use due to higher than
typical calamity days resulting from poor winter weather conditions. The mathematics class in
which Gavin participated was scheduled as the first bell of the day. When school delays
occurred, this class was missed. The high number of both school closings, and school delays
caused the intervention to be used more inconsistently than if attendance had not been a factor.
This consultation provided excellent experience working on a collaborative team to develop
and implement a Tier III intervention to best meet the intensive needs of a student. The
experience of utilizing the Response to Intervention (RtI) framework to intensify intervention
based on student need was also an outcome of working with the team during this consultation.
TIER III BEHAVIOR 167
Gavin was receiving Tier I core instruction and Tier II interventions for behavior and academics,
however was not making appropriate progress with those levels of support. The decision was
made to increase support through the use of Tier III intervention. During consultation, data was
being collected and used for the 504 Plan evaluation process. It was determined that Gavin
qualified for a 504 plan as his diagnosis of ADHD limits the major life activity of learning.
Being a part of both the intervention team as well as the team conducting the 504 Plan provided
me with excellent opportunities for enhancing my skills in child and family advocacy. Despite
the effective Tier III interventions acting in place, Gavin needed additional accommodations
made during testing situations which would allow him to perform to the best of his ability.
As highlighted in my personal model of practice, my desire to follow the scientist-
practitioner model when providing services in the school was fulfilled through this consultation.
The team developed a research-based intervention package and utilized data collected to inform
decision making during the consultation process. This consultation allowed for the use of my
graduate training implementing a behavioral model of practice to provide intervention which
matches the function of the student behavior with the selected intervention package.
Additionally, as reflected in my personal model of professional practice, the use of an
ecological/behavioral orientation while problem solving is an important personal goal through
consultation and this case allowed me to utilize this. The consultation provided excellent, new
experiences such as the use of the 504 Plan evaluation process. It also allowed for existing skills
to be practiced in order to best prepare me for future independent practice as a school
psychologist.
TIER III BEHAVIOR 168
References
Cartwright, C. A., & Cartwright, G. P. (1970). Determining the motivational systems of
individual children. Teaching Exceptional Children, 2 (3), 143-149.
Chafouleas, S.M., Riley-Tillman, T.C., & Sassu, K.A. (2006). Acceptability and reported use of
daily behavior report cards among teachers. Journal of Positive Behavior Interventions,
8 (3), 174-182.
Fabiano, G.A., Vujnovic, R., Naylor, J., Pariseau, M., & Robins, M. (2009). Technical adequacy
of a daily behavior report card (DBRC) for monitoring progress of students with
attention-deficit/hyperactivity disorder in special education placements. Assessment for
Effective Intervention, 34(4), 231-241.
Fabiano, G.A., Vujnovic, R.K., Pelham, W.E., Waschbusch, D.A., Massetti, F.M...Volker, M.
(2010). Enhancing the effectiveness of special education programming for children with
attention deficit hyperactivity disorder using a daily report card. School Psychology
Review, 39(2), 219-239.
Furner, J.M., Yahya, N.., & Duffy, M.L. (2005). Teach mathematics: Strategies to reach all
students. Intervention in School and Clinic, 41 (1), 16-23.
Gettinger, M. & Ball, C. R. (2008). Best practices in increasing academic engaged time. In A.
Thomas & J. Grimes (Eds.), Best practices in school psychology V (pp. 1043-1058).
Bethesda, MD: National Association of School Psychologists.
Montague, M., Enders, C., & Dietz, S. (2011). Effects of cognitive strategy instruction on math
problem solving of middle school students with learning disabilities. Learning Disability
Quarterly, 34 (4), 262-272.
TIER III BEHAVIOR 169
Riley-Tillman,T.C., Chafouleas, S.M., & Briesch, A.M. (2007). A school practitioner’s guide to
using daily behavior report cards to monitor student behavior. Psychology in the Schools,
44 (1), 77-89.
Shapiro, E.S. (2011). Academic skills problems: direct assessment and intervention (4th
ed.).
New York: Guilford.
TIER III BEHAVIOR 170
Appendix A
Daily Behavior Report Card Script
Steps Completed
Using the DBRC, rate Hunter’s behaviors during Math 6, at the end of 1st bell
During Math Intervention, review Hunter’s scores from the DBRC ratings with him
Explain reasoning behind ratings
Ask for his agreement or disagreement on the ratings
Discuss areas for improvement as well as successes
Discuss point totals and reward system with Hunter
TIER III BEHAVIOR 171
Appendix B
Daily Check-In Chart
I kept my eyes on the
teacher when she
was speaking.
I kept my eyes on my work during independent work time.
I got started right away,
kept working, and
completed my work.
I turned in today’s
homework (if applicable)
Parent Signature
Monday
Tuesday
Wednesday
Thursday
Friday
Point System
3: Followed the expectation throughout the entire class period. (75%+)
2: Followed the expectation during most of the class period. (50-75%)
1: Did not follow expectation today. (>50%)
TIER III BEHAVIOR 172
Appendix C
Point System
Earned Points Reward Earned
40 Prize from Teacher
50 15 Baron Bucks
65 20 Baron Bucks
75
25 Baron Bucks
100
5 minutes of “free time”
Maximum Points Possible in 1 Day= 12
Maximum Points Possible in 1 Week= 60
Log of Rewards Earned
Points Earned this
Week
Reward Earned Remaining Points
TIER III BEHAVIOR 173
Appendix D
Mathematics Problem Solving Worksheet Script
Instructions: Follow the procedures below, place a mark in each box on the left that is completed
during intervention administration.
Steps Completed
First Time Use: Attach the Math Strategy Instruction Worksheet to the inside page of student’s
planner
At the start of each math and math intervention class, prompt student to open his planner to the Math Strategy Instruction Worksheet
Review each of the listed steps with the student:
Use a practice problem while reviewing steps
Demonstrate how to complete each step using the practice problem
After demonstrating how to complete each step, prompt the student to demonstrate to you how he would complete the problem
Provide error correction and feedback
Explain to the student that he should use the worksheet to guide him as he completes problems during each math and math intervention class
Each time after first use: At the start of each math and math intervention class, prompt student to open his
planner to the Math Strategy Instruction Worksheet
Prompt the student to use the worksheet as he completes math problems.
Check in with student as he completes independent work and ensure that he is using the worksheet when needed.
TIER III BEHAVIOR 174
Appendix E
READ (for understanding)
Say: Read the problem. If I don’t understand, read it again.
Ask: Have I read and understood the problem?
Check: For understanding as I solve the problem
PARAPRASE (your own words)
Say: Underline the important information. Put the problem in my own words.
Ask: Have I underlined the important information? What is the question? What am I looking
for?
Check: That the information goes with the questions.
VISUALIZE (a picture or a diagram)
Say: Make a drawing or a diagram. Show the relationships among the problem parts.
Ask: Does the picture fit the problem? Did I show the relationships?
Check: The picture against the problem information.
HYPOTHESIZE (a plan to solve the problem)
Say: Decide how many steps and operations are needed. Write the operation symbols (*, -, x,
and /)
Ask: If I…, what will I get? If I…, then what do I need to do? How many steps are needed?
Check: That the plan makes sense.
ESTIMATE (predict the answer)
Say: Round the number, do the problem in my head, and write the estimate.
Ask: Did I round up and down? Did I write the estimate?
Check: That I used the important information.
COMPUTE (do the arithmetic)
Say: Dothe operation in the right order
Ask: How does my answer compare with my estimate? Does my answer make sense? Are the
decimals or money signs in the right places?
Check: That all the operations were done in the right order.
CHECK (make sure everything is right)
Say: Check the plan to make sure it is right. Check the computation.
Ask: Have I checked every step? Have I checked the computation? Is my answer right?
Check: That everything is right. If not, go back. Ask for help if I need it.
TIER III BEHAVIOR 175
Appendix F
Forced-Choice Reinforcement Menu
Name:_______________________________
Please choose the one from each pair that you would like best and mark and "X" in the blank that
comes in front of it. Remember, mark only one blank for each pair."
1. _____
_____
Teacher writes "100" on your paper. (A)
Be first to finish your work. (CM)
2. _____
_____
A bag of chips. (CN)
Classmates ask you to be on their team. (P)
3. _____
_____
Be free to do what you like. (I)
Teacher writes "100" on your paper. (A)
4. _____
_____
Classmates ask you to be on their team. (P)
Be first to finish your work. (CM)
5. _____
_____
Be free to do what you like. (I)
A bag of chips. (CN)
6. _____
_____
Teacher writes "100" on your paper. (A)
Classmates ask you to be on their team. (P)
7. _____
_____
Be first to finish your work. (CM)
Be free to do what you like. (I)
8. _____
_____
A bag of chips. (CN)
Teacher writes "100" on your paper. (A)
9. _____
_____
Classmates ask you to be on their team. (P)
Be free to do what you like. (I)
10. _____
_____
Be first to finish your work. (CM)
A bag of chips. (CN)
11. _____
_____
Teacher writes "A" on your paper. (A)
Be the only one that can answer a question. (CM)
12. _____
_____
A candy bar. (CN)
Friends ask you to sit with them. (P)
13. _____
_____
Be free to go outside. (I)
Teacher writes "A" on your paper. (A)
14. _____ Friends ask you to sit with them. (P)
TIER III BEHAVIOR 176
_____ Be the only one that answers a question. (CM)
15. _____
_____
Be free to go outside. (I)
A candy bar. (CN)
16. _____
_____
Teacher writes "A" on your paper. (A)
Friends ask you to sit with them. (P)
17. _____
_____
Be the only one that can answer a question. (CM)
Be free to go outside. (I)
18. _____
_____
A candy bar. (CN)
Teacher writes "A" on your paper. (A)
19. _____
_____
Friends ask you to sit with them. (P)
Be free to go outside. (I)
20. _____
_____
Be the only on that can answer a question. (CM)
A candy bar. (CN)
21. _____
_____
Teacher writes "Perfect" on your paper. (A)
Have only your paper shown to the class. (CM)
22. _____
_____
A can of soda. (CN)
Classmates ask you to be class leader. (P)
23. _____
_____
Be free to play outside. (I)
Teacher writes "Perfect" on your paper. (A)
24. _____
_____
Classmates ask you to be class leader. (P)
Have only your paper shown to the class. (CM)
25. _____
_____
Be free to play outside. (I)
A can of soda. (CN)
26. _____
_____
Teacher writes "Perfect" on your paper. (A)
Classmates ask you to be class leader. (P)
27. _____
_____
Have only your paper shown to the class. (CM)
Be free to play outside. (I)
28. _____
_____
A can of soda. (CN)
Teacher writes "Perfect" on your paper. (A)
29. _____
_____
Classmates ask you to be class leader. (P)
Be free to play outside. (I)
30. _____
_____
Have only your paper shown to class. (CM)
A can of soda. (CN)
TIER III BEHAVIOR 177
31. _____
_____
Teacher writes "Excellent" on your paper. (A)
Have your paper put on the bulletin board. (CM)
32. _____
_____
A pack of gum. (CN)
Friends ask you to work with them. (P)
33. _____
_____
Be free to work on something you like. (I)
Teacher writes "Excellent" on your paper. (A)
34. _____
_____
Friends ask you to work with them. (P)
Have your paper put on the bulletin board. (CM)
35. _____
_____
Be free to work on something you like. (I)
A pack of gum. (CN)
36. _____
_____
Teacher writes "Excellent" on your paper. (A)
Friends ask you to work with them. (P)
37. _____
_____
Have your paper put on the bulletin board. (CM)
Be free to work in something you like. (I)
38. _____
_____
A pack of gum. (CN)
Teacher writes "Excellent" on your paper. (A)
39. _____
_____
Friends ask you to work with them. (P)
Be free to work on something you like. (I)
40. _____
_____
Have your paper put on the bulletin board. (CM)
A pack of gum. (CN)
Reinforcement Inventory Scoring Key
__________ Adult Approval (A)
__________ Competitive Approval (CM)
__________ Peer Approval (P)
__________ Independent Rewards (I)
__________ Consumable Rewards (CN)
Modified by Gable, R. A. (1991) from:
Cartwright, C. A., & Cartwright, G. P. (1970). Determining the motivational systems of
individual children. Teaching Exceptional Children, 2 (3), 143-149.
