Assessment for Effective Intervention recent issues

A Conceptual Model for Evaluating System Effects of Response to Intervention

Shapiro, E. S., Clemens, N. H. — Thu, 12 Nov 2009 12:44:18 PST

Implementing a Response to Intervention (RTI) system could improve overall student achievement and the way in which students with disabilities are identified. In order to evaluate the effectiveness of an RTI system (i.e., "Is our RTI system accomplishing its stated goals?"), a set of data-based indicators are needed. This paper will describe a set of five measure-able indicators from three domains of evaluation that schools can use to obtain frequent feedback on the impact of their RTI system on reading instruction and achievement. The evaluation methodology provides multiple, sensitive metrics that can be used soon after RTI implementation begins, and does not require that schools wait for more long-term, singular outcome measures such as performance on high-stakes tests to determine if the RTI system is functioning as intended. The data used for each indicator and the way in which the data can impact decisions is described. Issues related to RTI evaluation and areas of further research are discussed.

Nationally Certified Educational Diagnostician (NCED): The Professional Credential for Assessment Specialists

Sutton, J. P., Frye, E. M., Frawley, P. A. — Thu, 12 Nov 2009 12:44:19 PST

A new professional credential, the Nationally Certified Educational Diagnostician (NCED), was established in 2006 for assessment specialists known as educational diagnosticians. The first cohort of 511 diagnosticians from 11 states received NCED certificates in January 2008. Discussed in this article are several issues of concern related to national certification of diagnosticians, including viability and distinctiveness of the profession, and quality and uniformity of practice. This article also discusses the chronology of events leading to establishment of the NCED program, qualifications for the credential, and future challenges to the program.

Educational Diagnosticians' Understanding of Phonological Awareness, Phonemic Awareness, and Reading Fluency

Chappell, J. C., Stephens, T. L., Kinnison, L., Pettigrew, J. D. — Thu, 12 Nov 2009 12:44:19 PST

This article summarizes the results of a study involving 42 educational diagnosticians from North Texas. The study was conducted to determine diagnosticians’ perceived understanding of early literacy development and their ability to effectively choose and interpret assessments of phonological awareness, phonemic awareness, and reading fluency. The results suggested that the educational diagnosticians who participated in the study were not sufficiently knowledgeable in identifying the numerous components of reading (e.g., phonological awareness, phonemic awareness, and fluency); nor were they prepared to choose appropriate assessment instruments to assess selected components of reading.

They Read, but How Well Do They Understand?: An Empirical Look at the Nuances of Measuring Reading Comprehension

Alonzo, J., Basaraba, D., Tindal, G., Carriveau, R. S. — Thu, 12 Nov 2009 12:44:19 PST

Much of the literature on reading development focuses on measures of early literacy skills (e.g., phonological awareness, phonics, fluency). Elementary educators interested in improving students’ skills in these areas can draw on a wealth of research studies. However, many studies of early literacy skills have not addressed comprehension, obviously an important feature of literacy. The authors used a one-parameter Rasch model to examine the relative difficulty of different multiple-choice reading comprehension items assessing students’ literal, inferential, and evaluative comprehension of fictional narratives. They also examined the difficulty of questions derived from concepts identified by state content standards as important components of reading comprehension, such as character, prediction, and plot sequence. The findings suggest a curvilinear relationship between literal, inferential, and evaluative comprehension, with literal comprehension being the easiest and inferential and evaluative comprehension more challenging. The findings also indicate that assessment objectives differ on the basis of difficulty. Implications of these findings for practice are discussed.

Psychometric Properties of the Texas Primary Reading Inventory for Early Reading Screening in Kindergarten

Nelson, J. M. — Thu, 12 Nov 2009 12:44:19 PST

This study investigated the psychometric properties of the kindergarten Texas Primary Reading Inventory (TPRI; Texas Education Agency and the University of Texas System, 2004—2006) Mid-Year screeners with a sample of 180 students. The TPRI screeners were designed to determine which students are not at risk for later reading failure. Results indicated adequate internal consistency reliability and concurrent and predictive validity for screening purposes. Classification validity analyses indicated low false negative rates and high negative predictive power but substantial false positive rates and low positive predictive power. The TPRI Mid-Year screeners were found to be adequate instruments for identifying students who are unlikely in need of further assessment. Recommendations are provided for using the TPRI within a response-to-intervention or other prevention-based service delivery model.

Psychometric Properties of the Foreign Language Attitudes and Perceptions Survey for College Students

Thu, 12 Nov 2009 12:44:19 PST

The psychometric integrity of the Foreign Language Attitudes and Perceptions Survey for college students was investigated. Two hundred and seventy-eight college students were administered a modified version of the survey (FLAPS-C). Internal consistency reliabilities, as determined by Cronbach’s alpha, were .74, attitudes; .80, perceptions; and .86, combined. Testretest reliability was equally robust (r = .83). Factor analytic data suggest a one-factor solution rather than two separate factors (i.e., attitudes and perceptions). Results provide preliminary support for the use of FLAPS-C as a measure of college students’ attitudes and perceptions toward learning a foreign language. The FLAPS-C’s potential utility for secondary and postsecondary special educators of at-risk students including those with learning disabilities is discussed.

Test Review: A Review of the Five Factor Personality Inventory--Children

Klingbeil, D. A. — Thu, 12 Nov 2009 12:44:19 PST

Direct Behavior Rating (DBR): An Emerging Method for Assessing Social Behavior Within a Tiered Intervention System

Chafouleas, S. M., Riley-Tillman, T. C., Christ, T. J. — Tue, 04 Aug 2009 15:20:01 PDT

Foundation for the Development and Use of Direct Behavior Rating (DBR) to Assess and Evaluate Student Behavior

Christ, T. J., Riley-Tillman, T. C., Chafouleas, S. M. — Tue, 04 Aug 2009 15:20:01 PDT

Direct Behavior Rating (DBR) is a method of social—emotional and behavior assessment that combines the immediacy of systematic direct observation and the efficiency of behavior rating scales. The purpose of this article is to discuss the defensibility and usability of DBR. This article provides a brief summary of (a) the past, present, and future directions of social— emotional and behavior assessment methods in schools; (b) the defining features of DBR; (c) the guiding principles for DBR development and evaluation; and (d) DBR research to date. Special emphasis is placed on single-item scale DBR (SIS-DBR) and three general outcome behaviors that are most relevant for use in schools. Research and recommendations for standard SIS-DBR instrumentation and procedures are reviewed, along with future directions for research and practice.

Using Direct Behavior Rating (DBR) to Screen for School Social Risk: A Preliminary Comparison of Methods in a Kindergarten Sample

Chafouleas, S. M., Kilgus, S. P., Hernandez, P. — Tue, 04 Aug 2009 15:20:01 PDT

In this study, preliminary evidence of the potential for Direct Behavior Rating (DBR) in screening assessment of school social behavior is provided through evaluation of the concurrent validity of DBR with a commonly used criterion measure. The teacher-completed form of the Social Skills Rating System (SSRS) was selected as the criterion measure given similarity to DBR in both broad construct of interest and intended use. Classroom teachers completed DBRs of academic engagement and disruptive behavior twice daily, along with fall and spring completion of the SSRS. Overall results support a moderate to strong association between the methods, with stronger correlations in the fall assessment period. In addition, evidence of teacher acceptability for DBR is provided. Along with specific recommendations for future research, implications are discussed regarding selection and interpretation of behavior assessment methods for screening purposes.

Examining the Use of Direct Behavior Rating on Formative Assessment of Class-Wide Engagement: A Case Study

Riley-Tillman, T. C., Methe, S. A., Weegar, K. — Tue, 04 Aug 2009 15:20:01 PDT

High-quality formative assessment data are critical to the successful application of any problem-solving model (e.g., response to intervention). Formative data available for a wide variety of outcomes (academic, behavior) and targets (individual, class, school) facilitate effective decisions about needed intervention supports and responsiveness to those supports. The purpose of the current case study is to provide preliminary examination of direct behavior rating methods in class-wide assessment of engagement. A class-wide intervention is applied in a single-case design (B-A-B-A), and both systematic direct observation and direct behavior rating are used to evaluate effects. Results indicate that class-wide direct behavior rating data are consistent with systematic direct observation across phases, suggesting that in this case study, direct behavior rating data are sensitive to classroom-level intervention effects. Implications for future research are discussed.

An Investigation of the Technical Adequacy of a Daily Behavior Report Card (DBRC) for Monitoring Progress of Students With Attention-Deficit/Hyperactivity Disorder in Special Education Placements

Fabiano, G. A., Vujnovic, R., Naylor, J., Pariseau, M., Robins, M. — Tue, 04 Aug 2009 15:20:01 PDT

Many children with attention-deficit/hyperactivity disorder (ADHD) are eligible for special education services because of problems with academic and/or social functioning. Thus, a considerable proportion of children with ADHD have individualized education plans (IEPs) that list operationalized goals and objectives for each student. Conceptually, the majority of these children fall within Tiers 2 and 3 of a tiered intervention system because of a need for more intensive behavioral supports. Given the potentially variable behavior of these students in classroom settings and a concurrent need to demonstrate the effectiveness of intervention outcomes, frequent and sustained monitoring of goals and objectives is necessary. For this purpose, direct behavior rating scales such as a Daily Behavior Report Card (DBRC) may serve as an efficient and effective mechanism for progress monitoring. In a sample of 63 students with ADHD receiving special education services, initial psychometric information for the temporal stability, interrater reliability, and content validity of an idiographic, multi-item DBRC is presented. Procedures for integrating a DBRC into standard progress monitoring procedures likely to be useful in intervention monitoring at Tiers 2 and 3 are discussed.

Rating Scale Items: A Brief Review of Nomenclature, Components, and Formatting to Inform the Development of Direct Behavior Rating (DBR)

Christ, T. J., Boice, C. — Tue, 04 Aug 2009 15:20:01 PDT

Ratings scales are a common component of many multisource, multimethod frameworks for socioemotional and behavior assessment of children. There is a modest literature base to support the use of attitudinal, behavioral, and personality rating scales. Much of that historic literature focuses on the characteristics and interpretations of specific scales, which are mostly Likert-type scales. There are many more scale types and item types that receive less attention within the literature and less application in practice. This article provides a brief summary of the literature relevant to formats, components, and nomenclature associated with rating scale item types. This article is intended to provide basic information that might inspire and contribute to the development and evaluation of novel rating scales with a variety of item types, especially those relevant to Direct Behavior Rating methods of assessment.

Extending Use of Direct Behavior Rating Beyond Student Assessment: Applications to Treatment Integrity Assessment Within a Multi-Tiered Model of School-Based Intervention Delivery

Hagermoser Sanetti, L. M., Chafouleas, S. M., Christ, T. J., Gritter, K. L. — Tue, 04 Aug 2009 15:20:01 PDT

To make valid decisions about intervention effectiveness in a tiered intervention system, it is essential to formatively assess treatment integrity along with student outcomes. Despite significant advances in technologies for ongoing assessment of student outcomes, research regarding treatment integrity assessment has not shared the same progress in that most available methods lack adequate psychometric evidence and require significant resources. Direct behavior rating has been discussed and evaluated as a tool for assessing student behavior, yet the technology could be extended to result in an efficient treatment integrity assessment method with utility in a multi-tiered model of school-based intervention delivery. This article reviews options in treatment integrity assessment, with emphasis on how direct behavior rating technology might be incorporated within a multi-tiered model of intervention delivery. Implications for both practice and research are addressed.

Conference Calendar

Tue, 04 Aug 2009 15:20:01 PDT

Editor's Note

Burns, M. — Mon, 04 May 2009 12:15:47 PDT

Written Language Assessment: Introduction to the Special Series

Polloway, E. A. — Mon, 04 May 2009 12:15:47 PDT

Rubrics: Heuristics for Developing Writing Strategies

De La Paz, S. — Mon, 04 May 2009 12:15:47 PDT

Rubrics are an integral part of many writing programs, and they represent elements of good writing in essays, stories, poems, as well as other genres and forms of text. Although it is possible to use rubrics to teach students about the processes underlying effective writing, a more common practice is to use rubrics as a means of assessment, after students have completed their work. Thus, the premise of this article is to show teachers how to transform rubrics into powerful teaching devices, using foundations from strategy instruction as a means for this endeavor. Two examples are used in the article to explain this process: (a) a close examination of the comparison genre; and (b) a planning strategy for writing expository essays, called PLAN and WRITE, that was previously validated with middle school students with and without learning disabilities.

The Learning-to-Learn Strategies of Adolescent Students With Disabilities: Highlighting, Note Taking, Planning, and Writing Expository Texts

Mon, 04 May 2009 12:15:47 PDT

This study focuses on an examination of the learning-to-learn strategies of seventh-grade students as they highlight, take notes, plan, organize, and write expository texts. Participants consist of 125 students, 41 with disabilities and 84 without disabilities. The results reveal that the students with disabilities have more difficulties in using the learning-to-learn strategies as they read, study, and write expository texts, although neither group is judged to be highly proficient. The implications point to a renewed emphasis on preparing content-area teachers to teach the literacy and learning strategies that support the development of learners who know how to read to learn and how to write to learn.

Written Language Expression: Assessment Instruments and Teacher Tools

Penner-Williams, J., Smith, T. E. C., Gartin, B. C. — Mon, 04 May 2009 12:15:47 PDT

Written language is a complex set of skills that facilitate communication and that are developed in a predictable sequence. It is therefore possible to analyze current skills, identify deficits, plan interventions, and determine the effectiveness of the intervention. To effectively accomplish these tasks, educators need to choose appropriate assessment tools. The authors discuss the strengths and weaknesses of informal written assessments and of standardized written assessments. A table of the better known and more frequently used standardized written language assessments is presented that includes test title, publisher, Web site URL, normative group, indication of its focus (spelling, style, or mechanics), age of normative group, and method of administration. This information will help readers in the selection of written language assessment tools.

Spelling Assessment of Students With Disabilities: Formal and Informal Procedures

Beirne-Smith, M., Riley, T. F. — Mon, 04 May 2009 12:15:47 PDT

Accurate spelling is a complex act that requires cognitive and linguistic knowledge of the phonological, morphological, syntactical, and semantic principles of our language. Students with disabilities frequently exhibit spelling difficulties related to language learning disorders and inefficient cognitive processing. These difficulties often are developmental in nature. Remediation of spelling disorders in students with disabilities requires accurate, comprehensive assessment. Accurate, comprehensive assessment involves the use of both formal and informal assessment measures. In this article, we discuss the theory and research that support the assessment of spelling in students with disabilities and we describe formal and informal spelling assessment procedures.

Informal and Qualitative Assessment of Writing Skills in Students With Disabilities

Miller, L. — Mon, 04 May 2009 12:15:47 PDT

While formal, standardized assessment instruments provide valuable and necessary information about students' various abilities and skills, the use of informal and qualitative assessment approaches has the benefit of leading directly to instruction based squarely on an individual student's needs, strengths, and existing skills. This article describes a series of assessments that teachers can use to evaluate their students' writing abilities, knowledge, and goals. The assessments are described in the context of ongoing instruction and are accompanied by examples from the procedures used with a sixth-grade student.

Validating Curriculum-Based Measurement for Students With Emotional and Behavioral Disorders in Middle School

Fore, C., Boon, R. T., Burke, M. D., Martin, C. — Fri, 06 Mar 2009 12:49:38 PST

The purpose of this study was to investigate the concurrent validity of curriculum-based measurement (CBM) for students with emotional and behavioral disorders (EBD) in middle school using the Maze (MAZE); Oral Reading Fluency (ORF); and the Woodcock-Johnson, Third Edition (WJ-III), Passage Comprehension (WJ-PC) and Reading Fluency (WJ-RF) subtests. Fifty-five students in grades sixth through eighth with emotional and behavioral disorders participated in this study. Results indicated a strong correlation between the MAZE and the WJ-PC, while moderate correlations were found between the CBM Oral Reading Fluency and the Woodcock-Johnson subtests (passage comprehension and reading fluency). Limitations of the study, implications for practice, and future research are discussed.

Measures for Determining English Language Proficiency and the Resulting Implications for Instructional Provision and Intervention

Albers, C. A., Kenyon, D. M., Boals, T. J. — Fri, 06 Mar 2009 12:49:38 PST

Although numerous English language proficiency (ELP) measures currently exist, many were developed prior to the No Child Left Behind Act of 2001 (NCLB). These pre-NCLB measures typically focused on social language proficiency, whereas post-NCLB measures are linked to ELP standards and focus on academic language proficiency (ALP). ELP measures are typically used for accountability purposes and to determine eligibility for services; less attention has been given to their utility in enhancing classroom instruction and intervention provision. Inconsistency in scores between pre- and post-NCLB measures frequently leaves educators wondering whether English language learners (ELLs) have the necessary ALP to benefit from classroom instruction. This study investigates the intervention validity of ELP assessment by examining the concurrent validity of various pre-NCLB measures to a recently developed post-NCLB measure. As hypothesized, results indicate moderate correlations between pre- and post-NCLB measures, suggesting that ALP-focused post-NCLB measures are likely to provide more utility for ELL classroom instruction and intervention provision.

An Examination of Current Assessment Practices in Northeastern School Districts

Madaus, J., Rinaldi, C., Bigaj, S., Chafouleas, S. M. — Fri, 06 Mar 2009 12:49:38 PST

Despite the central role of assessment in special education, there is a paucity of current research on instruments and methods used in schools. Special education directors ( N = 164) in five northeastern states responded to an electronic survey related to the use of assessment instruments and methods in their districts. Data are presented regarding district decision-making related to instrument selection and purchase as well as professional development for district staff. The specific use of tests in academic, adaptive, intelligence, and content areas are also presented, as is the frequency of use of 11 assessment procedures, including response to intervention. Implications for preservice and in-service training are discussed.

A Comparison of Spelling Performance Across Young Adults With and Without Dyslexia

Coleman, C., Gregg, N., McLain, L., Bellair, L. W. — Fri, 06 Mar 2009 12:49:38 PST

In this study, the authors analyzed 2,056 spelling errors produced by 130 young adults (65 with dyslexia, 65 typically achieving), which came from two sources: a standardized spelling test and an impromptu essay-writing task. Students with dyslexia exhibited higher spelling error rates across both tasks. To characterize the inaccurate spelling attempts of both groups, the authors conducted linguistic and item-level analyses. Among unconstrained errors (essay), students with dyslexia had more difficulty than their typically achieving peers with familiar, low-level items (indexed by word frequency and number of syllables). Among constrained errors (spelling dictation), group differences in phonetic plausibility, morphological awareness, and visual accuracy varied by item. These analyses were telling on low-frequency items for which the groups obtained similar (dichotomous) accuracy rates. The authors suggest that diagnosticians and educators employ error analysis to obtain critical information not typically reflected in the standard scores used to make learning disability identification decisions.

Intervention Validity of Cognitive Assessment: Knowns, Unknowables, and Unknowns

Braden, J. P., Shaw, S. R. — Fri, 06 Mar 2009 12:49:38 PST

The intervention validity of cognitive assessment batteries is considered within an historical context to identify what the evidence supports (knowns), what cannot be known (unknowables), and what is not yet known (unknowns). Two ways cognitive batteries could inform intervention are identified: a disordinal (i.e., aptitude-treatment interaction) model and an ordinal (i.e., g) model. Existing research provides little support for disordinal models and modest to strong support for ordinal models when applied to conditions of incomplete instruction. Limitations to what can be known (e.g., a failure to prove is not proof of failure) are identified to avoid holding intervention validity research to inappropriate evidential standards. Finally, issues that are within the realm of knowledge but for which there is currently insufficient or no evidence are identified as logical next steps toward understanding the intervention validity of cognitive assessment batteries.

Technical Adequacy of Early Numeracy Curriculum-Based Measurement in Kindergarten

Martinez, R. S., Missall, K. N., Graney, S. B., Aricak, O. T., Clarke, B. — Fri, 06 Mar 2009 12:49:38 PST

The current study examines the technical adequacy of four Early Numeracy Curriculum-Based Measurement (EN-CBM) screening tasks: Oral Counting (OC), Number Identification (NI), Quantity Discrimination (QD), and Missing Number (MN). Results from 59 kindergarten students assessed in the fall and spring reveal moderate to high test—retest and delayed alternate-form reliability. In addition, data from the four measures demonstrate adequate concurrent and predictive validity by comparing them to an established measure of early numeracy. The measures demonstrated growth over time from the beginning to the end of kindergarten. Finally, combinations of the measures were evaluated post hoc, and the combination of NI, QD, and MN produced the highest reliability, validity, and rate of growth coefficients. Overall, results support the technical adequacy of EN-CBM for use with kindergarten-aged children.