Assessment for Effective Intervention recent issues

From the Publications Director

Kipping, P. — 2008-08-08

From the New Editor

Burns, M. K. — 2008-08-08

Curriculum-Based Measurement for Mathematics: Introduction to the Special Series

Hosp, J. L. — 2008-08-08

Implications of Recent Research: Curriculum-Based Measurement of Math Computation

Christ, T. J., Scullin, S., Tolbize, A., Jiban, C. L. — 2008-08-08

Curriculum-based measurement of mathematics (CBM-M) comprises a set of procedures and instrumentation to assess the level and trend of student achievement in early mathematics. The purpose of this article is to review the recent research and psychometric evidence for CBM-M. Although recent developments in CBM-M include procedures to assess early numeracy and application problems, this review focuses exclusively on computation assessment. The results of this review provide evidence that CBM-M is sufficiently reliable and valid for some applications; however, interpretation must be informed by the context and the scope of assessment domain. Mathematics computation is a subdomain of mathematics curriculum and assessment, and therefore, the validity of CBM-M is limited by its construct representation (i.e., stimulus set and task demands). Nevertheless, the review provides support for ongoing development and use of CBM-M as both a general outcome measure and subskill mastery measure for computation. Implications for research and practice are discussed.

Establishing Technically Adequate Measures of Progress in Early Numeracy

Lembke, E. S., Foegen, A., Whittaker, T. A., Hampton, D. — 2008-08-08

The purpose of this study was to examine the use of three early numeracy measures to monitor the mathematics progress of students across time. One hundred and seven kindergarten and Grade 1 students were administered quantity discrimination, number identification, and missing-number measures once each month for 7 months. Alternate form reliability was adequate for instructional decision making, whereas criterion validity coefficients comparing the early numeracy measures to teacher judgment of student proficiency in mathematics and students' performance on a district-administered standardized test were lower than those observed in previous research. We used hierarchical linear modeling at each grade level to examine the ability of the three measures to model growth across time. All measures produced growth rates that were significant across time, for each grade level, with linear growth observed for the Number Identification measure only.

Examination of the Utility of Various Measures of Mathematics Proficiency

VanDerHeyden, A. M., Burns, M. K. — 2008-08-08

This article investigates the utility of various estimates of mathematics proficiency. The participants were 432 students in Grades 2 through 5. The delayed alternate form reliability of multiskill probes, retention probes, slopes of student growth, and trials to criterion were computed. The fluency probes were found to be both sufficiently reliable and significantly predicted proficiency on a group-administered accountability test. However, the trials to criterion and slope data were neither reliable nor significantly predictive. Finally, a receiver operating characteristic analysis was conducted to determine cut scores on the fluency probes for proficiency. Results found that 17 digits correct per minute for second- and third-grade students and 29 digits correct per minute for fourth and fifth graders demonstrated proficiency. These data resulted in overall correct classification estimates of .65 (second and third grades) and .73 (fourth and fifth grades).

A Curricular-Sampling Approach to Progress Monitoring: Mathematics Concepts and Applications

Fuchs, L. S., Fuchs, D., Zumeta, R. O. — 2008-08-08

Progress monitoring is an important component of effective instructional practice. Curriculum-based measurement (CBM) is a form of progress monitoring that has been the focus of rigorous research. Two approaches for formulating CBM systems exist. The first is to assess performance regularly on a task that serves as a global indicator of competence at the relevant grade level. The second approach is to systematically sample the year-long curriculum so that each skill is represented and receives the same emphasis on each alternate form. In this article, the systematic curriculum sampling approach is illustrated for monitoring progress in mathematics concepts and applications systems. A description of the system's components, background, and technical properties is provided. Then, a sample case explains how the CBM system can be used in a special education setting to monitor progress, plan instruction, and enhance communication.

Curriculum-Based Measurement for Mathematics at the High School Level: What We Do Not Know . . . What We Need to Know

Calhoon, M. B. — 2008-08-08

The purpose of this article is to explore the research and developmental needs for curriculum-based measurement (CBM) at the secondary level (9th through 12th grades) for mathematics. Much has been accomplished empirically on the validity, reliability, and utility of CBM as an assessment measure in mathematics at the elementary level. Encouragingly, a few CBM mathematics studies are beginning to emerge at the early (preschool through 1st grades) and middle school levels (6th through 8th grades); however, only one study exists on the use of CBM as an assessment tool for secondary-level mathematics courses. First, the author discusses the current mathematics predicament of secondary students with and without disabilities and their teachers. She then, within the different educational settings (general education, inclusion, resource, self-contained) available at the secondary level, discusses the types of CBM research (technical adequacy, instructional utility) needed.

Developing Progress Monitoring Measures for Secondary Mathematics: An Illustration in Algebra

Foegen, A., Olson, J. R., Impecoven-Lind, L. — 2008-08-08

Secondary special educators are in desperate need of progress monitoring measures in mathematics that appropriately reflect their students' instructional content. As educators across the country respond to increasing standards for challenging curricula, growing numbers of students with disabilities are participating in more advanced mathematics courses at the secondary level, including algebra and geometry. Unfortunately, there are few measures in existence specifically designed for these content areas. In this article, the authors describe the work of Project AAIMS (Algebra Assessment and Instruction: Meeting Standards), a federally supported effort to develop and validate progress monitoring measures in algebra. The authors' focus is on describing the process used and offering recommendations for professionals who wish to develop their own measures for secondary mathematics.

Curriculum-Based Evaluation and Math: An Overview

Kelley, B., Hosp, J. L., Howell, K. W. — 2008-08-08

Poor math performance is a major concern leading the current educational reform agenda. Many educational math critics are claiming a math crisis. Researchers and school personnel are trying to resolve this crisis. Some are looking at curriculum-based evaluation (CBE) as a process for solving this problem. CBE and curriculum-based measurement (CBM) are widely accepted validated tools for effectively yielding information necessary to make sound educational decisions. In this article, best practices in CBE and CBM are described as they pertain to math instruction. Specific steps of the CBE process are described as they pertain to math.

From the Editors

Elksnin, N., Elksnin, L. — 2008-05-16

From the Publications Director

Kipping, P. — 2008-05-16

Assessment of Culturally and Linguistically Diverse Learners With Disabilities: Introduction to the Special Series

Barrera, M. — 2008-05-16

Examining the Dependability of Academic Achievement Measures for English Language Learners

Solano-Flores, G., Li, M. — 2008-05-16

The dependability of academic achievement measures for English language learners (ELLs) is influenced by three facts: (a) Each ELL has unique strengths and weaknesses in each language mode (listening, speaking, reading, and writing) both in English and in his or her first language, (b) each test item poses a different set of linguistic demands that do not replicate across languages, and (c) raters may vary in their ability to properly interpret students' responses. Generalizability theory allows examination of the magnitude of score variation due to the language (or dialect) in which tests are administered. The interaction of student, item, and language (and dialect) is the most important source of score variation. The language in which students should be tested and the minimum number of items needed to produce dependable measures of academic achievement for ELLs may be different across communities within a broad group of native speakers of a language.

Instructional Strategies for Improving Achievement in Reading, Mathematics, and Science for English Language Learners With Disabilities

Shyyan, V., Thurlow, M. L., Liu, K. K. — 2008-05-16

Effective reading, mathematics, and science instructional strategies for English language learners with disabilities participating in state accountability assessments are identified by educators of these students. Hmong students with disabilities indicate the importance of various strategies for their learning in a standards-based, assessment-driven environment. For 72 educators, reading, then math, is seen as the highest importance content area, whereas for 25 students, mathematics, then reading, is seen as the highest importance content area. Several specific instructional strategies are identified as effective through the Multi-Attribute Consensus-Building process, although some variability emerges. Curriculum-based probes have considerable variability in educator weightings, but students weight them as important, feasible, and frequently used. Native language is not frequently mentioned and not particularly supported by students. Implications for further research are discussed.

Accommodating Students With Disabilities on State English Language Proficiency Assessments

Albus, D., Thurlow, M. L. — 2008-05-16

All English language learners (ELLs) are to participate in an annual assessment of English language proficiency (ELP) under Title III of the No Child Left Behind Act of 2001. This includes ELLs with disabilities, regardless of whether they are high-incidence disabilities (e.g., learning disability) or low-incidence disabilities (e.g., deaf or hard of hearing). This article summarizes findings from a national study of state ELP accommodation policies for ELLs with disabilities on these state assessments. It highlights the policy differences across states in the accommodations offered and also provides information on state-specific policies. Several recommendations based on the analysis are provided.

Universal Design Considerations for Improving Student Achievement on English Language Proficiency Tests

Liu, K. K., Anderson, M. — 2008-05-16

This article studies accessible assessment design to large-scale English language proficiency assessments that are now mandatory for elementary and secondary English language learners in public schools. Using a modified Delphi approach, a panel of 33 experts from the areas of assessment, English as a second language or bilingual education, and special education provided feedback on features of accessible English language proficiency tests. Experts rated the importance of a set of accessible test design considerations suggested by the literature and were able to add and rate additional considerations suggested by their own experience. Considerations related to the creation of concise and readable text in assessment items received consistently high ratings overall. In contrast, considerations relating to the creation of test items that allow for format changes (e.g., Braille, oral presentation, sign language) showed the most variability in ratings.

From Early Childhood Special Education to Special Education Resource Rooms: Identification, Assessment, and Eligibility Determinations for English Language Learners With Reading-Related Disabilities

Liu, Y.-J., Ortiz, A. A., Wilkinson, C. Y., Robertson, P., Kushner, M. I. — 2008-05-16

Results of an exploratory study of eligibility decisions made for 19 Spanish-speaking English language learners (ELLs), who qualified for special education in early childhood and subsequently received special education reading instruction, are reported. Full and individual evaluations revealed that only 9 students demonstrated significant Intelligence Quotient–reading achievement discrepancies. Because the study's purpose was to describe characteristics of ELLs with reading-related learning disabilities (LD), these students became the focus of analyses. After examining data from students' school records, an expert panel concluded that 1 student had reading-related LD, 3 had disabilities other than LD, and 5 had problems that could be explained by other factors. Issues associated with the special education identification process for ELLs are discussed and recommendations for improving practice are offered.

Assessment for Instructional Planning

Hosp, J. L., Ardoin, S. P. — 2008-02-20

Assessment is a common task in education that has many varied purposes. One of these is the use of assessment data to make decisions about instructional planning. This article presents a framework for how to use assessment data to plan instruction. It also provides some background on terminology of assessment for instructional planning, considerations for the development or use of assessment measures, and guidance for designing instruction.

Conducting Assessments in Technology Needs: From Assessment to Implementation

Wissick, C. A., Gardner, J. E. — 2008-02-20

Practitioners with an expertise in assistive technology and technology assessments are in demand to be full participants in the selection, planning, and implementation of instruction for students with mild disabilities. Frequently, practitioners with knowledge of assistive technology are assigned to evaluate students with sensory, physical, language, or severe disabilities. Our article highlights aspects of technology assessments and progress monitoring that can be used for students with mild disabilities. Given the impact that technology integration can have on the access that students with mild disabilities have to the general education classroom, we argue that all practitioners should be cognizant of protocols for assistive technology assessment and evaluation and that all evaluation teams should include an assistive technology specialist.

Understanding the Construct of Self-Determination: Examining the Relationship Between the Arc's Self-Determination Scale and the American Institutes for Research Self-Determination Scale

2008-02-20

Since the early 1990s, attention has been focused on the importance of self-determination in the education of students with disabilities. The purpose of this study was to further our understanding of the construct of self-determination by examining the relationship between the Arc's Self-Determination Scale and the American Institutes for Research (AIR) Self-Determination Scale student and educator versions. Using structural equation modeling, we found that the theoretical structure of the Arc's Self-Determination Scale and the AIR Self-Determination–Student Scale was supported by the data, while the proposed theoretical structure of the AIR Self-Determination–Educator Scale was not. The analyses suggested that each of the measures of self-determination was measuring a different aspect of the self-determination construct (i.e., it was not possible to create a higher order factor comprising each of the assessments); thus issues related to the goals of the research and the underlying theoretical perspective of each measure must be considered when determining the most appropriate measure of self-determination in research and practice. Implications and future research directions are discussed.

Analysis of Student Revisions on a State Writing Test

Crawford, L., Lloyd, S., Knoth, K. — 2008-02-20

Type and quality of revisions made by students between first and final drafts of a state writing test were scored using a revision taxonomy. Scorers categorized revisions first by unit (e.g., word, phrase, sentence), and then by type (e.g., addition, substitution, spelling). They then evaluated the impact of each revision on the readability of the final draft as "improved," "decreased," or "neutral." Descriptive analyses revealed three major findings. First, in terms of unit changes, word revisions were the most prevalent (approximately 40%) across students in Grades 5 and 8. Second, when coding for type of revision, substitutions accounted for approximately 45% of all revisions, while revisions classified as "additions" had the most positive impact on students' final drafts (i.e., positively impacting readability 66% to 76% of the time). An interesting third finding was the relationship between rate of writing and score on the writing test: Students in special education with Individualized Education Programs goals in writing made approximately the same number of revisions per 100 words as students in the general population but wrote 100 words less, on average.

Systematic Direct Observation of Time on Task as a Measure of Student Engagement

Spanjers, D. M., Burns, M. K., Wagner, A. R. — 2008-02-20

This study examines the relationship between systematic direct observation (SDO) of time on task (TOT) and student engagement as measured by a self-report task-specific measure. The research questions guiding the study are (a) what is the relationship between SDO of TOT and a self-report measure of student effort, and (b) what is the relationship between TOT and reading comprehension? An exploratory analysis compares the engagement among three levels of passage difficulty. Small partial correlations are found between TOT and self-reported engagement among 125 third and fourth graders (r = –.15 and r = .30, respectively). Nonsignificant partial correlations between TOT and reading comprehension are also found. Exploratory multivariate analyses of the three variables show significant results for both third and fourth grade, but subsequent univariate analyses find a significant effect only for comprehension. Thus, these data suggest the need for additional research regarding the assessment of TOT with SDO.

J. Lee Wiederholt 1942-2007

Hammill, D. D. — 2007-01-01

Leveling the Digital Playing Field: Assessing the Learning Technology Needs of College-Bound Students With LD and/or ADHD

Parker, D. R., Banerjee, M. — 2007-01-01

As increasing numbers of students with learning disabilities (LD) or attention-deficit/hyperactivity disorder (ADHD) transition to postsecondary education, they encounter a heightened need for proficiency with a wide range of learning technologies. Whereas the Individuals With Disabilities Education Improvement Act of 2004 (IDEIA) requires consideration of a student's assistive technology training needs in high school, the Americans With Disabilities Act does not require any evaluation or training services for postsecondary students with disabilities. In an era of measurable outcomes, it is critical for secondary school personnel to consider effective assessments and relevant interventions when college-bound students with cognitive disabilities lack proficiency with these technologies. Survey research at a highly competitive public university found significant differences between the technology needs, preferences, and fluency of undergraduates with and without disabilities. This article presents findings from that study as well as implications for teachers and evaluators who assist students with LD and ADHD in their transition planning for postsecondary education.

Item-Level Effects of the Read-Aloud Accommodation for Students With Reading Disabilities

Bolt, S. E., Thurlow, M. L. — 2007-01-01

Research support for providing a read-aloud accommodation (i.e., having an individual read test items and directions aloud) to students with disabilities has been somewhat limited, particularly when merely examining effects of the accommodation on overall test scores for general groups of students with disabilities. We examined data on accommodated and nonaccommodated performances of students with specific reading disabilities on math test items anticipated to be highly sensitive to accommodation effects due to their level of reading and math difficulty. We conducted analyses across 3 consecutive years of data from an elementary and middle school statewide assessment program. Within the elementary data set we identified a significant Accommodation Group x Reading Difficulty interaction effect, such that the accommodation appeared to more positively impact student performance on items that were classified as difficult to read. We describe other identified effects and discuss limitations of the study, as well as implications.

Measuring the Enacted Curriculum for Students With Significant Cognitive Disabilities: A Preliminary Investigation

Karvonen, M., Wakeman, S. Y., Flowers, C., Browder, D. M. — 2007-01-01

Based on recent federal legislation, alternate assessments for students with disabilities may now be based on alternate achievement standards, modified achievement standards, or grade-level achievement standards. Although all students with disabilities must access the general curriculum, those with significant cognitive disabilities often do so through extensions of grade-level content standards. Because curriculum is individualized for students with disabilities, experts cannot immediately apply to this population the methods for examining the taught curriculum in general education. The purpose of this article is to describe the development and use of a method for examining the enacted curriculum for students who take alternate assessments. We present initial item development, survey blueprint, expert review, and pilot test findings. Experts can use this tool to investigate the alignment of curriculum with alternate assessments and state standards and to design professional development for educators learning to access the general curriculum.

Investigation of the Psychometric Attributes of the Test of Silent Contextual Reading Fluency

2007-01-01

The purpose of this study was to examine the psychometric integrity of the Test of Silent Contextual Reading Fluency (TOSCRF) by D. D. Hammill, J. L. Wiederholt, and E. A. Allen (2006). The TOSCRF is a recently published assessment of reading fluency for ages 7 years 0 months to 18 years 11 months that can be administered individually or in a group. The TOSCRF has 4 alternate forms and is brief; administration time is approximately 10 min. Because of its brevity, availability of alternate forms, and the possibility of group administration, the TOSCRF holds promise as an efficient screening and progress-monitoring tool. The authors report evidence of reliability and validity but further, independently obtained validity evidence is needed.

Test Review: Mayer, J. D., Salovey, P., & Caruso, D. (2002). Mayer-Salovey-Caruso Emotional Intelligence Test. Toronto, Canada: Multi-Health Systems

Skaar, N. R. — 2007-01-01

Test Review: Early Childhood Research Institute on Measuring Growth and Development. (1998). Selection of General Growth Outcomes for Children Between Birth and Age Eight (Tech. Rep. No. 2). Minneapolis, MN: Center for Early Education and Development. Reviewed by Alisha K. Wackerle

Wackerle, A. K. — 2007-01-01

Test Review: Lambert, N., Nihira, K., & Lel, H. (1993). AAMR Adaptive Behavior Scales: School (ABS-S:2)

Lyman, W. C. — 2007-01-01

Test Review: Newcomer, P. L. (2001). Diagnostic Achievement Battery (3rd ed.). Austin, TX: PRO-ED

Decker, D. M., Lee Haight, S. — 2007-01-01