Developing and testing the feasibility of computerized adaptive tests of reading comprehension for students in grades 4 through 9.
Our primary project work consisted of conducting the basic science necessary to develop and field test a reading comprehension computer adaptive test (RCCAT) useful for accurately and regularly measuring middle school student progress toward improved reading proficiency. The outcome product was a prototype of what expanded computer adaptive test (CAT) reading comprehension progress measures looked like when the program was brought into full implementation in field tryouts.
This project’s research efforts were a “proof of concept”, which has resulted in a two-phase CAT assessment that incorporates (a) a summative, or classification score, for a student’s ability level and (b) formative, or diagnostic progress measures, appropriate for upper elementary and middle school settings.
Imagine a middle school social studies classroom with 31 students. The academic term is starting and as the teacher reviews the students’ class work and a unit test, she concludes that these students are not only having significant difficulty with understanding the conceptual materials in the textbook, but that the students have even more fundamental difficulties with reading literacy. She suspects that in general the students are not efficient readers, and many do not appear skilled at reading for information, summarizing what they read, and making inferences or evaluative conclusions.
To check on her tentative conclusions and to plan how she might best assist the students, the teacher provides class time for the students to complete an online assessment of reading comprehension abilities.
Unlike the days of paper and pencil tests, these students access the reading comprehension test on a computer linked to the Internet. As the students complete the brief assessment, the teacher immediately reviews an individual student’s performance, including his or her reading comprehension abilities, and then abilities of the class as a whole. The results will confirm and/or correct the general impression that the teacher formed about the students, and provide a profile of each student and the class.
The teacher’s next concern is planning activities that will most efficiently help students improve literacy skills and therefore social studies content knowledge. She directs students to another Internet-based reading comprehension assessment. The students read several passages at their individual reading ability levels, and answer questions that assess specific reading comprehension abilities. The teacher receives very specific information on which to focus instruction in reading comprehension strategies. Every two to three weeks the students have an opportunity to take additional on-line reading comprehension assessments that test their progress in learning and applying the new comprehension strategies. In this way, the teacher can provide important instruction in developing the literacy and strategic reading comprehension knowledge and social studies content of her class.
In short, our goal in moving toward a reading comprehension assessment framework with CAT is not to create an assessment unto itself, but rather to continue to use assessment to support teachers’ classroom instruction. We believe the RCCAT addresses basic questions about the nature and understanding of our most complex reading activity – reading comprehension – and the psychometric features of CAT in a diagnostic or formative framework.
Don Deshler & John Poggio
Project Contact: Daryl Mellard
Phone Number: 785-864-7053