Schooling in the United States relies on testing – a lot of testing – to sort students and evaluate programs and schools. When it comes to reading, tests gather information on student skills such as reading comprehension or reading ability. However, many of these tests don’t s provide information to teachers about exaclty how their students are reading. In a recent paper by Anthony Clairmont and myself, written with Dr. Diana Arya and Dr. Andy Maul, we provided initial evidence for the validity of a measure of reading strategy use that does just that.
Called the Strategy Use Measure (SUM), this testing measure is intended to be used by teachers to help them modify teaching and learning activities to improve adolescent readers’ use of four reading strategies, or “dimensions.” There were two questions we were hoping to answer through our study:
- Does each of the four reading dimensions represent a unique aspect of reading that a teacher could easily identify using the Strategy Use Measure?
- Can you use the same test for native English-speaking students as well as for native Spanish-speaking students?
The SUM and its Parts
Using elements from the BEAR Assessment System and the Collaborative Strategic Reading (CSR) as guidance, Dr. Diana Arya began designing the SUM over seven years ago while at the University of Colorado Boulder. Over the course of the measure design, the team has included reading experts, psychometricians, and linguists.
The SUM is intended to measure the following four distinct dimensions
Morphological Awareness (MA): Morphemes are the roots of words – the smallest units that make words work. For instance, the prefix (un-), as in “undo” or “unfortunate.” To account for students’ prior vocabulary knowledge, the SUM used made up words. For instance, students were asked what “unsad” means.
Contextual Clues (CC): We often use sentences or words around unknown or unfamiliar words to make meaning of those unfamiliar words. This section of the assessment used made up words embedded in passages to assess student capabilities in using surrounding words or sentences to figure out the meaning of those words.
Macro and Micro Relationships in Text (MMRT): Perhaps the most dynamic and emergent dimension, this is the theorized strategy used by early readers to make meaning of individual sentences or larger passages. It requires readers to make the inferential leap from large points to small details or the other way around.
Cognates (COG): Cognates are words that are shared across languages. The SUM had a section devoted to Spanish-English cognate knowledge, asking about English words that are similar to particular Spanish words.
What Can we Say about using the Strategy Use Measure and About Students?
Using statistical modeling (the Rasch model), we were able to show that student responses to the SUM supported the idea of at least four meaningful strategy use dimensions (as discussed above, though there certainly could be more!). For over half the students who took this test, a teacher using a multidimensional measure would make a meaningfully different decision about a given student than if using measure based on just one general dimension. This finding was supported by individual student cognitive interviews in which students, unprompted, stated the strategy they used to answer questions from the SUM.
Interestingly, students who reported speaking primarily Spanish at home, performed better on certain test questions and worse on others than students who reported speaking primarily English at home. For instance, on the question asking about the meaning of the made-up word “heterocoloreous,” Spanish speakers did much better than native English speakers.
One of the research team members, Anthony Clairmont, discovered that native Spanish speakers did much better than native English speakers on words or morphemes that were Latin-based. The fact that the skills being used were actually related to the construct of interest supports the idea that we are indeed measuring reading strategy uses.
Conclusions and Future Work
From an assessment perspective, we show that it is possible to create measures of reading that are potentially useful for tailoring reading instruction. We also found that this measure can detect differences in strategies used by students of different language backgrounds. Other areas for research and development continue. For instance, how do we differentiate students who read like native Spanish speakers and those who do not but identify as native Spanish speakers? What particular curriculum interventions can we use based on the findings?
Finally, a version of the SUM is currently being used to guide teacher instruction at a local suburban school to help understand and improve student learning.
About the Author
Daniel Katz is a third year graduate student in the Gevirtz Graduate School of Education at the University of California Santa Barbara. His research interests include measurement, quantitative methodology, and student success. He can be contacted at firstname.lastname@example.org.