Source: Journal of Teacher Education, Vol. 68(1) 85-101, 2017
(Reviewed by the Portal Team)
In this study, the authors examine the strategies reported by naïve assessment constructors. Naïve assessment constructors refer to those individuals with limited, if any, formal preparation for constructing classroom assessments.
Method
Fifty-three students, from five undergraduate educational psychology classes, agreed to participate and provided complete data for this investigation.
The authors used inductive thematic qualitative analysis to examine open-ended written rationales provided by two groups of 53 naïve assessment who they asked to select items for an end of unit test. They randomly assigned participants to an informed and uninformed condition (i.e., informed participants read an article describing a Table of Specifications).
Through recursive thematic analyses of participants’ explanations, the authors identified 14 distinct strategies that coalesced into three families of strategies: Alignment, Item Evaluation, and Affective Evaluation. This indicates that these may be preliminary or core strategies that are needed during test construction.
Next, the authors examined the strategy use of participants in the informed and uninformed groups. Participants in both groups consistently relied on rationales that reflected attention to issues related to Alignment for item selection. The authors argue that preservice teachers may need an opportunity to practice the strategies that support that use through experiences that provide them with feedback and opportunities to observe others engaged in a similar task.
Participants in the uninformed group relied on Item Evaluation and Affective Evaluation strategies more often than the informed group to guide item selection. These findings suggest preservice teachers who were not taught otherwise relied more on affective strategies and personal opinion. Therefore, teacher educators need to elicit preservice teachers’ intuitive strategies, and teach them how to determine when to rely on these strategies and when to utilize strategies that reflect more sound assessment practices.
The authors suggest that teacher educators can guide learners to more appropriate strategies within each family and facilitate deliberate practice on their use.
In addition, the emergence of (a) three families of strategies, (b) the use of multiple strategies by item selected, and (c) the use of multiple strategies across items by individuals indicates that these naïve assessment constructors weighed multiple perspectives when completing the test construction task. Naïve assessment need to engage in meaningful reflection on and dialogue with the goals and purposes of assessment activities as related to the goals and purposes of the overall educational enterprise.
The authors conclude that this investigation underscores the complex interplay of cognition, knowledge, and affect that are engaged when preservice teachers construct classroom assessments. The range and depth of strategies employed by novices completing this task evidences the need for explicit instruction in supportive strategies that can become automated skills.