Source: Journal of Research on Technology in Education, Vol. 42, Iss. 4, (Summer, 2010), p. 361-389.
(Reviewed by the Portal Team)
The current paper describes the development and validation of the Student Tool for Technology Literacy (ST2L).
The ST2L was designed to be a flexible tool for the assessment of student technology literacy in order to support the integration of technology into the curriculum and students' daily learning experiences.
This paper reports the development procedures and results of the pilot test conducted to validate the functioning of this online, interactive tool.
Prior to its development, the purpose of the ST2L was determined to be for low-stakes assessments for monitoring technology literacy of eighth grade students within the state of Florida. The results of the ST2L were intended to be used for NCLB reporting of aggregated school-level results, school district curricular purposes, and individual students identifying relative strengths and weaknesses in their technology literacy. Based on these stated low- stakes purposes, appropriate development processes were followed throughout the project.
For the initial stage of construct definition, the indicators were carefully developed and based on state and national technology standards (NETS-S). The development team also followed sound development procedures for item writing and review and conducted usability analyses to ensure that the user interface and the simulated performance-based tasks were as clear and intuitive as possible.
Finally, the pilot test team validated functioning according to a sound research design and sampling plan.
Participants
A pilot study to validate the ST2L was conducted in the spring of 2008 with 1,561 eighth grade students from 40 middle/junior schools in 13 school districts in Florida.
Analyses focused on item difficulty and discrimination by ability groups, completion time analysis, internal consistency reliability, and construct validity.
ST2L was found to be a sound assessment tool for the intended purpose of low-stakes assessment of technology literacy.
The process of developing and validating the ST2L can be used as a model for others who would like to develop innovative performance assessments of technology literacy skills.
With few modifications, the ST2L was ready for low-stakes implementation and should become a useful tool of school districts for reporting aggregated technology literacy scores of schools for NCLB purposes and for helping students and districts target technology-related curricular needs.
In the 2008-09 school year, the ST2L was made available for districts in Florida to use with their students. In addition to using the tool for NCLB aggregated school-level reporting purposes, teachers can adapt the delivery to meet their students' instructional needs.
Furthermore, teachers could have the students take the assessment again as the posttest and help them identify the growth in their technology skills by comparing the changes in their scores.
In addition, students can also use the ST2L to monitor and track the progress of their technology literacy skills. This information could guide their choices of courses or projects within courses.
Schools could also use the ST2L to measure and track the longitudinal growth of their schools' level of technology literacy skills. Special programs could be created to support the development of technology literacy of special groups of students or to support the development of the whole school's specific technology skills.
Finally, districts might use the longitudinal data collected to plan, fund, and monitor special technology initiatives.