Beyond fidelity

This study explored how contextual factors influenced the effect of educational practices on student reading achievement and describes an alternative means of assessing educational programs under conditions of multiple-treatment interference and innovation diffusion. Over 1,500 reading teachers at 69 elementary schools within a large diverse district completed surveys regarding multiple aspects of the reading program, actions of their reading leaders, and instructional program coherence at their schools. Nearly 13,000 students in grades 2 through 5 were assigned to those teachers. Factor analyses were used to separately identify patterns within survey items that measured educational practices, leadership actions, and instructional program coherence. Then, the students' achievement gains were adjusted for the effects of fixed demographic and organizational variables through hierarchical linear modeling. Finally, classroom level relationships between the adjusted achievement gains, and subscales computed from the factors that were identified, were examined through a path analysis. Educational practices were found to align to six factors labeled Technology, Training Utility, Advanced Skills, Basic Skills, Grouping, and Assessment. Leadership actions were found to align to two factors labeled Relationship and Task. Fixed effects at the student, classroom, and school levels were found to have an impact on both the initial status and growth components of student achievement. In the path model, Task was found to have a significant direct effect on Advanced Skills, while the effect of Relationship on educational practices was partially mediated by Instructional Program Coherence. Both Advanced and Basic skills were found to have positive effects on Adjusted Gain when taught at the appropriate level, and negative effects, when taught at the inappropriate level.