Ontario secondary teachers frequently use data for problem-solving in online courses, but gaps in training, infrastructure, and strategic application limit their ability to transform data into meaningful instructional improvements.
Objective: This comprehensive dissertation aimed to investigate how secondary school teachers in Ontario engage with data-based decision-making (DBDM) in online learning environments. The research sought to understand the extent to which teachers use student data to enhance their instructional practices, identify the factors that influence their data usage, examine global trends in DBDM research, and determine what professional development opportunities are needed to strengthen teachers' data literacy skills in digital learning contexts.
Methods: The research employed a multi-faceted approach consisting of three interconnected studies. The first study was a systematic scoping review following PRISMA-ScR guidelines, analyzing 45 articles from four databases (Education Source, ERIC, Web of Science, and Academic Search Complete) to map global DBDM practices in K-12 education from 2013-2023. The second and third studies utilized an explanatory sequential mixed-methods design, beginning with a quantitative survey administered to 102 Ontario secondary teachers (92 English-speaking and 10 French-speaking) who taught online courses between 2021-2024. This was followed by eight qualitative semi-structured interviews lasting approximately 30 minutes each. The research framework was grounded in Schildkamp et al.'s (2017) determinant model, supplemented by additional frameworks from Dunn et al. (2013), Moussavi et al. (2020), and Wayman et al. (2016). Data analysis included Spearman correlation, regression analysis, and thematic analysis using both deductive and inductive coding approaches.
Key Findings: The research revealed several critical insights into DBDM practices in online secondary education. From a global perspective, DBDM research is heavily concentrated in the United States (62%) and Netherlands (25%), with significant gaps in secondary education contexts and online learning environments. In Ontario specifically, 77% of teachers reported using data to solve immediate problems, while only 57% used it proactively for future planning, indicating predominantly reactive rather than strategic engagement. Statistical analysis identified collaboration (p<0.05) and teacher efficacy as the strongest predictors of data usage, while anxiety and concerns about data quality emerged as significant barriers. Teachers demonstrated moderate confidence in data interpretation (M=3.07) but struggled with earlier stages of the DBDM process, including data identification (M=2.91), technological tool usage (M=2.57), and instructional application (M=2.75). Qualitatively, teachers expressed frustration with inadequate training, describing most professional development as "superficial" or "entirely absent," forcing them to rely on informal peer support and trial-and-error learning.
Implications: This research makes substantial contributions to the field of educational technology and teacher professional development by addressing critical gaps in our understanding of DBDM in online secondary education. The findings challenge existing assumptions derived primarily from elementary and in-person settings, demonstrating that digital learning environments present unique challenges requiring specialized approaches. The development of a new online DBDM process model specifically tailored for digital learning contexts provides a practical framework that can guide both teacher training and educational technology development. The research highlights the urgent need for educational systems to move beyond simply providing access to data tools toward creating comprehensive support ecosystems that include leadership support, collaborative structures, and meaningful professional development. The identification of artificial intelligence as a potential solution for streamlining data interpretation aligns with emerging trends in educational technology while emphasizing the importance of ethical implementation and teacher agency.
Limitations: Several limitations should be considered when interpreting these findings. The empirical research was geographically limited to Ontario, which may not reflect experiences in other provinces or international contexts where educational policies, technology infrastructure, and support structures differ significantly. The study relied heavily on self-reported data through surveys and interviews, which may be subject to recall bias and individual interpretations of data usage practices. The research did not directly measure the impact of DBDM practices on student learning outcomes, focusing instead on teacher experiences and perceptions. Additionally, the lower representation of French-speaking participants (10 out of 102) may have limited insights into linguistic minority contexts. The voluntary nature of participation may have introduced self-selection bias, potentially overrepresenting teachers who are more engaged with or interested in data use.
Future Directions: The research identifies several important avenues for future investigation. Comparative studies across different Canadian provinces could illuminate how varying educational policies, professional development opportunities, and technological infrastructures influence teacher engagement with DBDM. Longitudinal research is needed to assess whether targeted DBDM training interventions translate into sustained improvements in teaching effectiveness and student achievement over time. As artificial intelligence becomes increasingly integrated into educational systems, studies examining how AI-driven tools can support data interpretation while maintaining teacher agency and ethical considerations will be crucial. International comparative research could provide broader perspectives on effective DBDM implementation strategies across diverse educational contexts. Finally, research directly linking improved DBDM practices to student learning outcomes would strengthen the evidence base for investing in teacher data literacy development.
Title and Authors: "Data-Based Decision Making in Online Secondary Courses: A Multi-Level Examination of Practices, Determinants, and Needs" by Areej Tayem
Published on: 2025
Published by: University of Ottawa (Doctoral Dissertation)