The Education Technology Industry Network (ETIN) and Empirical Education Inc. recently released the Guidelines for Conducting and Reporting EdTech Impact research in U.S. K-12 Schools to address the changes brought by the Every Student Succeeds Act (ESSA). These guidelines aim to clarify how research is conducted and how information is presented to users of edtech products. In the article “Measuring EdTech Impact in the ESSA Era,” experts explored the details of these guidelines and the strategies for success in measuring the impact of educational technology.
The updated guidelines take into account the prevalence of cloud-based edtech products, providing more access to teacher and student usage data. They also consider the compressed development timeline of edtech products and the shift in standards of evidence with ESSA replacing No Child Left Behind (NCLB). Districts are increasingly requesting their own student and teacher product usage data to conduct their own studies. The structure and definitions provided by these guidelines are valuable in helping districts obtain the data they need and effectively evaluate the impact of edtech products.
The guidelines are divided into four main sections: Getting Started, Designing the Research, Implementing the Design, and Reporting the Results. When starting a research study, utilizing a logic model from the product provider can help develop a model for how the product operates. Logic models can illustrate the factors that correlate with different outcomes. In the design phase, researchers should consider the four levels of evidence defined by ESSA and use a logic model as the first step. This can be followed by conducting a correlational study, a comparison study, and potentially a randomized experiment. Comparison studies are common due to the time and cost associated with randomized experiments.
It is crucial to handle confidential information with caution during the implementation phase of the research. With an increasing amount of personally identifiable student information available from both school districts and edtech products, privacy concerns regarding edtech usage data have become more prominent. When reporting the results of edtech product evaluations, it is essential to make all findings available. Publishing only the most favorable evaluation out of multiple studies does not provide a comprehensive view for the market. Reports should include enough detail to determine if the results are applicable to a specific context, helping schools make informed decisions about the suitability of a product.
Denis Newman, the lead author of ETIN’s guidelines for research on edtech impact, brings 35 years of experience in improving student-teacher learning processes and instructional technologies. As the CEO of Empirical Education Inc., he has conducted numerous randomized controlled trials (RCTs) and evaluations of school programs. Andrew Coulson, the Chief Data Science Officer, oversees data analysis and evaluation at MIND, focusing on expanding strategies and market operations. With a background in STEM and education, Coulson brings valuable expertise to the field of edtech research. Bridget Foster, EVP & Managing Director of ETIN, leverages her extensive experience in education leadership to help companies navigate the education market and enhance their global presence.
In conclusion, the Guidelines for Conducting and Reporting EdTech Impact research offer valuable insights for researchers, educators, and edtech providers seeking to measure the impact of educational technology in K-12 schools. By following these guidelines and adopting key strategies for success, stakeholders can effectively evaluate the efficacy of edtech products and make informed decisions to support student learning. To access the full guidelines and learn more about measuring edtech impact in the ESSA era, readers can download the complete document.