Findings from the APT Validation Study
Year Published: 2012
The Assessment of Afterschool Program Practices Tool (APT), developed by the National Institute of Out-of-School Time (NIOST), is an observational instrument designed to measure the aspects of afterschool program quality that research suggests contribute to the 21st century skills, attitudes, and behaviors youth need to be successful in school and the workplace. The APT is widely used across the country to support self-assessment and program improvement efforts. Increasingly, the APT is being used by external stakeholders for quality monitoring, assigning quality levels (e.g., as part of a Quality Rating and Improvement System), and identifying programs in need of improvement.
In 2010, with generous funding from the W.T. Grant Foundation, researchers from the Wellesley Centers for Women, Wellesley College, conducted the APT Validation Study with the aim of assessing the strength of the APT as a measurement tool. Based on observations of 25 afterschool programs serving grades K8 in Massachusetts, this study provides scientific evidence that the APT possesses many strong technical properties. Among the study' s many findings, researchers found that the APT captures key aspects of quality, such as whether a program is offering a welcoming environment or promoting youth engagement, which were found to be connected with positive youth program experiences and beliefs about themselves. APT ratings by a single observer are stable over time, which suggests that individuals can use the tool consistently to capture aspects of quality that are not overly sensitive to day-day fluctuations in practices. Practitioners do not tend to consistently rate their program higher or lower than outside observers who are unfamiliar with the program, suggesting that the APT can be used equally well by a variety of trained observers. Overall, the study suggests that the APT is an appropriate measure for examining afterschool program quality and is suitable for a number of lower-stakes purposes such as self-assessment and program support.
Read Executive Summary>>