The study was conducted in an urban school district with 43,000 students. Four elementary schools were selected. One school implemented the Standards Plus materials as the experimental treatment and three did not. All schools in the experiment had over 700 students. The schools were selected based on academic and demographic criteria. To ensure consistent and strictly compliant implementation, district office staff monitored and managed the Standards Plus implementation in the treatment school. All schools in the study, including the treatment school, employed the district’s adopted curriculum. All schools in the experiment, including the treatment school, were closely monitored by district staff to ensure they complied with all district policies and school improvement measures.
Schools in the experiment were matched for comparison in two ways. One criterion was performance on the state’s yearly achievement test. The other criterion was demographics.
The State established a one to ten scale for two comparison criteria. Schools were assigned two scores on a one to ten scale. On the achievement scale, a score of One identified schools in the lowest 10% of all schools based on statewide test scores. The other was based on school demographics. Schools were divided into deciles according to demographics (English Learner percentage, average parent education, percent of students participating in free/reduced-price meals, etc.). A score of One indicated the 10% of schools with the most challenging demographics.
All the schools in this experiment were designated by the state as One-One schools. In other words, the four schools in this experiment were among the lowest 10% of all schools based on State-wide achievement test results. They were also designated among the 10% of schools with the most challenging populations. The district accepted these comparable school indicators as valid and rigorous criteria to match schools for this experiment.
Performance data for this experiment came from the State’s Academic Performance Index (API). The API provides one number for every school from a state developed scale of 200 to 1,000. The assigned API number is based on scores from the state’s yearly achievement test. The API is calculated from all the student scores for that school in Language Arts and Mathematics.
For this experiment, Standards Plus implementation outcomes were determined by the state’s designated API. The state established growth targets for each school based on their distance from a state established performance target. The state’s performance target was a score of 800 out of a possible 1,000. The state required schools to improve their performance each year by 5% of the difference between their API and the statewide target of 800. For example, a school with an API of 340 would have a growth target of 23. A school with an API between 691 and 795 would need to gain five points.
The Federal Government accepted the API system as a valid measure for school success. The District applied the state established API scores as valid and reliable growth indicators for each school and to consequently determine the successful outcome of the Standards Plus implementation.
The table shows the state determined growth targets for each school and each school’s actual growth. All schools met their growth targets. The District was working under an improvement mandate and initiated multiple school improvement efforts. The District attributes the success of all the schools in the experiment to this circumstance. All the schools including the Treatment School implemented all the District initiatives and were closely monitored to ensure compliance. The Treatment School also implemented Standards Plus. The purpose of this experiment was to determine if the Standards Plus implementation led to a significantly different outcome than the Control Schools’ that did not implement Standards Plus. Analysis revealed that the Treatment School’s growth was significantly different than the Control Schools’ growth.
The district did not create the experiment with the intent to publish the outcomes or to provide reports to any other agency. The experiment was designed and conducted to fulfill the district’s desire to meet the needs of its most impacted populations. The experiment was conducted to establish the content validity of the Standards Plus materials and revalidate the findings of the implementation design described in the Standards Plus white paper. In both cases the district deemed that the outcomes of the experiment provided incontrovertible evidence that the Standards Plus materials fulfilled all requirements for a Level Two ESSA designation, and lacking only randomized assignment, met the criteria for a Level One ESSA designation. The outcomes of the experiment were so convincing that the District implemented Standards Plus in every elementary school. Anecdotally, the District received an Ed Trust Award for District-wide success in Raising Student Achievement, an unsolicited outcome of the District-wide Standards Plus implementation.