Validation and Methodology
Validation and Methodology
District impact rate studies
Four consecutive years. Over 150 district studies. Over half a million students. One consistent result: robust repeating of learning impacts.
Coverage and scale of district impact rate studies over time. Larger circles indicate more students per study, while color reflects number of studies per district, highlighting variation in study intensity and reach across districts.
We don’t stop Proving it. We keep measuring it. Because learning results are too important not to repeat.
Each year, on request MIND evaluates student outcomes for its partner districts across the country using any standardized summative assessments.
The broad demand for evidence “at our district” reflects real-world implementations at increasing scale and reveals that ST Math impact is consistent across settings and assessments, student demographics and repeated over school years.
Holding ourselves to a higher standard.
At MIND Education, we believe every school should see results.
Our Quasi-Experimental Design (QED) methodology extends far beyond the gold standard RCT and has been third-party validated to meet WWC and ESSA standards. This allows us to analyze every new cohort of schools, every year, using any standardized testing data. We continually evaluate and validate ST Math’s impact across various demographics, every year, ensuring our program delivers repeatable, consistent, equitable impact for all students.
Explore the MIND Research Portal
Go deeper into the science and evidence shaping mathematics learning. Explore efficacy studies, peer-reviewed research and publications, and opportunities to collaborate with MIND researchers.
ESSA Tier 2 – Quasi-Experimental Design (QED) studies
(QED) studies, ESSA Tier 2 and WWC supported, are more affordable, faster, and repeatable than RCT’s. MIND specializes and invests in rigorous QED’s, comparing outcomes in schools using ST Math to similar non-user schools.
We annually conduct QED studies on each new cohort of ST Math adopters, covering diverse educational scenarios, including district type, location, assessment, school performance, and demographics. All ST Math users, regardless of usage level, are subjects of these rigorous studies.
For example, our 2022 QED study matched comparison schools on a five-year performance trajectory before ST Math use. It confirmed ST Math’s performance advantage and showed that on average, schools using ST Math recovered COVID-related learning losses by the 2021-22 academic year, compared to longitudinally matched peers.
Verified impact you can trust.
A quasi-experimental WestEd study of ST Math, which was verified by SRI to meet the What Works Clearinghouse (WWC) Tier 2 quality evidence standards was conducted in 2013-16.
Each year we release a national scale report on ST Math impact covering all new schools across the USA:
-
2016/2017 Math Outcomes Analysis
Read analysis -
2018/2019 Math Outcomes Analysis
Read analysis -
2017/2018 Math Outcomes Analysis
Read analysis -
2021/2022 Math Outcomes Analysis
Read analysis
ESSA Tier 3 – Correlational studies
Tier 2 QED’s rely on state assessments, which are made public months after a school year ends and require multi-state samples and large control pools of non-user data, often unavailable in single districts.
How can we perform a timely single district study? ESSA Tier 3 supports correlational studies which can report out, at a single district, the relationship between the amount of ST Math content students complete and their growth on 3rd party summative assessments given three times a year. This “impact rate” covers: one district, this year, on its students, using its chosen assessment, regardless of how they used ST Math.
We have conducted over 100 correlation studies with ST Math on a wide array of assessments to help districts evaluate their investment and understand students’ math outcomes. If you’re interested in a correlational study for your district, please contact us to explore how ST Math can benefit your students.
ESSA Tier 4 – Demonstrated rationale
Even at ESSA’s lowest evidence tier, we maintain a high standard of demonstrated rationale and explanation for how ST Math generates outcomes. Our scientific basis, independently validated certifications, and ongoing evaluation efforts ensure that ST Math remains the market’s most robust, reliable, and effective tool for improving math education in your schools. ST Math includes:
-
Our Scientific Foundation
Learn more -
Implementation to Test Scores (Theory of Change)
Learn More -
Digital Promise’s Learner Variability Product Certification
Learn more -
Research-based Design Product Certification and EdTech Evidence Based Certification
Learn more -
ISTE Seal of Alignment
Learn more
Merely checking whether an edtech product ‘works or not’ fails to capture nuances of different educational scenarios, different amounts of usage, and the range of outcomes for different learners.
Andrew Coulson, Chief Data Scientist, MIND Education
Read: More Than a CheckmarkST Math & your implementation
Program implementations vary but ST Math’s benefits are linear: the more students engage with the program, the better their outcomes.
Students who complete more of their ST Math journey cover more math standards and accelerate learning compared to peers who invest less time. A study by Proving Ground, part of the Center for Education Policy Research at Harvard University, found that just 10 additional minutes of ST Math per week increased test scores.
ST Math’s Education Success Team provides tools and resources to ensure consistent, high usage. In Massachusetts, a proof-of-concept program showed success with 59 districts and 245 schools achieving 80-100% completion of their ST Math journey in a year, demonstrating the program’s effectiveness and scalability.
Explore Our ST Math Programs
Ebook: Demanding more from Edtech evaluations
Resources from vetting, adopting and supporting edtech solutions
Create your own results with ST Math.
Start using our neuroscience-driven puzzles today and watch their confidence soar.