When you're browsing the EdTech Index and see a product with effectiveness badges, you're looking at something fundamentally different from design certifications. While design badges tell you a product was built to work based on sound educational principles, effectiveness badges tell you whether it has been demonstrated to work in real classrooms with real students.
What "Evidence of Effectiveness" Actually Means
Effectiveness badges represent a critical next step in a product's research journey. After a company has documented its research-based design (ESSA Tier IV), the question becomes: Does this carefully designed tool actually improve student learning when used in schools?
Unlike design validation, effectiveness research measures real outcomes: Do students who use this tool show greater literacy gains? Do teachers who implement this program see improved math scores? Does the intervention close achievement gaps?
Understanding ESSA Tier III: Promising Evidence
ESSA Tier III studies are correlational research designs that examine the relationship between product usage and student outcomes. These studies help answer a crucial question that comes before large-scale experimental research: What does successful implementation actually look like?
Before starting up a more complicated, more resource-intensive quasi-experimental study or randomized controlled trial (RCT), it’s important to first understand and establish the program’s "recipe for success." What level of usage matters? How much professional development do teachers need? What implementation conditions lead to the best outcomes?
Tier III studies typically examine dosage and implementation fidelity, usage patterns, contextual conditions, and, possibly, how students using the tool compare with typical students who don't. While they can't show evidence of causation the way experimental studies can, they provide valuable evidence about what works and under what conditions.
This not only helps districts understand how the program works and what’s needed for successful implementation if they want to adopt the program, but it can also help districts decide whether it's feasible to roll out the product to more schools—and agree to participate in a larger-scale study while it happens.
The Effectiveness-Focused Evidence Providers
Several organizations offer ESSA Tier III certification. While each has its own review process, all require a well-designed correlational study with statistical controls, evidence of statistically significant positive effects on student learning, and publicly accessible documentation.
Digital Promise ESSA Tier 3+ Certification
Digital Promise ESSA Tier 3+ certification recognizes products with rigorous correlational research showing statistically significant positive impacts on student outcomes. This certification requires at least 50 students in the study, that the study be posted on ERIC, published in a peer-reviewed journal, or accepted for presentation at a research conference, and that the study be funded by a philanthropic, government, or research organization. A logic model and annotated bibliography (also required for their Tier 4 certification) accompany the application.
First in Math earned this certification after their study (validated and documented by LXD Research) showed that a school went from the lowest to the highest performing in its district across all math domains, including math fact fluency, procedural fluency, and word problem fluency—outperforming comparison schools by 33% and showing a 25% greater likelihood of proficiency on the state exam.
LeanLab ESSA Tier III - Promising Evidence Badge
LeanLab ESSA Tier III process involves conducting well-designed correlational studies showing statistically significant positive effects on student learning. This certification similarly signals that a company has gathered preliminary evidence of effectiveness in real educational settings.
Quill.org offers free literacy tools that deliver immediate feedback to support student writing. Quill.org earned this certification through its study that showed a correlation between higher completion of Quill activities and greater writing proficiency. The more students used Quill, the greater their writing gains.
EduEvidence Effectiveness Silver Certification
EduEvidence Effectiveness Silver certification (aligned with ESSA Tier III) is an international certification that validates that a product has correlational research demonstrating positive impacts on student learning. All LXD Research effectiveness studies come with a review and validation by EduEvidence, providing an international standard for research quality, in addition to the U.S.-based ESSA Evidence system.
Strongmind earned both the EduEvidence Effectiveness Silver certification through a correlational study of 295 students in grades K-1. The study compared a revised version of the curriculum with its original program using a cross-cohort design within a single school. Fluency data from Amira and DIBELS assessments revealed consistent growth in reading fluency, particularly following the implementation of the enhanced scope and sequence.
Instructure ESSA Badges
Instructure ESSA Badges (Tiers I–IV) are connected to an online platform where teachers can comment on products and districts can assess usage, adding a community-validated layer alongside research standards.
Reading Horizons Elevate earned the Instructure ESSA III badge with their study showing that students who engaged with the program demonstrated substantial gains on the NWEA MAP Reading assessments. The study also showed that classrooms with higher Reading Horizons Elevate course completion rates outperformed their peers, highlighting the effectiveness of the curriculum in improving literacy outcomes.
The Critical Limitation: Correlation Isn't Causation
Correlational studies can show that students who used a tool more frequently or more intensively had better outcomes, but they can't definitively prove that the tool caused those outcomes. Maybe the students who used it more were already more motivated. Maybe teachers who implemented it with fidelity were also stronger instructors in other ways.
This is why the research hierarchy exists. Correlational studies (ESSA Tier III) provide promising evidence, but carefully controlled larger scale quasi-experimental and experimental studies with random assignment (ESSA Tiers I and II) provide stronger causal evidence. That said, correlational research still provides real value—particularly for newer products or those serving specialized populations where experimental research is challenging, or where randomization would be ethically problematic.
It's also worth noting that RCTs and quasi-experimental designs (QEDs) that are conducted at only a single site or with a sample of fewer than 350 students are themselves classified as Tier III (Promising), even though these study designs would otherwise be considered more closely controlled or rigorous than a true correlational study design—so a study's design alone doesn't automatically place it in the top tiers, but the broader methodology including sample size and other factors related to the study design can influence an ESSA Tier rating.
What Makes Correlational Research Valuable
Correlational studies serve several critical purposes:
1. They Reveal Implementation Insights
Correlational studies help answer questions about dosage and implementation patterns that link to outcomes. A correlational study might reveal that students need at least 45 minutes per week of practice to show gains, or that teacher participation in professional development correlates strongly with student outcomes. This information is important for establishing which support structures correlate with success.
2. They Identify the "Active Ingredients"
Not all features of a product may matter equally. Correlational research can help identify which components drive outcomes. Maybe it's the frequency of practice, or the type of feedback students receive, or the alignment with classroom instruction. Understanding these patterns helps refine both the product and implementation guidance.
3. They Provide Early Signals of Impact
Because large-scale RCTs and QEDs can sometimes require years and significant resources to execute, correlational studies provide initial evidence that a product is moving the needle. If well-controlled correlational studies show no relationship between usage and outcomes—or negative relationships—that's important information that should inform whether to proceed with more rigorous research evaluations, or whether it may be worth revisiting the ‘recipe’ and design to see where improvements could be made.
4. They Establish Feasibility
Correlational studies conducted in real schools demonstrate that the educational tool can actually be implemented in authentic educational settings. This is crucial information that informs the design of future experimental studies. Correlational studies that pair qualitative surveys or interviews with real educators who used the programs can be especially useful for school leaders to use as an indicator of what worked about the program and whether it might be a good fit in their context as well.
How to Evaluate Effectiveness Research
When you see effectiveness badges on a product, here are some key questions to ask:
1. Does this evidence apply to my context?
How long ago was the study conducted? Where was the study conducted? Was the study sample similar to your school or context? These factors can help you determine whether the program might impact your students in a similar way.
2. What outcomes were measured?
Look for studies that measure meaningful learning outcomes, not only engagement with the tool or satisfaction. Did the study use standardized assessments? Curriculum-based measures? State tests? The quality and relevance of the outcome measures matter.
3. Who conducted the research?
Studies can be conducted by internal research teams, external third-party researchers, or independent research organizations. Each has advantages. Internal teams have deep product knowledge; external researchers bring objectivity. What matters most is transparency about methods and independent validation of findings.
4. Can I actually replicate the conditions that produced these results?
Understanding how the product was used is crucial. How much professional development did teachers receive? How much time did students spend using the tool? What support was provided? These details help you assess whether you can replicate the conditions in your own context.
Why a Portfolio of Studies Matters
Something important that often gets overlooked is whether a program has a single ‘shiny’ study or several studies showing its effectiveness.
A single Tier III study conducted in one district and grade level shows that a product showed promising results in that specific context. A portfolio of studies across multiple contexts—urban Title I schools, suburban districts, rural settings, different grade bands, different years—provides evidence that results aren't tied to a unique situation. The pattern of replication suggests the product is likely to work across varied contexts, and may be more likely to show results in your own context than a program with only a single study in a context completely different from your own.
This is why you shouldn't necessarily dismiss a product with multiple ESSA Tier III studies in favor of one with a single ESSA Tier II or Tier I study. The higher-rated ESSA Tier I or Tier II study provides stronger causal evidence in one context, which is valuable. But the portfolio of Tier III correlational studies provides evidence about generalizability across contexts, which is also valuable—and may be more relevant to your specific decision. And remember, some studies classified as Tier III may have used experimental or quasi-experimental designs but fell into the Tier III rating due to smaller sample sizes or single-site constraints—meaning the underlying methodology may actually be quite rigorous, which is another reason not to dismiss a Tier III study without looking more closely at the details.
This also highlights why the progression of research on a program is important. The process of systematically building evidence from design validation through more rigorous levels of effectiveness research is critical, with companies using the evaluation research to reflect on their program designs and to plan for more rigorous studies as the program’s evidence base grows.
Moving Forward
Educational programs should build their evidence base progressively: starting with "Is this built to work?" (ESSA Tier IV design validation), then "Do we see a positive connection between program usage and outcomes?" (ESSA Tier III), before moving to more rigorous questions about causation and comparison (ESSA Tiers I and II).
When you see effectiveness badges on a product, you're looking at evidence that goes beyond thoughtful design. You're seeing evidence that the product has been studied in real schools, with real students, and has shown promising relationships with positive outcomes—a significant validation that a product isn't just well-designed, but has shown early evidence of working in practice.
In our next blog in this series, we'll explore ESSA Tier II—the most common top-tier level of evidence—and review what makes these studies rigorous, how to interpret their findings, and the nuances for when you use this level of evidence to make informed decisions.
________________________________________________________________________________
Have questions about evaluating EdTech effectiveness research? Want to understand what validation badges mean for your specific context? Visit lxdresearch.com to learn more about our work helping educators navigate the evidence landscape.
The EdTech Index occasionally publishes guest posts from partners in the education research community. LXD Research is an education research consultancy that helps Edtech companies document evidence and earn validation badges. The views and examples in this post reflect LXD Research's client work and expertise. Explore validation badges and quality indicators mentioned in this article on the EdTech Index at edtechindex.org.
