Define Evidence of Effectiveness: Getting Started

How to Get Started

  1. Determine if your agency has an existing definition, continuum or framework. If not, examine whether such materials have been issued by your state agency.

  2. Review the examples in this guide and determine which approach best reflects the needs of your organization.

  3. Using one of the examples in the guide (or from other government agencies) as a starting point, draft a set of tiers that reflect your organization's needs. Consider (a) how you will apply your new framework; (b) the degree to which the programs you plan to assess are well established or emerging; (c) the capacity that exists in potential respondents; and (d) the role equity will play in your framework.

  4. Define each tier of the framework. Provide enough descriptive information that a potential grantee, service provider or evaluator can easily understand what is required to meet the tier.

  5. Consider sharing a draft of the framework with trusted partners to gauge understanding and identify potential gaps.

  6. Test the framework against completed interventions, grants or procurements. Consider what information would have been readily available, how the application of the framework would have impacted the process, and what additional information would have been needed.

  7. Finalize your framework, along with any accompanying guidance on how it should be used.

  8. Create opportunities to discuss your framework with potential respondents before it is formally implemented in upcoming procurements. This may include offering presentations to partners, delivering training, or making technical assistance available to respondents pre-submission.

Sample Evidence Frameworks from Federal Agencies

The U.S. Department of Labor’s Clearinghouse for Labor Evaluation and Research (CLEAR), includes descriptive, implementation, and impact studies for workforce development and employment-related programs on a wide variety of topics. CLEAR rates the evidence presented in impact studies as high, moderate, or low depending on how confident they can be that the study outcomes are attributable to the program. Review CLEAR’s rating criteria here.

The U.S. Department of Education’s What Works Clearinghouse (WWC) reviews and summarizes studies of education programs – including postsecondary career and technical education programs – and assigns those programs to strong or moderate evidence tiers