Why it Matters
Clearly defining “evidence of effectiveness” is a key step in including evidence standards in government contract and grant applications. By determining what counts as evidence, workforce development practitioners can evaluate and score proposals against those standards and set clear expectations for applicants.
In a broad sense, evidence of effectiveness refers to all the evaluations, analyses, and other work showing how well a program or policy worked in achieving its goals. To make procurement and grant-making more evidence-based, workforce agencies will need a specific evidence framework, with corresponding equity considerations, that fits their unique contexts.
Overview
An evidence framework or continuum is a tool to categorize practices, policies, and programs based on the level of evidence that exists to demonstrate their effectiveness. Based on the number, types, and results of evaluations, evidence frameworks often define what counts as high, moderate, or low evidence of effectiveness. These frameworks are not only useful for steering funds toward programs that are effective, but can also help identify programs where more information is needed and where to invest funds in new evaluations.
Evidence frameworks generally contain the following:
A set of tiers or levels to reflect the rigor, quantity, and outcomes of an evaluation. Tiers may be designated numerically or with a series of descriptive titles such as high, medium, low or promising, theory-based, and mixed effects.
A definition for each tier, usually including some quantitative indicators such as the number of evaluations, studies, or peer reviewed reports. May also include timelines such as 1 year of outcomes or 5 years of outcomes, as well as considerations for data disaggregation to understand equity within sub populations.
A designation for new or emerging programs that may not yet have evidence but are important to the work of the agency.
Frameworks may also include supplementary documents such as instructions for tier selection to be used by evaluators and respondents or outcomes requirements that tell applicants what data to collect, track, and report and the specific methodology to use.