Designing your Evaluation
Last Updated: August 2022
Choosing the right type of evaluation for your program and designing your evaluation requires consideration of many factors. The evaluation design should always reflect the priorities and concerns of stakeholders in addition to the goals of the program. The Centers for Disease Control provides in depth information about evaluation design at: Types of Evaluation.
Program evaluation includes identifying and engaging appropriate stakeholders, developing evaluation questions and associated indicators, determining data sources and data collection methods, and analyzing, interpreting, and translating evaluation findings for program improvement. In some cases, an external evaluator may be helpful in achieving this outcome. An external evaluator can also bring the desired independence from execution of the program. This adds credibility and integrity to the evaluation findings.
In general, the following guidance can be used to help design your evaluation:
Stakeholders
- Who will be involved in executing the overdose response plan?
- Who will be impacted by the overdose response plan?
- Have we considered racial equity and cultural competency in the creation of the plan?
- Who is intended to use the evaluation findings? How will these stakeholders be engaged in the evaluation?
- Who will conduct the evaluation?
- Who will be making decisions based on the evaluation findings of the overdose response plan?
Describe the Overdose Monitoring and Outbreak Response Plan
- Describe the need for the plan
- Describe the intended effect of the plan
- List the activities included in the plan
- Determine resources required by the plan
- Describe the current context of the plan (i.e., political, environmental, implementation status, etc.)
- Create or add the Logic Model created prior to the evaluation
Focus the Evaluation
- What is the purpose of the evaluation – for example, is it intended to gain insight into the needs and assets of the community? To identify barriers and facilitators in the implementation of the plan? To learn how to best describe and measure the activities and impacts of implementing the Overdose Monitoring and Outbreak Response Plan in the community?
- Who will use the evaluation (stakeholders, funding agencies, general public, etc.)?
- What will the evaluation be used for?
- What is an appropriate research design within the limits of available resources and approach to data collection? Remember, the best evaluation design may not be ideal in your program environment. See evaluation approaches described in the prior section.
- What are the protocols and agreements required for the evaluation? It is important for all the participating stakeholders to agree on a protocol for conducting the evaluation and to have the required agreements in place before the evaluation process begins. Depending on the type and design of your evaluation, you may need to establish data use agreements between stakeholders, or informed consent forms for participating individuals, etc.
- What specific questions will be created for individual evaluations?
- Determine which activities in your Overdose Monitoring and Outbreak Response Plan are the most important to evaluate
- Develop associated evaluation questions and indicators
- Identify data sources and data collection methods
- Develop short, intermediate, and long-term outcomes
- Develop a logic model to depict the evaluation process
Gather Credible Evidence
- Good indicators accurately reflect the concepts intended to be measured and are reliable for the intended use.
- Ensure that the data collected is adequate to detect the desired effects, and that there are secure systems in place to store the gathered data.
- A successful program evaluation often involves multiple data sources and utilizes data sources that the stakeholders find reliable.
Data Analysis and Interpretation
- How will th analysis of the data collected and interpretation of findings be approached?
- How are the findings useful in advancing the OMAR plan?
- Are the evaluation findings justified based on the plan standards established at the beginning of the program?
- What actions do the evaluation findings result in?
Ensure Use and Share Lessons Learned
- Prepare stakeholders to act based on the evaluation findings.
- Identify and plan to overcome barriers that may prevent translation of these findings into appropriate action.
- Prepare to communicate the evaluation process, findings and lessons learned to relevant audience in a customized manner.
Toolkit Resources
Get more insights by using our toolkit resources.
Go to Resources
Glossary
Learn the definition of the key words being used.
Go to Glossary
Thanks
Thank you, to all of our contributors.
View our contributors
Using the power of epidemiology to improve
the public’s health.
the public’s health.
Contact Us
Have any questions or recommendations, you can contact us at overdose@cste.org