Designing Your Evaluation

Choosing the right type of evaluation for your program and designing your evaluation requires consideration of many factors. The evaluation design should always reflect the priorities and concerns of partners in addition to the goals of the program. The CDC provides in depth information about evaluation design in their Approach to Program Evaluation.
Program evaluation includes identifying and engaging appropriate partners, developing evaluation questions and associated indicators, determining data sources and data collection methods, and analyzing, interpreting, and translating evaluation findings for program improvement. In some cases, an external evaluator may be helpful in achieving this outcome. An external evaluator can also bring the desired independence from execution of the program. This adds credibility and integrity to the evaluation findings.
The following guidance can be used to help design your evaluation.

Partners

To begin defining your evaluation’s partners, ask yourself:
    1. Who will be involved in executing the overdose response plan?
    2. Who will be impacted by the overdose response plan?
    3. Do we know enough about the population that will be impacted?
    4. Who is intended to use the evaluation findings? How will these partners be engaged in the evaluation?
    5. Who will conduct the evaluation?
    6. Who will be making decisions based on the evaluation findings?

Describe the Overdose Monitoring and Response Plan

Ensure the background and rationale for the response plan is clear in preparation for the evaluation:
 1. Describe the need for the plan.
 2. Describe the intended effect of the plan.
 3. List the activities included in the plan.
 4. Determine resources required by the plan.
 5. Describe the current context of the plan. (i.e., political, environmental, implementation status, etc.)
 6. Include the Logic Model created prior to the evaluation.

Focus the Evaluation

  1. What is the purpose of the evaluation? For example, is it intended to gain insight into the needs and assets of the community, identify barriers and facilitators to implementation, assess how activities and impacts of the overdose response plan are described and measured, and/or examine whether individuals or entities with decision-making authority are willing, able, and accountable for implementing recommended changes? 
  2. Who will use the evaluation findings (e.g., partners, funding agencies, general public, policymakers, program leadership), including those with the authority to make or influence programmatic, policy, or resource allocation decisions? 
  3. What will the evaluation be used for? For example, program improvement, accountability, funding decisions, policy or systems change, or advocacy among decision-makers. 
  4. What is an appropriate evaluation design within the limits of current capacity, available resources, and accessible data? Remember, the most rigorous design may not be ideal in your program environment. See evaluation approaches described in the prior section. 
  5. What protocols and agreements are required for the evaluation? All partners should agree on evaluation protocols prior to implementation. Depending on the evaluation type and design, this may include data use agreements, informed consent, or agreements with agencies or organizations that control data, resources, or implementation authority. 
  6. What specific evaluation questions will be developed? This should include questions that assess changes in knowledge, attitudes, decision-making, or actions among those involved in implementing or sustaining overdose response strategies, in addition to community-level outcomes. 
  7. Which activities in the overdose response plan are most important to evaluate (e.g., those that rely on leadership decisions, cross-sector coordination, or policy or system-level action)? 
  8. Develop associated evaluation questions and indicators, ensuring that they capture key information that can be shared with relevant stakeholders and decision-makers. 
  9. Identify data sources and data collection methods that align with the information you are aiming to collect. 
  10. Develop short-, intermediate-, and long-term outcomes that effectively demonstrate the program’s effectiveness over time. 
  11. Develop a logic model to depict the evaluation process, clearly illustrating the roadmap of inputs, activities, and intended outcomes. 

Gather Credible Evidence

1. Good indicators accurately reflect the concepts intended to be measured and are reliable for the intended use.
2. Ensure that the data collected is adequate to detect the desired effects, and that there are secure systems in place to store the gathered data.
3. A successful program evaluation often involves multiple data sources and utilizes data sources that the stakeholders find reliable.

Data Analysis and Interpretation

1. How will the analysis of the data collected and interpretation of findings be approached?
2. How are the findings useful in advancing the overdose response plan?
3. Are the evaluation findings justified based on the plan standards established at the beginning of the program?
4. What actions do the evaluation findings suggest?

Ensure Use and Lessons Learned

1. Seek stakeholder feedback on the potential actions suggested by the evaluation findings
2. Prepare stakeholders to act based on the evaluation findings.
3. Identify and plan to overcome barriers that may prevent translation of these findings into appropriate action.
4. Prepare to communicate the evaluation process, findings and lessons learned to relevant audience in a customized manner.

Evaluation Design: Health Outcomes for All 
Designing an Overdose Response Plan Evaluation to Advance Health for All 

What is Data Equity in Overdose Response?

In October 2025, the CSTE Data Release, Presentation, Access, Suppression, and Sharing (DRPASS) workgroup released the CSTE Data Release Reference Document(1), which provides recommendations for the equitable release of public health data. Per this document, “data equity is a set of principles and practices to guide the collection, analysis, and interpretation of data throughout the data lifecycle to support health equity, the state in which everyone has a fair and just opportunity to obtain their highest standard of health.” Considering data equity within overdose anomaly response can help promote health equity within jurisdictions. When designing overdose response evaluations, consider the following ways to integrate data equity principles: 

    • Carefully de-identifying and anonymizing data prior to release. 
    • Discussing how data suppression aligns or conflicts with equity goals. For example, “To protect individual privacy, data for people within certain groups have been suppressed. This may limit our understanding of the full scope of disparities affecting these people, and we are exploring ways to address this gap while maintaining privacy.” 
    • Verifying that harmful narratives are not being perpetuated in the data interpretation, that person-first language is used, and that the interpretation adheres to standard language when discussing marginalized communities. 
    • Recommending strategies to incorporate diverse perspectives, particularly those from underrepresented or directly affected communities.

Further methods to incorporate data equity into public health efforts can be found in the CSTE Data Release Reference Document. Many organizations may already be tracking progress toward equity without explicitly labeling their efforts as such. While this section of the toolkit does not provide a comprehensive guide to evaluating individual equity measures, it offers a starting point for incorporating equity into overdose response plan evaluations. The goal is to support conversations and actions that address disparities and promote equitable outcomes for people who use drugs. 

Embedding these practices into overdose response evaluations ensures that equity is a guiding principle throughout the data process. 

Why Evaluate Overdose Response through a Health Equity Lens?

According to Dr. David Satcher, former US Surgeon General and former CDC Director(2), “Getting from science to policy and practice can be greatly hampered by stigma.” Stigma has been an established contributor to the inequities encountered by people who use drugs and people with substance use disorder. Stigmatizing attitudes about people who use drugs can create impacts at the individual level, including a person’s ability to access treatment, and at the systematic level, such as a reduced willingness of policymakers to allocate financial resources to the issue(3). 

Planning and implementing an overdose response plan in any community mandates bringing together multiple resources and community partners who play key roles in decision-making. This presents a great opportunity to address overdose health inequities by designing evaluations that intentionally use equity as a value criterion. The Bay Area Regional Health Inequities Initiative (BARHII) developed a conceptual framework illustrating the connection between social inequalities and health and focuses attention on measures that have not characteristically been explored by public health epidemiology.  The BARHII framework and toolkit are designed to equip local health departments with better tools to address health inequities and is a great resource to start or advance your health equity efforts. 

How to Incorporate Health Equity into Overdose Response Evaluation?

NACCHO’s document, “Health Equity in the Response to Drug Overdose(4),” suggests individual-level factors can be positively impacted by placing an emphasis on social and institutional inequities and living conditions. Evaluating health equity knowledge and skills of personnel working in overdose response planning and offering training and avenues to incorporate these skills into their work can be a great step towards achieving equitable outcomes. Incorporating health equity measures in the outcome or impact evaluation of an overdose response plan will shed light on what interventions are needed in your community to eliminate barriers to equal healthcare access. The findings from these evaluations will contribute to an increase in awareness, inform changes required in program implementation and monitoring, and impact organizational policies and practices. 

Some ideas to create evaluation questions to advance equity are listed below

Capacity Building

Evaluate whether agency staff have skills and knowledge to advance equity. Measures could include the number of staff who have attended a training or workshop on health equity, the number of attendees who used skills or knowledge they learned at least once within three months of the training or workshop, etc. 

Organizational Culture and Policy

Examine various practices and policies that affect the organizational workforce, such as the percentage of staff who agree that advancing equity or justice is part of their job, the percentage of employee performance goals that incorporate equity or justice, etc. 

Community Engagement

Identify whether adequate opportunities are provided for meaningful participation of communities experiencing health inequities. Some examples include reducing barriers to community participation, shared decision-making with community members, etc.

Communication and Transparency

Examine communication effectiveness through measures such as the number of times the agency communicates data findings so they are useful for action by all partners, the percentage of reports that incorporate one or more of the social determinants of health in its data analysis, the number and quality of engagement activities with communities of color and low-income communities over the previous year, etc.

Equity in Procurement and Contracting

Evaluate equity through measures such as the percentage of budgets including line items for equity.

Together, these evaluation strategies can reveal systemic barriers, inform equitable program design, and support continuous improvement. 

Where can I learn more? 

Useful resources to guide equity-oriented evaluation efforts: 

Sources: 

    1. Council of State and Territorial Epidemiologists. (2025, October). CSTE Data Release Reference Document: Tabulated, aggregated, and macro data policy development recommendations for public health agencies. Retrieved from https://cstelearndev.wpengine.com/wp-content/uploads/2025/10/CSTE_Data-Release-Reference_FINAL.pdf 
    2. Satcher, D. (2020). My quest for health equity: Notes on learning while leading. Johns Hopkins University Press. 
    3. Yang, L. H., Wong, L. Y., Grivel, M. M., & Hasin, D. S. (2017). Stigma and substance use disorders: an international phenomenon. Current opinion in psychiatry, 30(5), 378–388. https://doi.org/10.1097/YCO.0000000000000351  
    4. National Association of County and City Health Officials. (2025). Health Equity in the Response to Drug Overdose. https://www.naccho.org/programs/community-health/injury-and-violence/overdose/health-equity-drug-overdose-response  

Continue The Toolkit

< Previous: Types of Evaluation

Next: Resources >

Prepare

Respond

Evaluate

Resources