We have had several enquiries regarding how to evaluate academic drug demand reduction programs and expect to have a webinar in January on the topic. Meanwhile, I have some thoughts that might be of use to you.
When conducting any evaluation, the first question that you need to ask yourself is “What am I trying to accomplish?” Having a clear purpose for your program is the first requirement for conducting an evaluation that is worthwhile. Is your program designed to train medical doctors, nurses, psychologists, other counseling staff or someone else? Is it designed to provide for a work force in a specialty treatment setting, in a general health care setting, an education setting? The question of what you are trying to accomplish needs to answer the questions “what for whom”. If your program does not have a clear purpose, then it will be hard to evaluate whether you are meeting your goals.
Once you have answered the question “What am I trying to accomplish with my program?”, then the next question is “How do I know I have achieved it?” Ideally, these two questions are something that you have considered before the program began. What metrics describe successful accomplishment of your program goals? Perhaps you are trying to increase the number of trained psychologists who are providing addiction treatment for example. Your goal would be measured by two factors – the number of psychologists who complete your program and the level of skill that the graduates have. The number of graduates is easy to collect, but how do you assess the skill level of your graduates? A simple fall back can be to look at the curriculum and compare it to other similar programs in terms of content, requirements to complete the program, and faculty qualifications. This is what many accrediting bodies do when assessing a program – they benchmark against a set of standards.
Regular review of curricula materials to make sure that your information is up to date and doesn’t include any misinformation is important as an ongoing evaluation activity regardless of the goal or how you measure the goal of your program. The goal of any education program is to convey knowledge and/or skills and should be to ensure that the knowledge and skills are grounded in the current best evidence in the field of study.
Another way to measure effectiveness of your programs is to survey or interview your graduates’ employers regarding the knowledge and skills graduates have. Asking what are the strengths of your graduates and what are the gaps in their knowledge and skills can help you figure out what changes may need to be made to your programs. Another approach is to ask not about your graduates in particular, but about new employees in general which may get you a more honest response.
Finally, you could assess the percent of your students that meet professional practice criteria if such criteria exist for their profession. What percent pass a credentialing exam, for example. If your country or jurisdiction does not have a credentialing requirement, it might be useful to encourage students to take an international exam, like the International Certified Addiction Professional (ICAP) series offered in many locations across the globe and soon to be offered online globalccc.org.