Step 6: Develop an Evaluation Plan
Review all steps in the process of planning a survey project:
- Step 1: Form a Project Team.
- Step 2: Form an Advisory Group.
- Step 3: Define Your Goals.
- Step 4: Plan a Communications Strategy.
- Step 5: Set the Stage for Conducting the Survey.
- Step 6: Develop an Evaluation Plan.
If at all possible, sponsors should incorporate some kind of evaluation into their CAHPS survey projects. It is important to consider evaluation options at the beginning of your planning for the project because many approaches require advance planning and must be carried out immediately after the survey results are released in order to get meaningful and useful feedback.
Reasons To Evaluate a CAHPS Survey Project
Feedback on Implementation. An evaluation provides important feedback on how well each aspect of the project went. For example:
- Did the survey process go smoothly?
- What kinds of problems (if any) did your vendor have?
- Did the survey results reach the appropriate audiences?
This kind of information will help you plan your next CAHPS project or similar survey research efforts.
Feedback on Effectiveness. An evaluation can provide information on the project's impact on the decisions and actions of your audiences, including consumers, purchasers, quality monitors, and health plans and providers.
How To Evaluate Your Project
There are numerous ways to evaluate a survey project, depending on what you want to know and how you want to use the information. The most common methods are one-on-one interviews, focus groups, and brief followup surveys.
To learn more about evaluation goals and methods, refer to Elements of an Evaluation Plan in AHRQ’s TalkingQuality website.
Useful Tips
Assign Responsibility. Designate specific members of the project team to be responsible for conducting the evaluation and summarizing the feedback.
Ask for Honesty. Urge those who are asked for feedback to be frank. It is the only kind of feedback that will help you.
Paying for an Evaluation
Evaluations can vary widely in cost, depending on the approach you choose. Take the time to research the cost of a few evaluation approaches, figure out which one would best meet your needs and financial constraints, and include that expense in the budget. In many cases, sponsors are unable to evaluate their projects because they fail to factor the cost of an evaluation into the original plan.
When You Can't Afford Much. A limited budget does not have to put a halt to your evaluation plans. Here are some examples of simple and inexpensive methods for eliciting useful information about your project:
- Provide an e-mail address for comments and suggestions from those who use the reports.
- Include an area for feedback at the end of your print or computerized guide that invites those who have used the report to share their comments.
- Establish a toll-free telephone number to answer questions about the report and solicit opinions from consumers and other audiences.
- Several weeks after the survey results are released, conduct focus groups or formal debriefings with all who were involved in project planning and implementation (if you can afford it, it may be helpful to hire a professional facilitator who was not involved in the project to conduct this debriefing).
- Conduct focus groups or "town meetings" of key audience representatives and other stakeholder groups.
