ASSIGNMENT 3 COMPARING, CONTRASTING AND APPLYING ALTERNATIVE EVALUATION DESIGNS/STRATEGIES
Assess the suitability, benefits and disadvantages of applying one of the following evaluation approaches to the project/program you selected for assignment 1.
Remember to firstly fully explain the approach you have chosen before applying it to your chosen program/project. Include a rational for any variations in the evaluation design or methods used and consider any political or ethical issues raised (700 words)
There are several different method and approaches to evaluate a policy or program associated with social welfare. Every approach comes with its merits and demerits and therefore a details analysis of some of the important approaches is undertaken in this paper. The paper starts with critical analysis of Patton’s utilization-focused evaluation in comparison to Pawson and Tilley’s Realistic Evaluation. Further, the application of the realistic approach is checked in associated with the evaluation of National Dementia Campaign to identify the possible improvements. A detailed analysis of the applicability of participatory approach to evaluation is also presented in relation to the same campaign to understand the importance of explicit participation for efficient evaluation.
Patton’s Utilization-Focused Evaluation versus Pawson and Tilley’s Realistic Evaluation
Under the approach of utilization-focused evaluation, the analysis is based on the notion that it is necessary to conduct the evaluations that are actually useful and used. On the other hand, realistic evaluation is based on generative theory of causation where a programme offers “chances that may be triggered into action through the capacity of the subject to make choices” (Patton, 1997, p. 38).
The method of utilization-focused evaluation starts with identifying the intended users of the evaluation and organizing them to participate in decision-making related to evaluation (Gregory, 2000). Further, they participate in wide-ranging discussion focused on methodological appropriateness accuracy, cost, etc. and collect the required data along with analysing it and forming judgments (Rebien, 1996).
Here it is to be noted that the primary aim under this method is not to promote participatory practice but to provide detailed information to the intended users of the evaluation. To achieve this objective, Patton (1997) provided a set of Principles of Participatory Evaluation to form the basis of evaluation aiming at identifying the principles that the intended users wish to adopt. However, these principles are mere options and have the standing more of a wish list instead of some clear practical guidelines for the process of evaluation.
Another point of criticism lies in the pragmatic perspective of this approach where evaluator is expected to locate strategically enthusiastic and competent people while ensuring that the primary participants are “lower down” in the hierarchy (Rebien, 1996). This shows that those lower down in the hierarchy are strategically located, but it raises a question about strategic influential power of these people.
Another assumption lies in the adamant position of Patton about the composition and size of the task force that should be limited that may affect the results of the evaluation process and may not give generalized results (Patton, 1997). Furthermore, it is assumed that all the identified participants are intended users. Here, it is assumed that everyone has the ability to simply learn through exposure to the process of evaluation. However, in real world scenario, there are questions as to what extent intended uses actually participate in such evaluations or that such task generally remains expert-driven. We may find government officials or other influential parties to act as the decision-makers in relation to policy formulations, evaluations and implementations instead of the community members who relatively remain powerless (Gregory, 2000).
Similar to Patton’s evaluation approach, Realistic evaluation is also not explicitly participatory (Gregory, 2000). However, there are some clear methodological guidelines here to conduct the evaluation and not limiting the participation as in case of Patton’s utilization-focused evaluation (Pawson and Tilley, 1994). Under Patton’s approach, the participation is limited for promoting the goals of decision-makers whereas in realistic evaluation the focus is on knowledge generation.
There is clear separation of roles of the researcher and the subject in realistic evaluation but it restrict the hypotheses generation to background literature and knowledge of the researcher. This makes the approach limited to a particular type of knowledge. Here, the views of Reason (1994) can be cited to explain that in case of research focused on people, the propositional knowledge given as results of the research should be grounded in the “experiential and practical knowledge of the subjects in the inquiry” (1994: 42). However, in case of realistic evaluation, the conclusions are generated by the researcher who actually do not have any experience of being researched and therefore the results of such a study or evaluation fails to reflect the experience of the researcher as well as that of the subject.
On this level, the researcher is subject of his or her own research in the Patton’s evaluation approach while this is not the case in realistic evaluation (Rebien, 1996 and Gregory, 2000). This may be cited as a reason for restricted results of the realistic evaluation where it fails to embrace the range of knowledge held by people except that presented in the available literature.
In summary, both the method lack focus on explicit participation and the literature lack the guidance in proper method to conduct participatory evaluations. However, realistic approach is more justified in terms of generalized results and not limiting the participation as followed in case of Patton’s approach to evaluation.
Applying the realistic evaluation approach to the plan in assignment 1 can surely improve the evaluation of National Dementia Campaign. The approach can help in considering the campaign in an in-depth manner with focusing on region-specific contexts. It will also help in evaluating the mechanisms and outcomes throughout the rollout procedure of the campaign in various regions (Wilson, 2016). Further, the approach allows application of specific research methods in the form of longitudinal pre and post-training interviews coupled with use of stratified sampling to select the participants for the evaluation (Gregory, 2000).
Originally, the evaluation of National Dementia Campaign in Assignment 1 is based on the program logic method where the method is used for understanding the goals and objectives of a service under a program. This method of evaluation is focused only on presentation of the projects’ activities, goals, implementation, evaluation and communication in simple terms. It explains the effectiveness of a program and can be used for proper planning and implementation of the program (Issel, 2009). However, applying a realistic approach has the potential to represent the interest, views and experience of a range of organizations and services like independent care sectors, hospitals covered under NHS, various charities like Alzheimer’s society and the local government organizations covering the area of residential and nursing care. Therefore, applying a realistic approach can give a complete different picture with improving the evaluation by evaluating whether National Dementia Campaign is able to resolving the issue of inconsistencies in care standards across private sector and local authority programs, care standards between dedicated dementia services and other general care services and focusing on improving the synergies between various services and improving cross-sector communication.
Participatory Action Research
PAR is an integrated approach to evaluation involving the participation of members of the concerned community, group or organizations. It is based on a three-part procedure of social investigation, education and action focused on creation of social knowledge among the target group of people. In other words PAR is an educational process focused on analysing the structural reasons of the issues through collective discussion and communication (Baum et al., 2006). It aims to identify corrective action for short as well as long term based on the collaborative evaluation of a program, campaign or problems associated with the area under research (Gregory, 2000).
Reason of change
A change in method of evaluation is suggested to allow the element of participation and representation of those affected or influenced by the campaign in the research or evaluation. The method originally used does not allow explicit participation resulting in reinforcing the health promotion campaigns or strategies. The PAR method will allow overcoming of professional dominance and improve the strategies through rhetoric participation of professional and community perspectives (Bennett, 2004). This will make the evaluation enriched resulting in defining a plan of action to bring improvement in health of people with dementia.
Political and ethical issues
The participatory process has political issues also as any change originating through participation results in change in the power distribution (Pigozzi, 1982). In other words, there may be an influence on participants to respond in predictable ways because of the redistribution of power that can result in some kind of power loss or losing the control over resources. Further PAR approach comes with the ethical issues associated with informed consent of participants, maintaining their anonymity and confidentiality of their personal information. There may also be issues of imbalance of power between the participants and the researcher. The researcher or evaluator needs to ensure that these ethical issues are avoided to protect participants from any harm while safeguarding them from the location of ‘power’ in the group (Bennett, 2004).
The Participatory action research (PAR) is at par in comparison to other approaches as the basis of evaluation is reflection, collection of data, and action working towards a goal of improving health and reducing inequities in health (Bennett, 2004). This is achieved by involving the people to the evaluation who can take actions for bringing improvement in their own health. This makes the approach suitable to evaluation of National Dementia Campaign by acting as a guide to identify the action that should follow after the evaluation. It will act as an iterative cycle in a dynamic manner through proactive involvement of organizations, hospitals, and care houses involved in providing dementia care.
The PAR approach will offer the benefits of paying proper attention on power relationships and blurring the line between researcher and the researched giving effective and offering practical and successful plan of action. It will provide the benefit of opportunity for the dementia patients to voice their opinion and demands in planning and running services while stimulating the choices and alternative methods of treatment (Baum et al., 2006). A joint planning approach will help in developing innovative and engaging ways for the service providers and community members to work in collaboration ensuring self awareness, self confidence and hope to patients (Baum et al., 2006).
A basic limitation of the method that can act as a demerit to apply this approach for evaluation of National Dementia Campaign is that no two groups of communities can be homogenous. There always exist a multitude of interrelated axes between the communities and groups. The differences due to gender, age, wealth, ethnicity and power will give different definitions depending upon the interest of every group and the way in which each of the group, organization or community interpret the intentions of the evaluation (Cornwall and Jewkes, 1995). This may in turn affect the results of the evaluation and suggested course of action resulting in manipulation of the research as per the agenda of the powerful members of a research group (Cornwall and Jewkes, 1995).
It can be concluded that applying a participatory research approach to evaluation will help in an in-depth analysis of National Dementia Campaign and provide efficient results associated with gaps in the achievement of objectives, reasons of such gaps and possible strategies to ensure success of the campaign as a whole. Additionally, different evaluation methods like realistic approach can also prove to be useful in conducting a detailed evaluation while avoiding the demerits of the program logic approach to evaluation.