Realist Evaluation Approach: Australian Nurse-Family Partnership Program

Answer :

Introduction

Evaluation is a social process implying the need for a participatory approach. There are several different methods or approaches to conduct an evaluation of a policy, program or campaign where every approach aims to present a critical review of the policy and identify the strengths and weaknesses of the policy or the program and suggesting a future plan of action to make the program effective in achieving the intended outcomes. Each of these approaches has their merits and demerits and any program can be evaluated using many different approaches considering the purpose of the study or the research. Focusing on these different approaches to program evaluation, this paper starts with comparing the realist evaluation approach and utilization focused evaluation to understand the similarities and difference between these evaluation methods. It further moves to reconsider the evaluation method applied in assignment 1 and check the possible improvement that can be achieved by applying a realist evaluation approach to the evaluation of Australian Nurse-Family partnership Program (ANFPP). The second part of the assignment focus on understanding the suitability, benefits and demerits of applying participatory action research approach and identify the rationale of selecting it as an evaluation design for the program selected in assignment 1. 

Pawson and Tilley’s Realist Evaluation versus Michael Quinn Patton’s Utilization Focused Evaluation

The Realist evaluation approach and Patton’s Utilization focused evaluation approach are similar in the sense that they both do not focus on explicit participation. Apart from this similarity the approaches are different from each other in several aspects that can be understood after explaining the basic philosophy and procedures followed under each of these evaluation methods (Johnson et al., 2009).

The utilization-focused evaluation is a method of analysis where the aim is to conduct the evaluations based on their necessity and usefulness. Therefore, such an evaluation procedure starts with identification of the intended users of the concerned policy of program (Kohn et al., 2007). The next step in the process is to categorise or organize the participants and allow them to enter into a detailed discussion over the policy, its uses, its formation, implementation, interventions, uses, benefits and applicability. A wide range of discussion among the participants help to reveal the detailed information about the methodology, cost involved, accuracy and possibility of success in achieving the goals and objectives of the concerned policy or the program (Patton, 1997). Therefore, the focus is on providing detailed information to the intended users of the evaluation. 

Here, it is also to be noted that Patton (1997) gave certain principles of evaluation to act as the basis of analysis and reaching the principles that the intended users wish to adopt. These principles seems to be useful at an initial level but process to be a list of wishes of the intended users without any practical foundation directing the evaluation procedure in a systematic manner. Furthermore, the approach is criticised for being traditional in nature interfering with adaptation and innovation. In practical situations, no one reads and use big reports of evaluation. Here it is assumed that to make this approach successful, it is necessary to limit the number of participants to the research and allow only intended users to be the identified participants to the study. The assumption that every participant is able to simply learn through exposure to the evaluation process does not prove to be applicable in practical life. Moreover, the powerful participants to such a group like the government officials; NHS hospital owners and other influential parties may influence the decision-making procedures, evaluations and implementations. This will give evaluation results as per the wish list and desires of the powerful parties and not of the intended users or the target community that should be benefited under the said policy or program (Gregory, 2000). 

Comparing these limitations of the utilization-focused evaluation approach with the realistic evaluation approach, it can be said that this method is based on some clear methodological guidelines. It does not limit the participation allowing a wider scope of the research or the evaluation of a policy or program (Pawson and Tilley, 1997). The aim of realistic approach is to explain the way outcomes can be achieved and the focus is on the process and the context where the change is expected to take place. In simple terms, the aim of realistic evaluation is to explain the procedure involved in particular contexts between the intervention introduction and the results of such an intervention (Issel, 2009). Here, it is assumed that the mechanisms introduced by interventions do not work in isolation but interact with each other making it necessary to discover the potential cause of the change brought by these mechanisms. 

With this aim, the realistic approach penetrate beneath surface events and understand the ways to remove issues and problems associated with the mechanisms and identify the appropriate alternate mechanisms to achieve the objectives of the intervention (Porter and O’Halloran, 2011). In this manner, realistic approach is more justified and detailed approach to policy evaluation. The focus is on identifying the tendencies of outcomes in different situations while working with different participants or communities. Such an approach helps in identifying combinations of effective casual mechanisms that can prove to be most supportive for achieving the targets under health-promotion interventions (Fetterman and Wandersman, 2007). 

However, realistic evaluation approach is often criticised for remaining limited in terms of knowledge as it is based on the available literature and the researcher’s understanding and knowledge about the policy or the program (Golding et al., 2016). Here, the conclusions and interventions come directly from the researcher without giving importance to the experience of subjects of inquiry. The researcher does not have any experience as held by the subjects of inquiry thereby affecting the validity of the results of the evaluation procedure (Kneipp et al., 2013). Here, it is important to understand that realistic approach to evaluation may give limited results that may not be actual representation of the perception and expectations of the actual users of the policy or the program. Such a situation will affect the quality and result-orienting capacity of the plan of action, improvements and mechanisms identified under such a study (Hurworth and Harvey, 2012). 

Therefore, Patton’s evaluation method and realistic model of policy evaluation hold their merits and demerits where realistic model is considered as a source of in-depth study in comparison to utilization-focused evaluation. The researcher should select the appropriate approach as per the objectives of the evaluation and the nature of people or communities being affected by the results of the study. 

Applying Realist Approach to Evaluation of Australian Nurse-Family partnership Program (ANFPP)

The original method of program logic applied to evaluate the Australian Nurse-Family Partnership Program is theory driven with a focus on implementation of the program and associated causal connections. The approach is criticised for downplaying the range of several other causal influences and contexts while remaining focused on explaining the details of the way program work, the target audience, circumstances of program work, implementation mechanism and the reason of perceived success of such mechanisms (Brad et al., 2010). This can be explained through an everyday example of a situation of headache where the input is to get pills, output is to take pills and outcome is to feel better. So, the program is explained as a causal connection between input, output and the outcomes. 

However, applying the realist approach to evaluation to ANFPP will allow a focus on the complexities of the program and explain the reasons of not achieving the outcomes as well as the possible ways that can be implemented in future to achieve such outcomes. It will allow an in-depth study in to the reasons of the program not working or if working then what are the strengths of such implementation (Brad, 2013; Davies & Dart, 2005). It will also allow the identification of the contexts in which ANFPP will work and where it will prove to be unsuccessful or inapplicable depending upon the target set of audience, nature of communities, applicability of defined mechanisms and frameworks required to achieve the targets. A widened focus on nature of mechanism and not limiting it on observed cause-outcomes along with allowing a detailed study by adding participants from different areas like families affects, nurses involved, local government organizations and hospitals will allow a cross-sectional study. Such a study will improve the results of the evaluation by considering the experience and perception of different audience and parties being influenced by ANFPP thereby giving efficient mechanisms to improve the results and make the target achievement easy and beneficial for all the concerned communities.  

Applying Participatory Action Research approach to Evaluation of Australian Nurse-Family partnership Program (ANFPP)

Selecting one of the approaches to evaluation of ANFPP allows a detailed evaluation of the program through participatory action research approach (PAR). It is a method of action research where the focus is on collecting the data in a systematic manner and analysing it for the purpose of “taking action and making change through generation of practical knowledge” (Gillis & Jackson, 2002, p. 264). Here, the ultimate goal is to bring in some social change through a specific action defined under the evaluation. The approach proves to be useful specifically in the field of healthcare policy evaluation as it is based on the philosophical underpinning that there are multiple realities existing that can be even shared and therefore argues that it is impossible to find objectivity on the field of social studies (Isabel et al., 2009). 

This particular approach is suitable to the selected program as it will provide the right to nurses as well as families of patients to analyse and determine the appropriateness of the mechanisms suggested under ANFPP and participate meaningfully while analysing their own solutions. It allows high degree of power and control to the stakeholders to the program leading to a sustainable development by reshaping the knowledge of participants as well as the researcher about the program under evaluation (Miller and Lennie, 2005). 

The method of evaluation is changed from the originally used program logic method, as there is a need to conduct an in-depth study and understand the reasons of success and failures of ANFPP in several forms. It is necessary to move beyond the simple explanations about the program and identify the areas that need improvement, interventions and mechanisms that need to be changed and identify those that should be introduced as per the preference of the target community of Aboriginals and Torres Strait Islander in Australia. 

However, it is to be note that there may arise some political and ethical issues while using the approach as participants may be influenced to respond in predictable manner due to fear of loss of power or control over the resources. While implementing this approach to evaluation, it is necessary that the researcher take care of maintaining the confidentiality and anonymity of the participants ensuring ethical conduct of the research (Suzanne et al., 2014 and Wilson et al., 2018). Such a careful approach will ensure that the participants are protected against any harm or influential responses by the powerful groups acting as participants to the study (Maher, 2001). 

The benefit of this evaluation approach will come in the form of identifying the reasons of high mortality rate and communicable diseases in the target community. The in-depth identification of reasons of the low engagement of aboriginals with work (Freeman, 2016), education and issues of poverty and housing will help in defining the interventions or mechanisms that are acceptable by the people in these communities (Davidov et al., 2014). Despite of the proper implementation of ANFPP, the mothers are not able to get the required diet and it is necessary to involve such mothers, their children and other family members in the evaluation study to uncover the reasons of such issues and possible course of action that can help them improve the situation. The participatory action research will result in identifying the interlinked and complex issues of high poverty, housing issues, domestic violence and substance abuse (Rollans et al., 2016). As it is considered a dynamic educative approach to social investigation, it will allow a socio-political action in the community through its benefits of being a cyclic process of finding facts and defining actions based on the facts leading to reflection and further inquiry to improve the situation and people’s life (Jackson and Kolla, 2012). 

Despite of having several merits, PAR comes with some key challenges to the researcher and the participants. The first demerit lies with including the members of the target community as participants to the study who may not be capable of maintaining their commitment and understanding to the evaluation process. There may be perspectives that are too divergent and unacceptable as per the long-held values, beliefs and abilities of target communities making it difficult to reach consensus for identification of the appropriate plan of action or interventions. There are issues of egalitarian relationships leading to misunderstanding related with perception of participants and the program to be addressed along with the conflicting ideas about the analysis and interpretations made under the process of evaluation. 

Conclusion

It can be concluded that various approaches to evaluation hold participatory and action-oriented elements and mechanisms. In order to identify the best possible approach it is important to understand the nature and target of the evaluation report that should be integrated with the program to be addressed. There is a need to implement the evaluation approaches in a manner that such evaluation reports can speak truth to power in a way that is listened to, comprehended, valued and acted on. It is important that balancing the accountability with learning approaches an integration of evaluation and program to ensure meeting the demands and expectations of various stakeholders to a program.