A toolkit for diagnosis, intervention design, and evaluation
Methods for the diagnosis of problems and measurement of change have been developed over hundreds of years at the coalface of science, academia, and practice. Although many have tried to reinvent wheels, the majority don’t end up round.
The people involved in systemic change programmes – funders, practitioners, and evaluators – come from a wide range of backgrounds with a wide range of skills. As such, those who advise programmes adopting a systemic approach frequently get requests for shortcuts from those unfamiliar with different aspects of the programme lifecycles who seek to quickly understand what they could use or what others have used to develop, implement, and evaluate interventions and why.
As such, this guide aims to succinctly represent the toolkit of methods which is useful for systemic change programmes. It briefly covers what the method is, where it should be deployed, and some strengths and weaknesses. It also points to some resources for further reading if you want to go into greater depth in the discussion around the tools. While the toolkit is as brief as possible, one key purpose is to highlight that are ways of conducting research relevant to the way we design and measure interventions which many people have dedicated themselves to over centuries. We should be deliberate about how and why we select different methods and use them appropriately. A focus group discussion is not the same as wandering into a village and having a chat with the crowd that forms around you. Generalising conclusions based on quantitative survey of a small, badly-designed sample, is likely to have less currency than few well designed in-depth interviews. There are some objective quality criteria that we need to bring to bear and this is a simple step to helping with this process.
The toolkit does not cover or substitute for guidance on which questions to ask or how to develop potential solutions by presenting the logic and frameworks to assist practitioners across the intervention lifecycle. These aspects of a systemic approach are covered extensively elsewhere in literature and training. This toolkit references many of the frameworks used in MSD programmes to help to couch the tools in terms of their complementarity.