Unlike most other VAWG initatives, the programme aimed to tackle both intimate partner violence (IPV) as well as violence experienced by women and girls in public spaces. The programme sought to do so through targeted interventions with women's self-help groups (SHGs) and work with men and boys. This was a complex evaluation into highly sensitive issues and as such, it needed a highly specialist team of VAWG and monitoring and evaluation (M&E) experts.
Our team developed an evaluation design with two integrated components:
- A randomised control trial (RCT) in 250 slums
- A broader analysis looking at the programme's relevance, cost-effectiveness and sustainability, and the effects at the city and state level.
The data collection strategy was based on a mixed-methods approach. This included a quantitative survey with over 7000 direct and indirect beneficiaries across treatment and control slums, direct observation techniques, participatory focus group discussions (FGDs), key informant interviews (KIIs), and analysis of programme monitoring data. As part of the evaluation, the SDDirect ream developed and implemented robust ethical procedures to ensure careful and appropriate data collection and handling on this highly sensitive topic.
In August 2016, the team finalised the endline report. The quantitative component of this evaluation found little evidence that interventions or combinations of them had positive impacts on the hypothesised outcomes, including reductions in IPV, violence and harassment in public spaces, and the associated norms and attitudes. The qualitative analysis highlights some important perceived benefits of SHG membership including improved social networks and confidence. The Life Skills module may also have been effective in encouraging men and boys to challenge unequal gender roles in the household. However, the qualitative data also highlights persistent harmful norms and attitudes, which continue to drive and sustain VAWG in urban slums in Madhya Pradesh. The report offers important recommendations and practical insights for future VAWG programmes and evaluations, including:
- Robustness of VAWG measures - as part of this evaluation, we assessed the reliability of commonly used VAWG prevalence measures by both comparing consistency over time and direct measures with indirect measures. Both analyses highlighted weaknesses in the standard sensitive measures commonly used in evaluations of VAWG programmes.
- Achieving change in VAWG - programmes with long-term investment, sustained resources and realistic timeframes are likely to be more effective and to have both greater and more sustained impact. Although there is no well-esablished minimum dosage for VAWG prevention interventions, our findings support emerging learning from other SDDirect projects (e.g. What Works to Prevent VAWG) that intensity of delivery is key for achieving results. In particular, this involves ensuring sufficient programme duration (typically 3+ years for social norms programmes) and enough regular hours of delivery, effective mobilisation and maintaining high regular attendance.
- Strengthening local VAWG response - prevention programmes should engage in strengthening VAWG response as a key strategy to ensure sustained social change and adhere to 'do no harm' principles. It is important to ensure that those who are mediating are provided with appropriate training and support to allow for adequate response to reports by mediators and referrals to appropriate bodies.
- Importance of monitoring data to measure implementation fidelity (the extent to which the intervention was delivered as intended) - evaluations can benefit from more rigorous quantitative monitoring data to be included in quantitative analyses.
