Menu


Apr
12
2017

Our Evaluation News

 

Over the last few weeks we’ve been giving our website a bit of a spring clean. As part of that we’ve added a new page showcasing the work we do for partner organisations including our work on evaluations. This is a growing part of what we do, helping other organisations to understand what impact their work is having and how it can be improved. Although not as high profile as some of our larger research projects, evaluations are one of the quickest ways we can use evidence to improve people’s lives, because they provide rapid feedback and suggestions for improvement direct to services. We therefore thought it was an opportune time to use the newsletter to showcase a number of our recently completed evaluations.

KCC Mental Wellbeing Programme

Kent County Council’s (KCC) Public Health department commissioned a series of projects and services to improve wellbeing across Kent, and asked McPin to carry out a mixed methods evaluation of them.

In order to evaluate their success, we first developed a Theory of Change for each – a method that identifies the change an intervention seeks to achieve, and how. This helped inform the detail of our subsequent evaluation by making the links between programme activities and planned impacts explicit, and allowing these to be tested through a range of reach and outcomes data.

One of the key measures used was the Warwick-Edinburgh Mental Wellbeing Scale (WEMWBS), which demonstrated the positive impact the interventions were having. These findings were enhanced with interview data explaining how wellbeing is improved.

The Mental Wellbeing Programme was an innovative and experimental ‘pilot’ in public mental health commissioning, with the diversity and the flexibility of the individual projects pivotal to the success of the Programme as a whole. Crucially, the robustness of the evaluation has led to a number of the projects being recommissioned by KCC over the long-term, or securing additional funding. You can read the summary report on our website.

Camden VoiceAbility

Between June 2016 and January 2017, we conducted an evaluation of a peer mentoring service run by Camden VoiceAbility, in which ‘mentors’, who have experience of mental health problems, provide short-term, goal-focused support to ‘mentees’, who need greater support with their recovery.

The evaluation involved understanding the experience and impact of peer mentoring from the perspective of mentors and mentees. As the service was co-developed with service users, it was essential that the evaluation was also shaped by members of the service. We used an initial workshop to find out what questions people who were part of the service felt were important to address. We then used interviews and group workshops to allow mentors and mentees to explain their experiences in their own words.

Key outcomes for mentors were furthering their personal recovery, skill development, improved understanding of mental health and wellbeing, and enhanced social relationships and networks. For mentees, the key outcomes concerned achieving practical goals, such as resolving a benefits problem, increased levels of motivation and hope, improved symptoms, improved understanding of mental health, and the building of social connections.

Interestingly, we found that mentors reported more extensive gains than mentees, perhaps because of their greater length of contact with the service and the benefit of being part of a supportive network of other mentors. We recommended that the service should consider ways in which these benefits could also be facilitated for mentees. The project also showed the value of having an independent evaluator. At the time of the evaluation, mentors collected satisfaction and outcome data from their own mentees limiting critical feedback. Having an independent evaluator, who is sensitive to service user needs, can help overcome this problem and uncover areas for service improvement.

Performing Places

We recently completed our evaluation of the Performing Places project. This was an innovative project in which Professor Sally Mackey and her team from the Royal Central School of Speech and Drama worked with people who lived in supported accommodation provided through the St Mungo’s Charity. The work consisted of a series of arts and performance based activities, completed over 15 weeks, which focused on how people may feel in their local environment, and how those feelings may be disrupted to create a new relationship with ‘place’. We used a combination of observational and qualitative research methods to explore how the project operated and its impact. The key outcome was that people attending the performing places project experienced improvements in wellbeing and developed a better relationship with their local environment through this work. By completing performances in their local area, and through ‘gifting’ small pieces of art to their local area, people felt they were able to contribute to their local environment and take a more active role in the community.