4. Conclusion from the NGOs’ perspective: what are good evaluations made of – what does the situation look like?

Why should youth organisations consider the impact of their activities? The question could also be reversed. Is it possible to manage and develop the activities of organisations without evaluating their impact? Evaluation is a crucial part of strategic management and deciding how we know that we have successfully achieved our goals.

In the youth sector, the lack of resources is a constant challenge that is not going anywhere. Many actors are competing over declining funding. Impact is not just about achieving more with fewer resources, but also about how to justify funding for your activities. Youth work is not a big sector in society overall, but its impact is substantially bigger than its size. We should also be able to highlight this side of the issue when we talk about funding. By evaluating our activities, we can also determine whether we are doing the right things and whether our resources are allocated correctly. In a nutshell, we could say that the impact of youth work is the extent to which an activity improves the life of young people and thereby society as a whole. 

To put it simply, we could argue that impact is about achieving goals; impact is, after all, always constructed in relation to goals. For this reason, goal-setting is the very first step in building evaluation. Defining the goals and the related measures is a prerequisite to evaluation. Defining the target group is also essential in order to know the group whose changes we are examining. Evaluation in youth work is often based on feedback from young people, not so much a common, shared goal. Could we achieve more if we set out to conduct evaluations by first establishing common goals together?

This measuring process is not always straightforward in youth work. How can we measure the improvement of young people’s well-being, the growth of inclusion or the reduction of loneliness? And even if evaluation were possible, how do we measure the input of youth work compared to the impact of other actors, such as the school system or health care? Not an easy task, but this is precisely the reason why we need impact evaluation in youth work. It ultimately comes down to how we highlight the changes resulting from our work in the young people and in society as a whole.

How, then, do we know that change has occurred? Changes can be assessed using various metrics and questionnaires, by examining not only quantity, but also quality, by evaluating not only subjectively, but also objectively. We must not forget about financial assessment either, or the relationship between the spent funds and the outcome. In part, impact evaluation always has to do with the language that is used. It is easy to state the number of participants in each activity, but it is harder to verbalise the significance of the activities to the participants. Similarly, youth work gets easily dismissed in people’s minds as simply playing pool and drinking coffee, although it is essentially something very different. Instead of methods, we should be able to describe the impact. 

The third sector has come to realise the significance of impact evaluation in the last few years. They can appreciate the need for this work, but are not yet allocating sufficient resources to it. The demands of funding bodies naturally play their part in prompting organisations to develop the evaluation of their activities. The NGO sector is, in this respect, polarised. Whereas some organisations evaluate their activities systematically, others hardly do so at all. There are also major differences in evaluation competence. Support and education for developing evaluation are already available, but organisations may not necessarily have the resources to accept this support. This actually causes a kind of vicious circle where developing the activities and obtaining more resources would require impact evaluation and making the results of the work visible, but at the same time, declining resources lead to cuts in development or evaluation is not seen as an essential element when developing the activities. 

This project has been one way of trying to improve the understanding of the state of impact evaluation in youth organisations. The observations made during the project support the previous view that evaluation is seen as important and organisations want to develop it, but they do not have sufficient resources for development. This is precisely the reason why we need the support of national actors for this work. This publication contains some examples of evaluation, but we should clearly develop and continue the sharing of good practices. We need support for developing evaluation that is aimed specifically at youth organisations and even sparring with individual organisations to demonstrate impact. As stated in the beginning of this chapter, sometimes it all comes down to simple things. Once the goals are clear, measuring them will be easier. Once we have our metrics and figures, we need to “translate” them into the language of impact. Here, it helps if we answer the question “how does our work change young people and society?”

One of the main challenges of evaluating the work done with young people is how to measure something that did not happen. How do we measure the fact that someone did not become socially excluded or that someone believed in their chances more than they did before? And what was the role and significance of youth work in all this? To what extent can the changes that occurred in young people be attributed to youth work, and to what extent are they the result of, say, education and health care, as mentioned above? When speaking about the impact of youth organisations and youth work in general, we are often dealing with complex, multifaceted phenomena. In these situations, it is up to us to define our role in the positive development paths we try to create. Consequently, the importance of making impact visible is highlighted in evaluation.

The impact evaluation of youth organisations is naturally in the interest of the organisations themselves, but it is also important for the parties steering and funding the activities. Evaluation is therefore a common interest. Evaluating the activities of the organisations is significant for the entire NGO sector, but also for the ministries and municipalities funding the activities and for society as a whole. It is socially important to direct funding at the activities through which it will help to achieve the most desirable results. That is why the support for organisations in this work should be developed, and this development should be done together with the funding providers and partners of the activities. Organisations need effective tools and training on how to use them, but also interorganisational dialogue on what should be measured and how. And this dialogue cannot be separated from the discussion on values. What is the significance of the civil society on citizens’ well-being, what is the value of communities, for example, and how do we value, say, the relationship between productivity and inclusion?

Against this backdrop, it is actually surprising that there is so little research on the impact of youth organisations. This was revealed by the literature review of this report. In order to develop third-sector activities, it is clear that we need research knowledge and objective evaluation to support the work. Since there exists a clear societal interest for developing impact evaluation, you would think that funding for such work could be found.

Impact is a topic that is often talked about in connection with funding. With good reason, too: surely we all want to allocate funding to actions and measures that help us achieve the most changes. However, impact evaluation should not be done only in order to secure funding for our activities for the upcoming years. This can be one reason, and a good reason too, but the starting point for evaluation should primarily be the aim of doing the right things. Without evaluation and monitoring the impact of the activities, we cannot know for sure if there is a demand and a need for the things we are doing, and whether they are leading to the desired outcomes. Without evaluation, we are shooting at random, hoping that at least one of the bullets will hit home. Through evaluation, we can improve our shooting accuracy and save bullets. By evaluating the impact of our activities, we can improve young people's well-being.