Smart Kalasatama - What can we learn about evaluation methods?

TIME

14.00 – 15.30, 3 September

GOAL

To co-ideate different possibilities for evaluating living labs with qualitative data from the social point of view and to find tools for using and communicating this type of data in practice.

CHALLENGE

Measurable quantitative metrics are most often utilised to evaluate the success of smart cities, but they can be blind to many impacts of smart interventions on social networks.
Simultaneously, it is possible that smart interventions are designed to perform under certain metrics, which paints an unreal picture of their impact. For this reason, in the final stages of Smart Kalasatama, we have evaluated its social impact through qualitative research methods.
Gathering qualitative data on opinions and attitudes of the residents towards the city services has given valuable insights on how to improve Smart Kalasatama and future projects. Even if this is a common practice in projects with citizen involvement, the information gathered through qualitative research does not often strongly influence on decision-making processes compared to measurable quantitative data. How do we increase the impact of qualitative research on the decision-making process and how do we apply the learnings in practice? The challenge of this workshop is to reflect on the validity and importance of qualitative data to evaluate living labs and ideate on the methods to utilise it during the decision-making.

OBJECTIVE

The participants will search for different context-based processes to evaluate living labs with qualitative research. The objective of using qualitative research is to paint a better picture of the impact living labs have on everyday life of people and how it cannot be disconnected from its planning. Participants will reflect on the importance of using different types of data and the appropriate circumstances when qualitative data is useful for evaluating the success of a living lab.

Furthermore, our second objective is to find ideas on how to present the learnings from qualitative research to decision makers who are not necessarily research-oriented people and show its value. We want qualitative data evaluation to increase its importance in planning of living labs, particularly in those oriented towards improving wellbeing or those funded with public money.

OUTCOMES

TANGIBLE

The participants of the workshop will take away ideas for their own evaluation processes through comparing them to the one in Smart Kalasatama and the experience of other participants. We also aim to create the first iteration for a strategy to communicate qualitative data to non-research-oriented decision-makers, this can be in the form of visual tools or qualitative data processing methods, that can facilitate the understanding of the data gathered during the evaluation process.

INTANGIBLE

Participants will add on their knowledge about evaluating complex projects from a social point of view. They will have better understanding of the importance of qualitative evaluations, as well as the type of results to expect and how to use them. The participants will have seen and discussed examples of different evaluation processes from others and reflected on the different possibilities and tools for using qualitative research in practice for evaluation and later decision-making process in their own living lab context.

BRIEF OUTLINE / METHODOLOGY

Opening words

10min.- Presentation: deep-dive evaluation
20min.- Fish-bowl discussion:
– On experiences by participants on evaluation processes
– What information cannot be represented with numeric metrics?
– What are the advantages of doing qualitative research?
– When is qualitative research appropriate and when is it not?
– What is the relationship between qualitative data and the goals of your project?
These discussions will be facilitated with a pre-drawn mind map of qualitative data, where participants can take notes on the discussion and add ideas.
10min.- Drawing of process map of useful tools in different stages of the project.
The process map will be facilitated with a generic living lab process drawing with different stages of a project, where participants can write new ideas for when and how to use qualitative evaluation
5min.- Break
10min.- Facilitated discussion using the process map about previous experiences of the participants in each stage of the research. How did you use the qualitative research in that stage? How did you process it?
20min.- Ideation on innovative ways to process qualitative research data: This will be a facilitated creative process
Conclusion

AUDIENCE

This workshop is aimed for people who are part of living labs oriented towards resident wellbeing, smart cities, sharing economy, collaborative practices, etc. The workshop will be interesting for people looking for new ways of evaluating projects without excluding people who want to get familiar with evaluation processes to apply them in their own context.