I am a very attentive reader of Patrick Meier blog, iRevolution. This blog has been in the last 3 months my best source of information, ideas and discussion on crowd-sourcing and crisis mapping. In one of his last post Patrick comment on a paper written by Ankit Sharma from the London School of Economics (LSE) titled “Crowdsourcing Critical Success Factor Model” (PDF). The main discussion there is the ability to define a M&E framework for crowd-sourcing projects and what variables influence the success of those type of projects.
I definitely agree that there must be a M&E system, and this is as necessary as it is to have a clear objective when you start a crowd-sourcing project. In this way you can have a “proper management of the vision and strategy” which “primarily ensures sufficient crowd participation” and you can also be able to analyze the independent and dependent variables that will allow you to better understand the five peripheral factors explained in Ankit’s paper. However there is a factor that needs to be taken into consideration if we speak about a M&E framework valid to evaluate crowd-sourcing projects. I will make the example of what I know, which is coming from my experience as Director of the Situation Room @ SIPA for Ushahidi-Chile.
As also said by Patrick, we also had no clue when we start working on the platform of what we wanted to do. There was no other goal then populating the map (to quote a great member of the team about this: “we had no time to think about what we wanted to do, we were mapping”). We started thinking that our goal was to help people in need of course, but this was a pretty general goal, and it didn’t really tell us how we wanted to do it in practical terms, i.e. what was success for us? I don’t know if the goal was to have enough crowd to participate in the project or just enough crowd to benefit from the project.
Interestingly enough for us the goal of the work become the fact of handling in the platform to Chileans, which meant for us that the goal was to make that platform useful for the future. But this cannot be the final objective of a crowd-sourcing project per se: it can be very good for donors purposes, because, sadly enough, it also provide a way out (needed in the case of Ushahidi, platforms if run by universities in emergencies cases). What I want to say here is that we defined our goal in the process of running the project, because the conditions didn’t allow us to have a precise goal before.
Looking at this, I think that a M&E framework for crowd-sourcing project needs to be adapted to the forms that the crowd-sourcing project takes: I don’t know the other three projects analyzed in Anaki’s article, but in the case of Ushahidi the goal will change over the time. And this will happened in emergencies but also in non-emergencies context. This is because if we start from the consideration that there must be a bi-directional relationship with the crowd, then we have to accept that the crowd will change your project and your goal. And this is the good part of it: the bi-directional relationship implies an impact on both the project and the crowd, hence the project will transform parts of the goals or all of it over the course of its development.
Does this means that we cannot do an evaluation of the crowd-sourcing project and will fall necessarily into the fallacy of the retroactive goal imposition? I don’t think so, but this factor should be taken into consideration in designing a successful crowd-sourcing M&E framework and in deciding what the goals of suck projects will be: the elasticity of the platform/system you use, meaning the ability to redefine itself according to the inputs, affect the definition of the goal and consequently the imposition of a static framework of M&E.
The Unwilling Colonizer 