Revealing Transparency and its relation to Community Empowerment
Posted: November 29th, 2011 | Author: mikel | Filed under: events | 1 Comment »Writing this on the bus ride up to Long Island for the Transparency and Accountability Initiative “Bridging Session”. Some of my favorite geeks will be there, pressure cooking over three days with groups doing hard work on transparency in the field of natural resources. This intensive workshop format is my favorite way to break new ground. The OPEN PLAN in Nottingham in 2005 led to my start with OpenStreetMap, and Infoactivism Camp in 2009 in Bangalore led in a way to Map Kibera and GroundTruth Initiative, and continues to be a watershed experience for me. Gunner facilitated in Bangalore, and is the ringleader this week, so I have good hopes this event will open up entirely new directions.
Now, I’m pondering what Transparency and Accountability as a movement is actually about, and what it lacks. Ok, mostly about what it lacks.
No doubt, Open Data and Open Government, freely sharing budget information and aid dollar flows, this all is amazing and difficult work. I’ve had my own triumphs and tragedies trying technological interventions in large organizations, targetted change in how they collaborate and share knowledge. Especially at the UNDP with WaterWiki, to share all sorts of knowledge on water governance across institutional boundaries, and with more thematic generality, and more mixed results, with UNDP workspaces. Yes, great to see the UNDP recently embrace open data (though they have always been particularly transparent on their project financing). In any case, I know this isn’t easy work at all.
However, I see this kind of transparency work, so far, reinforcing traditional power structures, rather than substantially engaging the people and communities that are supposed to be the beneficiaries of all of this cash and projects that open data describes and monitors. The focus is on big aid organizations, and big governments, institutions which by their nature love data, and are equipped to use it. Open Data is in spirit and narrative a close cousin of Open Source, but on examination of practice it operates in a traditional top down way. For instance, Open Data Kenya was a quite exciting development, a developing country with a troubled history of corruption, leading the way to sharing its data holdings. Yet in the nearly half year since its launch, I find a troubling lack of real impactful applications of that data, an active community around that data, or a real vibrancy of new data and discussion on site itself. Was it all hype? I hope not. Now I’m not in Kenya anymore, so maybe I’m missing something, but I just don’t get the sense that the hacking and conversation continues. When we look at what Open Data means for OpenStreetMap, yes, all the data is open to contribution and reuse, but everyone deeply involved agrees, the real strength of OSM is the active community that really really cares about the data.
Within this field, there is confusion especially in the word “participatory”. Participatory technology and participatory development share some vocabulary and rhetoric, both are about true engagement in a process, but in practice, there are significant differences in approach. For instance, participtory development stresses a slow pace, building trust, not pushing forward with interventions until there is true ownership in the process from the community. It’s slow, clumsy and messy. Participatory technology inherits the fast pace of Silicon Valley, pushing outputs online as fast and flexibly as possible. They react in all sorts of unanticipated ways, which we know all too well from Map Kibera. Joining the two methodologies is powerful, as people from widely varying backgrounds and different socioeconomic status can find neutral ground in their love of shiny new gadgets and gaining friends on Facebook. But beyond that spark, to really have technology lead to change takes building relationships, 3 years at least for a full arc rather than 3 months.
And there’s confusion in the term “crowd sourcing”. I try to avoid it as much as possible. Any truly functional “crowd sourcing” project has a definite structure. It will no doubt be a flexible and interesting and open structure, but definitely not simply a mob just waiting for a mobile app to land on their phones that changes the world for them by pressing some buttons. A functional open project is going to have the complexity of any human society, but with a few distinguishing core principles of radical inclusiveness.
The danger in this confusion is that excitement over transparency can actually obscure a lack of true engagement.
It’s really helpful to go back to the origin for all these movements, with is Open Source. In a “true” open source effort, the participants maintain an ongoing, flexible relationship. Organizations and initiatives come and go, but the code and the direct personal relationships are always there, able to proceed when ready. Coding can happen when a big company pays for work, or when a hacker is just super psyched about a new feature. The Bazaar is a complex community. Despite sharing this model, there’s a contrast in the world of technology, transparency and accountability. When it comes to actual work, suddenly the technology and especially the relationships in the community, are predominantly the Cathedral old model. Relationships between funders, technologists, and communities are not very much in the open, they are not that flexible, and are beholden to short term funding and project structures. Technology pilots can reinforce the worst of a “brief case NGO”, where the relationships crucial to proper participatory development can’t form.
For an example, I lay open the ups and downs of a small side branch of Map Kibera, in budget tracking at the very local, community level. With the base map of the Kibera slum, the promise was that citizens could monitor government funded projects locally. We were partnering with a Community Based Organization, who’s mission was information collection, sharing, and action. However, this org had very little information management capacity itself … piles of unprocessed papers and surveys, powerpoint presentations abused into functioning as databases. So we built a better, simple web app to collect very local reports on these projects, and map them. And then we worked intensively with the local organization to use the tool, to structure their monitoring, and use in their outreach. They loved it. It even won an honorable mention in Apps4Africa, and Secretary Clinton mentioned it in a speech. Soon after, the CBO presented the tool at a community forum on the local budgets, and hell broke lose. Some of the reports were out of date … the projects had completed (in some fashion, apparently), so distrust was thrown onto the tool itself. And the CBO never really had the time to train fully on the system, and before you knew it, relationships were breaking down over money and ownership issues. The idea is still sound, but it will take time to build the capacity with the right coalition of organizations in Kibera, a lot of slow messy and necessary community work. Despite that reality, the story could easily be “we won Apps4Africa and transformed Kibera”. Who’s doing the real transparency work for these projects in communities?
There’s a role emerging which straddles this participatory technology / development divide, moves the hype towards reality. Maybe it’s a new kind of NGO, or network, or some other configuration, that works very close in the community, with a long term view, in the grassroots, but also very adept at technology. Map Kibera as a project has metamorphasized as the Map Kibera Trust to fill this role. In East Jerusalem, Grassroots Jerusalem fits closely. They work every day in marginalized Palestinian communities, build relationships not with big NGOs, but small community groups which may be nothing more than a prominent person’s living room. And at the same time, they are building their understanding of technology, of mapping in OpenStreetMap, of using Facebook, YouTube and Ushahidi to get stories out, and helping their grassroots colleagues to get involved … slowly. It’s a dual role, very very much in the field, getting dirty, experiencing first hand every day what these technologies actually mean for communities.
For the technologists and organizations coming together this week, I believe we need to focus on how we can help support more organizations to emerge into this role. Does it mean, say, embedding those experienced in these methodologies with Rising Voices grantees, exploring new configurations of actors like in communities, university students, and government in Tandale mapping, or finding ways for locally communities from across to globe to support each other directly?
I do think the Transparency and Accountability Initiative is acknowledging this. In the report Impact case studies from middle income and developing countries, they find
Our third and largest category consists of technological interventions that are tailored to advance the very specific agendas of particular non-governmental or governmental organisations by amplifying their capabilities and strategies. Cidade Democrática, the Kenyan Budget Tracking Tool, Uchaguzi and Kiirti all fit this pattern. In this category, success depends upon a successful marriage between particular technologies and the capabilities and efficacy of particular organisations that seek to utilise them. We feel that most of the potential for technology to have an impact on accountability will lie in this third category.
and they recommend
Interventions in the third category are more likely to succeed when those who create the technology are embedded in local NGO networks, so that they understand the motivations and strategies of organised users and can tailor their efforts to fit them.
A promising direction for this week!
The Amazing OSM Community, and the Tasking Server, Maps Swaziland
Posted: November 10th, 2011 | Author: mikel | Filed under: mics, openstreetmap, swaziland, tech | Leave a comment »One month ago, I asked what to do with 10 million GPS points, and it turns out one awesome answer is map an entire country.
I wanted to experiment with the new HOT Tasking Server, so I set up a task using the Swaziland GPS tiles, but I didn’t bother to promote the effort at all. One month later, and the task is 100% complete, and about 25% verified. Amazing work by the community, and a quickly proven, easy to use, compelling tool.
Here’s Mbabane, the capital of Swaziland, before the editing.
And here are the GPS traces of Mbabane, today.
And finally, the map today.
New roads have been filled in throughout the country. It would be interesting to calculate the growth in road features over this month.
There are still lots to do. There are gaps in the GPS coverage, and Bing might help. This was only roads, and unnamed, unclassified roads at that. It’s really now up to the small and growing community of mappers in Swaziland to bring the map alive.
I asked some of the incredible top contributors about why they took part in the mapping, and how the tasking server helped out.
The tasking tool is really nice to keep the motivation up and to keep track of what’s already done. You see the progress and the small chunks are mostly manageable.
It is good for my inner self. It is rewarding on both a greater scale (creating a needed map) as well as a smaller scale. Also, I find tracing a very relaxing activity that leaves my analytical programmer’s mind open for podcasting, talking or just recuperating from the day.
It was a chance to contribute to a very underdeveloped part of the map and make a real difference to what was available in that country. This is also a country that would take a long time to reach a critical mass of roads without outside work. Hopefully this will give a base for people when they look at a rendered map, which in turn might encourage other people in/near Swaziland to fill in the gaps.
I like mapping because you get to look at somewhere in more detail than you normally would. I now have a real feeling for the geography of Swaziland.
I’ve been interested in HOT for a while and participate when I can, but it’s not always clear how to contribute. The task manager made that easy. To the larger question of why do this at all: it’s like knitting a sweater for millions of people at once.
I think it’s important to get the tile size right. For the Swaziland tracing, it was perfect. I peeked at another task (tracing buildings in Indonesia) and a single tile was too overwhelming.
stethoscope’s insight on the right tile size was among many great points of feedback. We’re learning lots about how to improve the experience of the tasking server even more, that’s getting captured in issue requests, and Pierre continues to push development.
It was after Haiti that this idea began brewing, with identified need for “Mechanical turk style process for working through and importing individual features from large imports” and “Tools for ongoing coordination and identifying needs, addressing the problem of what to map now?”. And really, it goes back to the search for Jim Gray. At my talk at Microsoft last weekend, I was fortunate enough to meet some of the team who worked on the first mechanical turk process for collaborative imagery interpretation … the inspiration has results!
What to do with 10 million GPS points?
Posted: October 3rd, 2011 | Author: mikel | Filed under: mics, openstreetmap, swaziland, tech | Leave a comment »10876681 to be exact. Recording once per second over 4+ months, eight GPS units were taken over every road in Swaziland (background on this survey). If not for some mishaps with batteries and missing SD cards, etc, the count would likely be double that.
Under the project description, the guidance had a lot of freedom for working with the GPS data: “Develop simple software demo interface, incl. analytical tools and manual for monitoring fuel consumption in household surveys.”. (In other words, do cool stuff). After some experimentation, we decided to go with three demonstrations: animation of the traces, tiles for use in OpenStreetMap data collection, and fuel consumption estimate plugin in JOSM. All the code is up on GitHub.
Visualization has always been an inspirational tool for data collection and holistic comprehension, especially in OpenStreetMap. Tom’s animation of London GPS tracks in 2005 is still inspirational, and more recently, ITO’s visualization of edits following the Haiti quake. I’ve used party render many times to animate traces during a mapping party, like this one in Mumbai, and experimented with different styles, like at Yuri’s Night.
The challenge here was to make a visualization on such a large number of points. Split up the task into processing and loading the GPX files into a PostGIS database, and then producing frames of an animation from the db. Having the database would also allow production of other products, like tiles. This set of scripts processes directories full of GPX; change the paths depending on your layout.
Thought about experimenting with SpatiaLite or ElasticSearch too, but that will wait for another time. libcinder would be an amazing platform to try, once geo support is in place.
The animation scripts generate a series of images using Mapnik, time slices from the db, and then the frames are assembled using a series of ImageMagick compositions and effects, and then the frames are combined into a movie using ffmpeg.
The result pretty clearly show the progress of each survey team over the months, with Swaziland as a whole emerging by the end. If there had been more time, would have experimented with more dramatic effects, movement of the camera view, music.
Tiles allow for careful exploration of this large data set. This map has shows all 10 million points rendered together. You can notice that a large number of the GPS traces cover new ground for OSM, so can be used as a tracing source for OSM. Typically in OSM, GPS traces are uploaded/downloaded for tracing, but with the high volume of points, it’s just not practical. The number of points over a particular segment of road gives some sense of its “importance” and “classification” (more used, more likely a major road).The tiles can be configured for use in tracing in Potlatch (http://rockburger.com/mics/tiles/!/!/!.png) and in JOSM (http://rockburger.com/mics/tiles/{zoom}/{x}/{y}.png).
Tiles were again generated using a modified Mapnik script. I experimented with the style and colors using TileMill, by converting a sample of the GPS data into a Shapefile (as of last week, TileMill now has PostGIS support), and converting the carto style sheet into a mapnik config file (carto can just be run on the command line). TileMill doesn’t yet have direct tile output, and I just needed a directory of tile files.
For the gas monitoring application, decided to build it into a JOSM plugin, as it will run on all platforms and should already be part of the MICS GPS toolkit. Started from the ElevationProfile plugin which already generates some stats on GPX files. The modifications add a calculation of fuel usage, based on city and highway fuel consumption rates. The compiled plugin can be downloaded, and then installed in the plugins directory of your JOSM profile; in the Plugins panel of JOSM, simply activate the plugin.
There are lots of ways this can be improved in future applications. ElevationProfile only works on a single GPX file, while this should be adjusted to work on groupings. Additional statistics would be interesting: distance per day, distance average by hour of day, distance average by day of week, distance speed minute by minute; and aligning those stats to points on the map would give some metrics for classification of OSM roads. GPX tracks should have some preprocessing, to simplify and filter out bad readings. Other software to experiment with is Viking an open source, cross platform personal GPS data management tool, and TopCube, which builds desktop apps on node.js apps.
In all this was a lot of fun, opened up some fresh programming avenues for me for dealing with large volume geodata, and should inform further development for MICS.
Household Surveys and OpenStreetMap
Posted: September 29th, 2011 | Author: mikel | Filed under: mics, openstreetmap, swaziland | Leave a comment »Time to introduce our work last year in Swaziland with Unicef, integrating OpenStreetMap techniques into nationwide household surveys.
Way back in 2008, I met Bo Robert Pedersen on the beautiful UN Nairobi compound, to drink coffee and talk maps (without any inkling of what would be happening for in Kenya a year later). Andrew Turner and I were on our way to South Africa for FOSS4G, had scrambled to buy some GPS units a couple days before in NYC (yea 17th St Photo!), and were talking to nearly everyone we could find about maps. We were riding the success of Mapufacture’s acquisition, having a great time mapping in beautiful places, and had found Bo by looking at the most active OSM editors in Nairobi.
Bo just happened to be the regional coordinator for MICS4 (or Multi-Indicator Cluster Survey), an international household survey program coordinated by Unicef, and one of the world’s largest statistical data sources on household demographics and living conditions, and women’s and children’s health and threats to their well-being. These surveys employed GPS units in a fairly simple way, for each survey team to record the central location of survey clusters, and Bo saw an opportunity to make much greater use of GPS. These surveys traverse literally every single road in the country, and simply collecting GPS tracks would unleash a treasure of track points for tracing in OpenStreetMap. And that was simply the tip of the iceberg.
Over the next year and a half, the ideas bubbled, and while in the midst of Map Kibera, we were given an opportunity to pilot these ideas in Swaziland. We designed a pilot project to develop a GPS training program for the MICS survey teams, collect and analyse and trace GPS traces, and explore innovative applications of the tools. Primoz Kovacic and I travelled to Swaziland, developed manuals, and did our best to get the crew psyched for mapping. Traditional household surveys and OpenStreetMap share a lot in common in technique and activity, but differ widely in approach. Surveys are highly controlled, top-down, and jobs, while OSM is self motivated, bottom up, and voluntary, and the interaction between the two was fascinating, especially in comparison to the ongoing work of Map Kibera. One of the first key things was a set of manuals for the MICS program, which are shared here CC-by-SA. The GPS Coordinator Manual may particularly be of wider use, at it describes precisely how to configure Garmin GPS units for surveying with OpenStreetMap.
In the next few blog posts, I’ll present the results, explore the technical experiments with the resulting data, speculate on other applications and approaches, and make available to the OSM community this treasure of data for tracing in the map.
Tandale Highlighted by the World Bank
Posted: August 31st, 2011 | Author: mikel | Filed under: tandale | Leave a comment »Good look at the work of ICT group at the Bank, including our project in Tandale.
“In Dar es Salaam, urban planning students from local Ardhi University are geocoding with assistance from community volunteers. The project is also being facilitated by Ground Truth, —the creators of Map Kibera—and receiving support from Twaweza, a local NGO that focuses on access to information and citizen advocacy around education, health care and clean water.”
And a nice post from our support partner, Twaweza.
And a blog post by Josh Goldstein.