Introducing new technology in international development is hard. And all too often, the key details of what actually happened in a project are hidden — especially when the project doesn’t quite go as planned. As part of the MakeSense project team, we are practicing transparency by sharing all the twists, turns and lessons of our work. We hope it is useful for others working with sensors and other technology, and inspires greater transparency overall in development practice.

Please have a look at GroundTruth’s complete narrative history of the MakeSense pilot here on Medium, or download a PDF of the full report here. 

The MakeSense project was supported by Feedback Labs and the project team included GroundTruth, Internews, InfoAmazonia, Frontline SMS/ SIMLab, and Development Seed. MakeSense was meant to test feedback loops from “citizen-led sensor monitoring of environmental factors” in the Brazilian Amazon, providing structured, accurate and reliable data to compare against government measurements and news stories in the Amazon basin. Over the course of the project, DustDuino air quality sensor devices were manufactured and sent to Brazil. However, the team made several detours from the initial plan, and ultimately we were not able to fulfill our ambitious goals. We did succeed in drawing some important learnings from the work.

Lessons Learned:

Technical Challenges

• Technical Difficulties are to be expected

Setting up a new hardware is not like setting up software: when something goes wrong, the entire device may have to go back to the drawing board. Delays are common and costly. This should be expected and understood, and even built into the project design, with adequate developer time to work out bugs in the software as well as hardware. At the same time, software problems also require attention and resources to work out which became an issue for this project as well, which often relied upon volunteer backup technical assistance.

• Simplify Technical Know-how Required for Your Device

The project demonstrated that it is important to aim for the everyday potential user as soon as possible. The prototype, while mass-produced, still required assembly and a slight learning curve for those not familiar with its components, and also needed some systems maintenance in each location. Internews plans for the DustDuino’s next stage to be more “Plug-and-play” — most people don’t have the ability to build or troubleshoot a device themselves.

• Consider Data Systems in Depth

This project suffered from a less well-thought-out data and pipeline system, which required much more investment than initially considered. For instance, the sensor was intended to send signals over either Wi-Fi or GSM, but the required code for the device itself, and the destination of the data shifted throughout the project. Having a working data pipeline and display online consumed a great deal of project budget and ultimately stalled.

• Prioritize Data Quality

The production of reliable data, and scientifically valid data, also needs to be well planned for. This pilot showed how challenging it can be to get enough data, and to correct issues in hardware that may interfere with readings. Without this very strong data, it is nearly impossible to successfully promote the prototype, much less provide journalists and the general public with a tool for accountability.

Implementation

It is important to be intentional about technical vs programmatic allocation, and not underestimate the need for implementation funding. It is often the case that software and hardware development use up the majority of a grant budget, while programmatic and implementation or field-based design “with” processes get short shrift in the inception phase. Decision making about whether to front-load the technology development or to develop quick but rough in order to get prototypes to the field quickly, as referenced in the narrative, should be made intentionally and consciously. Non-technical partners or team members should be aware of the incentives present for technical team members to emphasize hardware/software development over often equally critical local engagement and field testing processes, and ideally have an understanding of the basic technical project requirements and operations. This project suffered from different understandings of this prioritization and timeline.

Funding Paralysis

The anticipation of a need for future funding dominated early conversations, highlighting a typical bind: funding available tends to skew to piloting with no follow-up opportunities for successful pilots. This means that before the pilot even produces its results, organizations must begin to source other funds. So, they must allocate time to business development as well, which can be difficult if not impossible, and face pressure to create marketing materials and other public relations pieces. This can also in some cases (although not with this pilot) lead to very premature claims of success and a lack of transparency. During this project, there was some disagreement among team members about how much to use this pilot fund to support the search for further investment — almost as a proposal development fund — and how much to spend on the actual proof of concept through hardware/software development and field testing.

This is a lesson for donors especially: when looking for innovative and experimental work, include opportunities for scale-up and growth funding or have a plan in mind for supporting your most successful pilots.

Teamwork

A consortium project is never easy. A great deal of time is required simply to bring everyone to the same basic understanding of the project. This time should be adequately budgeted for from the start. Managing such a team is a challenge, and experienced and very highly organized leadership helps the process. FrontlineSMS (which received and managed the funding from Feedback Labs) specifically indicated they did not sufficiently anticipate this extensive requirement. Also, implementing a flat structure to decision making was a huge challenge for this team. Though it was in the collective interest to achieve major goals, like follow-on funding, community engagement, and a working prototype, there were no resources devoted to coordinating the consortium nor any special authority to make decisions, sometimes leading to members operating at cross purposes. Consistent leadership was lacking, while decision-making and operational coordination were very hard given quite divergent expectations for the project and kinds of skills and experience. This is not to say that consortium projects are a poor model or teams should not use a flat structure, but that leading or guiding such a team is a specialty role which should be well considered and resourced.

Part of the challenge in this case was that the lead grantee role in the consortium actually shifted in 2015 from FrontlineSMS to SIMLab, its parent company, when the FrontlineSMS team were spun out with their software at the end of 2014. The consortium members were largely autonomous, without regular meetings and coordination until July 2015, when SIMLab instituted monthly meetings and more consistent use of Basecamp.

Communications

Set up clear communications frameworks in advance, including bug reporting mechanisms as well as correction responsibilities. Delays in reporting bugs with Development Seed and FrontlineSMS APIs contributed significantly to the instability of the sensors in the field. Strong information flow about problems, and speedy remote decision-making, was never really achieved. At the same time, efficiency in such consortia is paramount, so that time isn’t taken from operational matters with coordination meetings — so a balance must be struck. This project eventually successfully incorporated the use of BaseCamp.