The Humanitarian Information Dashboard 7 months later

[This blog post is an abstract of the “Next Steps Report” produced by Internews and Aptivate in August 2015]

Seven months later, I am really excited to talk about what we have done with the Humanitarian Information Dashboard (HID). The system was designed to track and analyze community views and humanitarian related data during emergencies. The platform aggregates feedback from affected communities to give humanitarian responders unprecedented insight into community information needs and make informed decisions that save lives.

The HID supports Internews’ belief that information is an essential part of humanitarian relief and helps Internews respond to issues of information overload during a time of crisis. The main goal of the dashboard is to create:

• greater trust and security around information;
• an improved efficiency of response efforts during emergencies;
• identified knowledge gaps within communities affected by a humanitarian disaster;
• more targeted information delivered to communities affected by a humanitarian disaster.

Review of developments to date

A great deal of useful, valuable and complex work has been completed during the first phase of the HID development, done between February and August 2015 with Aptivate, an established NGO that believes in the power of knowledge and communication to alleviate poverty, suffering and conflict, and in the right of every individual to inform and be informed. Aptivate is dedicated to developing ICT services that facilitate communication for unconnected communities, empowering ordinary people across the developing world to improve their lives and has been selected by Internews to work on the this project. Associated with the developments achieved to date, there has been some important learning that has strengthened Internews and Aptivate with regards to their understanding of what a successful HID comprises and what opportunities and challenges exist for future development.

What has been delivered

• A working prototype of the HID for use by Internews in Liberia, designed for low bandwidth and with a responsive design interface (key requirements of the HID user environment).
• Supporting documentation and deployment support in Liberia.
• Architectural design and technical implementation of the REST API – a significant foundation for future developments of the HID based on robust scale and connectivity infrastructure.
• Inception report and a more refined scope for the HID based on iterative discovery, development and testing with users.
• A comprehensive evaluation of existing practice, data streams and software tools used in gathering information in global humanitarian response efforts.
• Refined vision and implementation scenario modelling from field work, personas and user stories, user consultation and participatory discovery processes.
• Huge insights into field office reality and field office working within which any HID solution needs to operate.
• A more detailed understanding and documentation of future development options in the form of a features and requirements backlog and associated future planning.
• Field user testing and a more defined understanding of digital literacy, functions and tasks of the target users.
• Enhanced capacity of Internews staff based on training and support on agile product owner role and user testing.
• Scope of the project refined to focus on Liberia office needs – that has been an important development – this is a big delivery in itself. Embedding in the office has been crucial.
• An effective project management and implementation team working environment between Internews and Aptivate staff. With improved knowledge and decision making around the project as a result.

What is unique

• The REST API is a unique contribution to this project and the wider information for development community.
• The demonstration prototype of the HID is a unique and promising application. It is built using Open Source technology that has the potential for adoption and adaptation by others working in this area.
• HID end user functionality that allow tasks and actions to be completed that it was previously not possible to do.
• User testing and field usage of the HID prototype that has allowed previously abstract ideas to be field tested and trailed in a field deployment scenario.
• Strengthening and enhanced dialogue around “communication and information as aid”
• An understanding of how the HID can facilitate reciprocal and active engagement between humanitarian response efforts and affected communities, and consequently encourage responders to hear and respond to the most urgent questions from the community and that the community responds and provides feedback to relief agencies to ensure its efforts are relevant and actionable.

HID REST API Development Significant effort was used in developing the HID REST API. A REST API is a software architecture style for building scalable web services. REST gives a coordinated set of constraints to the design of components in a distributed hypermedia system that can lead to a higher performing and more maintainable architecture. This was identified as important during the inception and discovery phase of the project. The REST API has the following features; ●      It provides a common architectural framework for future technical development. ●      It enables potential future changes to information inputs or feeds, their analysis and visualisation, without the need for re-engineering what has been done before. ●      It provides a scalable architecture that will support information feeds and updates in real-time with the potential for significant data volume handling in response to incoming messages and information (Event Driven or Real time). ●      It provides a framework to support integration of different web technologies, meaning that the system is programming language and technology agnostic. External modules could be linked or development more readily as a result. ●      It provides a separation of the UI from the back-end, meaning the HID could have multiple interfaces, using different appearances, even created using different technologies.   This also allows for replacing component parts of the system (e.g. changing database choice) when a need is identified without needing to replace what has been done before. ●      It provides a caching framework so that the system could potentially be refined in future to support true ‘offline’ use, with re-synchronisation when re-connected to the Internet. ●      It enables future extension so that other systems can integrate with the HID, either by sending out or passing in data, using a standard format. ●      It enables scaling to support potential analysis of very large quantities of data from social media and other sources. ●      It supports improvement measures (such as load balancing and queueing) to handle and maintain performance under high loads of incoming data or data processing. ●      It can help maintain performance when storing very large amounts of data.

What has been learned

• A much clearer understanding of the role and field use reality of the HID has resulted from developments and learning to date – in terms of user requirements, user digital literacy, field teams tasks and operational reality and technical requirements. However, more still needs to be learned from a wider range of field deployments of prototype HIDs.
• A clearer vision about what the priority implementation environment, context and deployment model is required to prioritize the next stages of development.
• A good understanding of HID potential user and requirements (personas) has been completed. These were developed as generic personas but they have been most closely tested and validated in the Liberia country office context. Additional effort will be required to extend their testing and validation to other country offices during any future phases of development.
• The initial plan of reviewing existing tools and taking those and integrating them into the HID was misplaced in what it could achieve. Aptivate was over ambitious in what they could achieve and Internews had too high expectations about what this would result in.
• Moving from piecing existing tools together to custom development meant the project ballooned in complexity.
• Investment in robust scale and connectivity infrastructure takes time and effort and, as a result, has significantly slowed technical development during this first phase of work.
• The design decisions around scale and connectivity infrastructure may be useful for continued development but they will continue to make rapid end user functionality more difficult and expensive. A decision should therefore be made about continuing development within the initial design decisions framework.
  • Important scale and connectivity infrastructure back end, architecture and data modelling has been completed. This will allow future work on processing of data / links (which can be a highly complex undertaking in terms of processing data and volume data storage issues) to be addressed, if that is still a priority.
• Real time visualization within the HID will be both a challenge and significant opportunity in the next stage of development.
• Team structure and working has been refined and improved. An effective agile development relationship has been established between Aptivate and Internews. This bodes well for future developments.

Delivery of value discussion

The prototype HID that has been developed, has been tested and deployed in a single limited environment (for the Liberia office). This small country office had a 3-person team dedicated to collecting (and helping journalists respond to) rumors from communities affected by the Ebola crisis, linking journalists with key decision makers, government bodies and NGOs, conducting SMS polls and essentially acting as knowledge intermediaries to local journalists. This information is sourced from a mix of data streams, in particular; RapidPro, GeoPoll, XLXS files.

The HID in this first phase was envisioned to streamline this information gathering process as it operated in Liberia. It allows users to tag and categorize content from these different sources (RapidPro, GeoPoll, XLXS) through uploading data via spreadsheet. This information can now be handled in a single location rather than collected and processed manually and coordinated via emailing spreadsheets within the office.

Processing data has been supported through batch tagging and deletion capabilities, as well as administrator-configurable bar charts showing data by category against selected dates.

In terms of delivery of value to Internews in Liberia, there has been real value delivered in the following areas:

1. Operational efficiency of Internews Liberia field team

• Reduced manual effort or day to day tasks around information flows and analysis
  • Visualizations (bar charts) can be created for sharing on the HID showing categorization of information needs (Questions) from the community. These are generated automatically for tagged data and can be set to each display the same information for different time periods enabling comparison. More can be done to increase the ease by which different time periods can be compared – such as being able to visualize two different time periods on one chart.
• Improved team coordination
  • Team members can now coordinate tagging and data cleaning for both Questions and Rumors through a single location, removing the need to pass spreadsheets by email for coordination. Data can be added to the HID through spreadsheet upload (specific format for Questions and specific format for Rumors) or by manual entry.
• Improved usability and ease of system use contributing to increased productivity
  • The HID enables batch processing – deletion of items and categorization of Question types – making the work of initial data handling easier.
  • Data can easily be sorted by imported date, creation date, or by category where applicable.
  • The most recent additions to the system can easily be viewed. The user can choose to see the full set of information held about an item on a per item basis.

2. Increased effectiveness and improved impact of field interventions

• Increased analysis capability by field teams
  • Questions and Rumors can now have tags applied on a per item basis. This will form the foundation of some powerful data exploration and analysis capabilities once search and filtering have been added to the system. In the interim this data can still be captured for future use.
• Improved data or information validity through increased data handling, machine processing and improved accessibility to information
  • Categorization capabilities, such as the implemented example of question categorization, enable consistent application of terms to items.
• Improved information visibility within teams
  • Shared access and storage of information within the HID has moved information out of individually owned spreadsheets (with associated access, security and backup risks) into a single place, visible and accessible to all HID users.
  • Information handling visibility and progress tracking e.g. staff can observe progress and quality of information handling and tagging.
• Improved opportunities for communication of complex information issues tailored to core field partners and user groups
  • Partners (invited users) can now visit the dashboard to see current information as provided by the team through the use of configured tables, bar charts and textual updates.
• Improved information reliability and associated trust
  • Data handling process is now shared by users of the HID (in the country office), providing a central canonical storage location for information, increasing data security and hence reliability.
  • More open and shared data handling processes build confidence and trust in data and processes among HID core and invited users.

In addition, there has been considerable value delivered in terms of organizational learning within Internews and Aptivate, that can help guide future HID development.

Regardless of next steps, some core value and important developments have been made;

• Core architecture has been designed and developed.
• Prototype delivered for use in the field in Liberia.
• Experience of all involved available to guide future development.
• A much clearer set of functions and features and implementation options for Internews to consider to guide further development of the HID. These include a much more nuanced set of effort and complexity costing estimates.
• Learning, ideas and data to guide the next stages of HID development and deployment.

Now, we are happy to share the code developed so far on GitHub and we invite all interested organizations, developers and agencies to download it, contribute to it or let us know what do they think.

The next phase of development will start once Internews has secured more funding for it, so stay tuned!