This is the ninth in a series of principles based on the textbook Humanitarian Engineering: Creating Technologies that Help People, by Kevin Passino. The series is adapted by the author for Engineering for Change. Download Humanitarian Engineering, 3rd Edition, for free.

Humanitarian engineering projects can suffer many unintended consequences. Things that can hinder success include social failures (conflict within the project team), technology failures (not functioning according to specifications), unfinished projects, environmental and health failures, and adverse social and economic impacts. Assessing outcomes can help mitigate these kinds of consequences and define a project’s success.

Assessment involves determining technology effectiveness and side-effects. Outcome assessment is a key part of a project for a variety of reasons: (i) so success is valid and quantifiable; (ii) to ensure the desired effects are found in practice (e.g., positive effects on health, education, or quality of life); (iii) to ensure that undesirable effects of consequences are not found (e.g., side-effects of pollution, adverse economic impact, or adverse social impact); (iv) to provide a measure of solution quality to assess if there is a need for redesign and improvement of the technology; (v) to provide a base-line for solution quality to build on for future design iterations; and (vi) to provide an on-going quality assessment to ensure long-term success of the project.

Assessment of outcomes should be done in a participatory fashion, that is, with the help of the community. A participatory approach: (i) provides a more accurate assessment of solution quality as the community members are typically the end-users; (ii) provides better continuous monitoring of solution quality as community members are often alongside the technology and see whether it is working properly or not; and (iii) gives the community ownership of the assessment process and hence the right to point out whether it is working or not.

For a participatory approach, there are a number of ways to proceed to get information (these are similar to the initial community assessment approaches): (i) interviews with the users of the technology, (ii) focus groups that may include users and non-users of the technology, and (iii) surveys of the whole community about the technology effectiveness. Another approach, often considered the “gold standard” for outcome assessment is the randomized controlled trial.

Regardless of how you proceed, sometimes the objective is to assess how well the technology is performing due to an intent to “scale-up” (disseminate) the technology to a wider region.