Thomas Delahais, Fiona Ottaviani, Annabelle Berthaud & Hélène Clot (2023)
Evaluation and Program Planning, 97, Online first
Abstract. While evaluators and advocates of alternative indicators both reflect on values and the collective construction of judgements, they do not talk much to each other. The ambition of this paper is to bridge this gap between evaluators and proponents of alternative indicators, by highlighting how an approach dedicated to the construction of alternative indicators can support evaluation practice. To do so, we draw on a participatory project conducted in Grenoble (France), that focused on the construction of indicators of sustainable wellbeing (IBEST). The project and its outputs have been used to support three evaluations over the past five years, which were each conducted with the aim of challenging policy objectives and evaluating interventions in terms of their contribution to the wellbeing of local people. Our results show that the work carried out within the framework of this type of project can feed into every stage of an evaluation. It is therefore in the interest of evaluators to integrate these alternative indicator approaches into evaluation. Moreover, this enrichment of evaluation could be useful in other places that have developed or wish to develop wellbeing indicators to support alternative policy making.
Extract: “The “economics of convention” field has coined the term “quantification” to describe the full process of developing indicators (Desrosi`eres and Kott, 2005). Quantification is not only about measuring; it also includes “convening” about (i.e. agreeing collectively), for instance, the object to be measured, how it should be measured, the norms, values and beliefs associated with the measurement, etc. (see Box 1). In this process of convening and measuring, reality is not just reflected, but is constituted in a certain way. This convening process is often passed over in silence in standard approaches to indicator-building and remains opaque. This is an issue in that the “framing” of reality has consequences for the practices of policy stakeholders: indicators are not neutral and can be used as instruments of power and domination by imposing a certain perspective on problems and available solutions (Bruno, Didier, & Vitale, 2014; Espeland and Sauder, 2007). However, to use a metaphor familiar in the evaluation realm, the quantification process can open the black box of indicator-building and shed a light on its “machinery” and its consequences on policy.” (p. 3)