Ecology and SocietyEcology and Society
 E&S Home > Vol. 25, No. 3 > Art. 25
The following is the established format for referencing this article:
Wiget, M., A. Muller, and A. Hilbeck. 2020. Main challenges and key features of indicator-based agroecological assessment frameworks in the context of international cooperation. Ecology and Society 25(3):25.

Main challenges and key features of indicator-based agroecological assessment frameworks in the context of international cooperation

1Institute of Integrative Biology, Department of Environmental System Science, ETH Zurich, 2Department of Socio-Economic Sciences, Research Institute of Organic Agriculture FiBL, 3Institute for Environmental Decisions, Department of Environmental Systems Science, ETH Zurich


Agroecology increasingly gains importance in the discussion about sustainable food systems. To facilitate the transition from conventional farming to agroecological farming, adequate methods and concepts to measure and assess impact and productivity of agroecological farming systems (AFSs) are needed, which consider their multifunctionality and other specific characteristics, here called agroecological sustainability assessment frameworks and tools (ASAFTs). In the past years, many agricultural sustainability assessment tools and frameworks were developed but their suitability and applicability to AFSs was not investigated. To close this knowledge gap, we aimed at identifying, reviewing, and discussing published ASAFTs in the context of international cooperation, providing an overview of the current challenges, needs, and requirements in assessing AFSs at the farm level with the means of indicators. Desktop and scientific database research was conducted to identify and discuss published indicator-based ASAFTs at the farm level. The analysis was based on the following four framework elements that the authors considered to be essential for ASAFTs: (1) the adaptability to local conditions all over the world, (2) the involvement of farmers in the development process, (3) the consideration of the multiple functions of an agroecosystem in the definition and measurement of its productivity, and (4) the accounting for interactions between multiple agroecosystem functions and their measurement. Only a few analyzed assessment frameworks at least partly consider these essential elements and were designed specifically for AFSs. However, our study also showed that these frameworks were (1) restricted in their geographical application scope, (2) quite heterogeneous and barely comparable, and (3) based on productivity indicators that do not fully capture the multiple functions of AFSs. Therefore, we identified the need for the development of appropriate agroecological productivity indicators and common standard or reference frameworks for assessing AFSs, which will be crucial for upscaling agroecology.
Key words: agroecological assessment frameworks; agroecological transition; farmers’ involvement; indicator interaction; multifunctionality


The call for a paradigm shift in agriculture and agricultural policies is getting louder as industrial farming does not offer sufficient support for achieving the Sustainable Development Goals (SDGs) by 2030, i.e. food security, food sovereignty, ecological sustainability, environmental protection, and climate change adaptation and mitigation (De Schutter and Vanloqueren 2011, IPES-Food 2016, da Silva 2018). To meet the SDGs by 2030, fundamental changes in agricultural production and policies and decisive political action are required. This was already outlined in the International Assessment of Agricultural Knowledge, Science and Technology for Development (IAASTD 2009) more than a decade ago (IPES-Food 2016, da Silva 2018). One approach to sustainable agriculture and food systems is agroecology. Agroecology has become a core element in many of the current debates on agriculture transformation, while it was rarely mentioned only a few years ago (Pimbert 2015, Hatt et al. 2016, FAO 2018a). Production in agroecological farming systems (AFSs) does not rely on intensive use of external inputs. Rather, it is based on an integrated and holistic social-ecological approach, utilizing local resources and socially accepted and culturally adapted technologies, while maximizing the ecosystem service provision (Altieri 1989, 2002, Levidow et al. 2014, AFSA 2016, Hatt et al. 2016, FAO 2018b). As Toledo and Manuel (1990) and Altieri (2004) put it (as cited in Altieri and Toledo 2011:589), “Agroecological systems are deeply rooted in the ecological rationale of traditional small-scale agriculture ...”

Altieri et al. (2011) describe six agroecological attributes: (1) productivity, (2) resilience, (3) economic viability, social equity, and cultural diversity, (4) conservation of natural resources, biodiversity, and ecosystem services, (5) input independency and resource use efficiency, and (6) environmental protection. Thus, agroecological food systems go beyond mere food production. This is also reflected in the 10 principles of agroecology formulated by the FAO (2018b): (1) diversity, (2) synergies, (3) efficiency, (4) resilience, (5) recycling, (6) cocreation and sharing of knowledge (describing common features of AFSs); (7) human and social values, (8) culture and food traditions (describing context features of AFSs); (9) responsible governance, and (10) circular and solidarity economy (describing the enabling environment of AFSs). Thus, agroecology is clearly different from (1) organic agriculture that is based primarily on defined standards and regulations of production, (2) conventionalized organic agriculture (cf. Darnhofer et al. 2010), (3) sustainable intensification, as described by Bernard and Lux (2017), and (4) climate-smart agriculture, as explained by Pimbert (2015).

Agroecological small-scale farming is key for agricultural sustainability and food security, in particular in developing countries (IAASTD 2009, Altieri et al. 2011, De Schutter and Vanloqueren 2011, AFSA 2016, Goswami et al. 2017, da Silva 2018). Therefore, agroecological sustainability assessment frameworks and tools (ASAFTs) are needed to adequately capture the productivity and evaluate the multifunctional performance and outputs of AFSs (De Schutter and Vanloqueren 2011, da Silva 2018, GTAE 2018). Such ASAFTs would provide the required data basis for result-based policy making in the context of agroecological small-scale farming and related policy frameworks (López-Ridaura et al. 2002, Flores and Sarandon 2004, López-Ridaura et al. 2005, IAASTD 2009, De Schutter and Vanloqueren 2011, AFSA 2016, IPES-Food 2016, Sukhdev et al. 2016, Trabelsi et al. 2016, Goswami et al. 2017, Muller et al. 2017, da Silva 2018, TEEB 2018).

Civil society organizations (CSOs; e.g., AFSA, GTAE, and SWISSAID) are the main promoters and disseminators of agroecology, particularly in developing countries (cf. AFSA 2016, GTAE 2018). They are in search of suitable and widely accepted assessment frameworks to monitor and evaluate agroecological projects and to compare the strengths and weaknesses of different (agroecological) farming systems. Different expert panels, some scientists (IPES-Food 2016, Trabelsi et al. 2016, Dendoncker et al. 2018, or Lovell et al. 2010, for example), and international organizations, institutions, and initiatives (e.g. da Silva 2018, TEEB 2018) are working on this, while conventional farmers, governments, and their research organizations seem to be less active.

In ASAFTs, the evaluation criteria and frameworks of agricultural performance have to go beyond the classical agronomic and economic indicators, e.g., yield, labor productivity, or invested money per hectare, and have to become much more comprehensive and tailored to agroecological practices and principles (De Schutter and Vanloqueren 2011, AFSA 2016, TEEB 2018). New ways of measuring impact and productivity have to be established based on four key aspects: (1) local conditions, (2) the involvement of farmers (their needs and experience), (3) the consideration of the multiple functions of an agroecosystem in the definition and measurement of its productivity, and, hence, (4) the analysis of interactions among the multiple functions and their measurement, i.e., indicators. Consequently, ASAFTs are defined in this study as assessment frameworks that integrate all four key aspects.

An assessment framework is, thereby, a “... theoretical and procedural structure that underpins [the] sustainability assessment ...” (Bonisoli et al. 2018:1081), including the definition and choice of the underlying objectives, methods (i.e., assessment tools), and assumptions (i.e., thematic dimensions, operational levels, time frame, and spatial scope of a sustainability assessment; Goswami et al. 2017). Hence, the framework should help to document any interpretation made during an assessment and highlight also its subjective nature. Integrated assessment frameworks are frameworks that integrate different kinds of knowledge, i.e. knowledge from different disciplines and stakeholders, with the aim to further support societal learning and decision-making processes (TIAS 2020). Assessment tools, finally, are analytical techniques to conduct the analyses and evaluations within an assessment framework (Gasparatos et al. 2008, Gasparatos 2010, Gasparatos and Scolobig 2012).

Several studies review the numerous agricultural sustainability assessment frameworks developed in the past years and identify their strengths and weaknesses (Binder et al. 2010, Marchand et al. 2014, Schader et al. 2014, Schindler et al. 2015, Dabkiene 2016, de Olde et al. 2016a, Slätmo et al. 2017, Bonisoli et al. 2018). However, almost no studies evaluated agricultural sustainability assessment frameworks regarding their suitability to assess AFSs. An exception is Trabelsi et al. (2016), who compared three different agricultural sustainability assessment frameworks (DIALECTE, IDEA, and RAD) and concluded that these are hardly suitable to assess the performance and differences of agroecological transition farms. To remedy this, Trabelsi et al. (2016) developed an assessment framework based on a modeling approach to facilitate strategic decision making, guidance, and assistance for transitioning farming toward agroecological practices, and to assess the performance and impacts of the changes made (Trabelsi et al. 2019). However, the main challenges and required key features of ASAFTs were not specifically addressed.

To close this knowledge gap, in this study we identify and review existing ASAFTs described in the scientific literature in the context of international cooperation. We focus on indicator-based ASAFTs at the farm level. Indicators are considered the most common tool to assess agricultural sustainability (Bonisoli et al. 2018). Furthermore, indicator-based sustainability assessments are seen as most suitable to capture the complexity and multifunctionality of sustainable farming systems (Goswami et al. 2017), and best suited to serve as a basis for evidence-based policy making. It is also at this level that farmers’ decisions most directly influence the various sustainability dimensions (Marchand et al. 2014, Hodbod et al. 2016, Latruffe et al. 2016), and agricultural policy most commonly targets farms as decision-making units. Limits of this focus on the production side at the farm level are addressed in the discussion.

The two central research questions addressed here are the following: (1) Do indicator-based assessment frameworks particularly designed to evaluate AFSs at the farm level exist? (2) What are the strengths and weaknesses of these frameworks in view of the four key aspects that ASAFTs must integrate (locality; farmers' involvement; multifunctionality; interactions)?


A literature research was conducted in summer 2018 using Web of Science, with a keyword search merging the terms “agroecology” or “agriculture” with the terms “sustainability assessment,” “sustainability indicators,” “multifunctionality assessment,” “multifunctionality indicators,” “productivity assessment,” or “productivity indicators” into a total of 12 combinations. The terms “agroecology” and “agriculture” were chosen to also identify broad assessment approaches and to avoid assessments specifically targeted at codified approaches such as organic, or at niche approaches such as permaculture, etc. The aim of the literature research was to get an overview of (1) the existing indicator-based agricultural sustainability assessment frameworks at the farm-level, and (2) their relevance and applicability to evaluate AFSs in the context of international cooperation. In order to obtain an overview of the current situation in the field of agroecological monitoring and evaluation, only journal articles, reviews, and book chapters published over the past decade (2008-2018) were considered. The articles, reviews, and book chapters had to be available in English and belong to the 50 most relevant records according to Web of Science, i.e., ordered “... based on a ranking system that considers how many of the search terms are found in each record [title, key words, and abstract]” Clarivate 2018). Finally, all literature records that dealt with indicator-based, integrated (i.e., multidimensional, noncrop- and nonproduction-specific) assessment frameworks that evaluate agricultural food production systems were selected.

Figure 1 captures the structure of the literature review and analysis. In total, the literature research using the 12 key word combinations yielded 316 hits (Box A in Fig. 1). Three of these 316 papers reviewed indicators that are used in the literature to assess agricultural sustainability (cf. Hayati et al. 2010, Latruffe et al. 2016, Rasmussen et al. 2017; Box B in Fig. 1). Nine additional articles reviewed agricultural sustainability assessment frameworks (cf. Binder et al. 2010, Marchand et al. 2014, Schader et al. 2014, Schindler et al. 2015, Dabkiene 2016, Slätmo et al. 2017, Bonisoli et al. 2018, as well as de Olde et al. 2016b and de Olde et al. 2017a, which were mentioned among the first seven reviews, and deemed as relevant and, therefore, included in the final analysis). Finally, all nine reviews (Box C in Fig. 1) were analyzed regarding the evaluated frameworks, the mentioned challenges, the evaluation categories (Box D1 in Fig. 1; Table 1), and the focus on the suitability and applicability for agroecology and developing countries (Box D2 in Fig. 1; Table 2). From this literature research, we identified 19 integrated assessment frameworks (Box E and G in Fig. 1) that were included in the analysis (Box F and H in Fig. 1). They were found in the reviews (Table 3) and/or described in additional articles that resulted from the literature research (Table 4).

Four key aspects of ASAFTs

To investigate whether the 19 individual assessment frameworks are suitable and applicable to evaluate AFSs in the context of international cooperation, they were evaluated based on four key aspects: (1) local conditions; (2) farmers’ involvement, (3) integration of agricultural multifunctionality, and (4) analysis of interactions (Tables 3 and 4). The emphasis on these four aspects in this study is explained by their relevance in agroecological farming systems:

  1. The ability to account for local conditions, i.e., socio-cultural, environmental, and economic conditions, is crucial for a frameworks’ relevance and applicability in various world regions. In developing countries of the South, mainly small-scale, low-input farming feeds the people (Altieri 2002, 2004, Altieri et al. 2011). In the North and transitional economies, industrialized, intensive farming dominates agriculture, mostly producing animal feed and increasingly used for energy generation, rather than feeding people (IAASTD 2009, Cassidy et al. 2013, IPES-Food 2016). As a consequence of the significant difference between farm management systems in the Global North and those in the Global South, ASAFTs need to be adaptable to local farm management systems. This includes the adaption of measurement units and assessment methods to the measures and agroecological practices of local farmers. ASAFTs also need to be applicable and understandable to the farmers themselves (cf. aspect 2). In the Global South, there is often a lack of capacity and infrastructure to conduct sustainability assessments (cf. Minae et al. 2008). However, an ASAFTs should be based on a globally valid understanding of agroecological farming to guarantee a level of comparability while being adaptable to the local conditions in different regions. Practitioners, i.e., farmers, farmers’ organizations, CSOs, or international organizations, engaged in AFSs in different world regions can then better exchange experiences and share knowledge on their local agroecological methods and practices on a national and international level. Finally, this adaptability is an essential feature of ASAFTs because any further agricultural colonization of the Global South—as it happened by the means of food quality and sustainability standards (Mari 2017)—has to be avoided.
  2. Farmers’ involvement in the whole framework development process is an essential agroecological principle (cf. cocreation of knowledge, agroecology of knowledges; Altieri 1989, Altieri and Toledo 2011, Méndez et al. 2013, Coolsaet 2016, FAO 2018b), and crucial for the applicability, acceptance, and utility of an assessment framework at the farm-level (IAASTD 2009, Schindler et al. 2015, Coteur et al. 2016, de Olde et al. 2016a, Herrera et al. 2016, de Olde et al. 2017b, Rasmussen et al. 2017, Slätmo et al. 2017, Bonisoli et al. 2018). According to Coteur et al. (2016), there are four prerequisites for applicability and on-farm strategic decision making utility of an assessment framework that are all based on farmers’ involvement. First, an on-farm sustainability assessment framework has to be embedded in a surrounding context, i.e., institutional, social, and natural environment, which is also addressed and supported by Butler Flora (2004), because farmers know their environment and context best. Second, an assessment framework needs to be flexible because sustainability definitions and goals might change over time. These goals and definitions depend on local practitioners and their perception. Third, including farmers’ knowledge contributes considerably to agricultural sustainability and resilience (Šūmane et al. 2018). And finally, Coteur et al. (2016) stated that an assessment framework has to support and promote the communication of the sustainability performance to various stakeholders of the food system. Therefore, farmers’ involvement in the development process of an assessment framework is fundamental because researchers often fail to use an appropriate language to communicate findings, to clarify farmers’ needs and, hence, to answer farmers’ questions (Levidow et al. 2014). Rasmussen et al. (2017) have also identified a divide between researchers and practitioners in how to define agricultural sustainability, and most existing assessment frameworks are developed by experts without actively involving stakeholders and considering their needs (Binder et al. 2010, Schindler et al. 2015, de Olde et al. 2016a, Slätmo et al. 2017, Bonisoli et al. 2018). Farmers’ ability to influence how agricultural sustainability is framed and assessed is, thus, very limited, reducing the utility and the learning effect to farmers (Schindler et al. 2015, Slätmo et al. 2017). In other words, for ASAFTs it is important that they are also farmer based, i.e., reflecting farmers’ interests and understanding, and, hence, facilitate the frameworks’ applicability and usefulness to farmers.
  3. The multifunctionality of an agroecosystem refers to its multiple functional outcomes, i.e. socio-cultural and biophysical (agro)ecosystem functions like food security, ecological integrity, soil fertility, recreational area, etc., that go beyond the economic functions and food and fiber production (Van Huylenbroeck et al. 2007, Binder et al. 2010, Gliessman 2014, Hodbod et al. 2016, Bonisoli et al. 2018). The principle of agroecology is to optimize all these agroecological functions equally rather than maximizing productivity while minimizing ecological harm and impact. Multifunctionality is closely related to productivity, resilience, diversity, and other agroecosystems attributes/services, and, thus, mirrored in diverse agroecological practices. This critical principle should be reflected in an agroecological assessment framework (Rickerl et al. 2004, Renting et al. 2009, Altieri et al. 2011, Gliessman 2014, Dendoncker et al. 2018, FAO 2018b). This principle differs substantially from those of conventional understanding of sustainable agriculture, i.e., conventionalization of organic farming, the New Green Revolution, climate-smart farming, or sustainable intensification, etc., (Darnhofer et al. 2010, Holt-Giménez and Altieri 2013, Levidow et al. 2014, Pimbert 2015, Bernard and Lux 2017). Therefore, the multifunctionality assessment of AFSs is essential for legitimation of future policies and incentives internalizing positive and negative socio-environmental externalities and supporting the transition toward agroecological practices (Martín-López et al. 2014, Dabkiene 2016, Sukhdev et al. 2016, Dendoncker et al. 2018, TEEB 2018). Thus, ASAFTs must necessarily integrate multifunctionality in the definition, indication, and evaluation of agroecological productivity (IPES-Food 2016). This could be achieved by an appropriate combination of indicators (cf. aspect 4) covering different agroecological functions and/or by an appropriate productivity indicator/index that includes multiple agroecological functions and goes beyond the assessment of yields. Examples could be the “land equivalent ratio” (HLPE 2019), or a comprehensive accounting for energy flows of agricultural metabolisms (Guzmán et al. 2018).
  4. An indicator interaction analysis is understood as an evaluation of trade-offs and synergies between and among individual indicators of a sustainability assessment framework (Binder and Wiek 2007, Binder et al. 2010). Therefore, this analysis can be used to account for synergies and trade-offs between multiple agroecosystem functions. This is central for success (or failure) of an agroecological system that tries to optimize all functions equally. AFSs are quite complex and based on multiple interactive social-ecological processes and functions. Most agricultural assessment frameworks do not implement an analysis of (indicator) interactions, although trade-offs between sustainability dimensions, goals, and indicators can then neither be identified nor managed (Binder et al. 2010, Slätmo et al. 2017, Bonisoli et al. 2018). Agroecological analysis has, therefore, to follow a holistic approach, considering and measuring the performance of several interacting functions (cf. aspect 3). A careful interaction and trade-off analysis of agroecological functions and corresponding indicators could facilitate the equal inclusion of the numerous agroecological aspects. Furthermore, analyzing synergies and trade-offs between different indicators is the basis for modeling dynamic agroecosystems, which is highly influenced by contextual data. Such modeling of complex system dynamics supports the understanding of the system. The analysis of indicator interactions and, hence, trade-offs and synergies among elements within the system is, therefore, considered as an essential aspect of ASAFTs (Altieri 2002, 2004, Minae et al. 2008, Altieri et al. 2011, Mari 2017).

Limiting factors: credibility, salience, and legitimacy

Recently, de Olde et al. (2018) emphasized three limitation factors that lower a frameworks’ implementability in practice, in political decision-making processes, and, hence, its contribution to the transition toward a more sustainable agriculture if they are missing: credibility, salience, and legitimacy. There is a proliferation of frameworks and tools that all have the ambition of certainty and comprehensiveness in assessing agricultural sustainability. However, most of these frameworks are insufficiently credible, salient, and/or legitimate because they lack robustness, are not adapted to the needs of the end users, and/or are based on assumptions and valuations that are not commonly elaborated or accepted (de Olde et al. 2018). This finding is also supported by the incompatibility and nontransparency challenges mentioned in several of the reviews that were analyzed (cf. Marchand et al. 2014, Schader et al. 2014, Dabkiene 2016, de Olde et al. 2016a, de Olde et al. 2017a, Bonisoli et al. 2018). Objectives, assumptions, methodology, tools, definitions, complexity, as well as data requirements differ quite substantially among individual frameworks, making the assessment results barely comparable and questioning the validity of the assessment (de Olde et al. 2017a). If these framework elements are not clearly framed, the sustainability assessment becomes nontransparent. This nontransparency can lead to misperception of the sustainability performance, distorted results, and follow-up actions and, hence, to a reduced reliability and relevance of the framework (de Olde et al. 2017a). Hence, for ASAFTs, which aim to facilitate the transition toward agroecological farming, transparency, comparability, salience, credibility, and legitimacy are fundamental. These features are highly interlinked with the four key aspects discussed above. To overcome these challenges and limitations, de Olde et al. (2018) proposed that assessment frameworks have to become more transparent, harmonized, participative and motivating to farmers. Harmonization among locally adapted assessment frameworks allows a certain level of comparability and an exchange of experiences, and can be facilitated by an international understanding of agroecological farming and common guidelines or goals. Considering local conditions as well as accounting for the multiple functions of an agroecosystem and their measurements are also important with regard to the credibility of ASAFT. Only if the local conditions, the different stakeholders and perspectives, as well as the various interacting agroecological functions and measurements are considered and well documented, can a certain scientific adequacy and transparency be guaranteed. Further, the participation and involvement in the development process allows people to assume co-ownership of the assessment framework, thus, increasing legitimacy (Rametsteiner et al. 2011).

Finally, it was evaluated whether the frameworks were designed to assess and capture AFSs. If an assessment framework was based on agroecological principles and concepts, the framework was seen as specifically designed to evaluate AFSs. The four key aspects, i.e. locality, farmers’ involvement, multifunctionality, and interactions, were then assessed in more detail to conclude about the suitability or relevance of a framework for assessing AFSs.


All nine reviews included in this study compared various frameworks based on different evaluation categories and research objectives (Table 1). Only one review focused on the frameworks’ suitability to assess AFSs (cf. Slätmo et al. 2017), and only one analyzed the frameworks’ adaptability to farming systems in developing countries (cf. Schindler et al. 2015; Table 2). In total, 19 frameworks were identified in the primary and secondary literature in our study. All of them were analyzed with focus on the four key aspects, i.e., their adaptability to local conditions, involvement of farmers, agroecological multifunctionality, and indicator interactions (Tables 3 and 4). Five assessment frameworks were identified that were specifically designed to assess AFSs and positively responding to a majority of our evaluation criteria (i.e., the frameworks of López-Ridaura et al. 2002 [MESMIS], López-Ridaura et al. 2005 [MMF], Lovell et al. 2010, Trabelsi et al. 2016, Dendoncker et al. 2018). In the following, those five frameworks are discussed in more depth after presenting the results of analyzing all 19 frameworks in general.

Description of existing agricultural sustainability assessment frameworks

In total, 19 integrated sustainability assessment frameworks were selected and analyzed (Tables 3 and 4). Most of the assessment frameworks were developed for the Global North, in particular for agricultural systems of European countries. Only the MMF and the MESMIS frameworks were specifically designed for or applied in developing countries, while the RISE and the SAFA frameworks were developed for global application. The remaining frameworks were developed for or applied in European countries and established without explicitly involving farmers. Only three frameworks were developed for or implemented in non-European countries (Brazil, Cuba, and USA).

Only three reviewed frameworks focus specifically on AFSs. With the exception of Trabelsi et al. (2016), these frameworks involved farmers during the whole development and assessment process. Trabelsi et al. (2016) do not explicitly mention farmers’ participation but they take the view that farmers have to participate in innovative processes, be recognized as stakeholders and not only as beneficiaries, and, therefore, get involved in decision making on agricultural sustainability. Other frameworks showed participatory approaches to different degrees in indicator selection and validation, or the final discussions of the assessment result. While Fernandes and Woodhouse (2008), for example, conducted a workshops with ecological, agroecological, and nonecological farmers to agree on an indicator set, other frameworks like the one of Castoldi and Bechini (2010) included farmers only in data collection or setting the weights of certain indicators and evaluation elements. However, most of the remaining frameworks do not involve farmers at all and farmers are often only seen as a data source.

Finally, multifunctionality and productivity are rarely explained nor clearly or explicitly defined in the assessment frameworks. Generally, the multifunctionality concept was not considered in the definition or measurement of productivity, which is measured as crop yield per area, economic profitability, or farming efficiency. However, in the MMF and the MESMIS frameworks, natural resource management systems are considered as complex systems within which different interlinked activities aim to achieve multiple economic, environmental, and social objectives, providing various outputs, goods, and services. In addition, multifunctionality was explained in the four frameworks described by Lovell et al. (2010), Andersen et al. (2013), Trabelsi et al. (2016) and Dendoncker et al. (2018). Only one article explicitly stated how productivity is understood.

Because all reviewed frameworks were based on a multidimensional sustainability definition, most included the three typical sustainability dimensions, i.e., the ecological, economic, and social dimension. In three frameworks, these sustainability dimensions were complemented by a policy or governance dimension (cf. the RISE and SAFA frameworks) or, as in the case of Ryan et al. (2016), by an innovation dimension. In contrast, in two frameworks (the MESMIS framework and the MMF) the sustainability dimensions were replaced by the five sustainability attributes: productivity, stability, reliability, resilience, and adaptability, complemented by the attributes of equity and self-empowerment in the MESMIS framework. Only the assessment framework of Castoldi and Bechini (2010) solely considered the environmental and economic sustainability dimensions. Trabelsi et al. (2016) complemented the three sustainability dimensions with a health and a crop protection dimension and Lovell et al. (2010) replaced the economic dimension with a production dimension.

The majority of the assessment frameworks did not include an indicator interaction analysis. In most of the frameworks where indicators were analyzed, the indicators are aggregated into an individual index or visualized in a score spider diagram. Only the integrated ecosystem service assessment framework described by Dendoncker et al. (2018) was not based on a composite indicator, and assesses the ecosystem service provision based on a range of indicators.

Description of the five assessment frameworks specifically designed for AFSs

I. Methodological framework for the sustainability evaluation of AFS like peasant systems

The MESMIS framework (López-Ridaura et al. 2002) is a methodological framework for the sustainability evaluation of peasant natural resource management systems (MESMIS is the Spanish acronym of Framework for Assessing the Sustainability of Natural Resource Management Systems). These peasant systems are closely related to agroecological farming systems. They are defined as complex systems within which different interlinked activities help to achieve various environmental, economic, and social objectives, including the provision of different goods and services. MEMSIS provides a structure for developing an indicator-based assessment framework for natural resource management systems based on four premises: (1) The sustainability of the system is defined by seven core attributes (productivity, stability, reliability, resilience, adaptability, equity, self-empowerment). A system is sustainable if it achieves a high level of performance in all seven attributes; (2) The assessment is limited to a predefined geographical region, a spatial scale, and a certain time period because natural resource management systems are quite diverse and subjected to different cultural and environmental conditions; (3) The framework development and application are participatory processes including farmers, technicians, community representatives, and other stakeholders; (4) Sustainability cannot be measured per se, but only through a system comparison because the definition of sustainability is relative. The operational structure of MESMIS comprises six steps:

  1. The natural resource management systems have to be identified and characterized; flowcharts are often used to characterize the components, in-/outputs, activities, social and economic features, and the organization of the system.
  2. The critical features of systems are defined by identifying the aspects that enhance or constrain the seven core attributes and, thus, affect the systems’ sustainability.
  3. For each attribute, different critical system features are defined and then translated into indicators. For each critical system feature, diagnostic criteria, i.e., general analysis objectives, are defined and adequate indicators are selected. These indicators need to cover all seven attributes mentioned above as well as general sustainability criteria (social, ecological, economic).
  4. Actual measurement of the indicators.
  5. Synthesis and analysis of the results including their visualization and the analysis of relations between indicators, identifying positive and negative feedbacks.
  6. Conclusions and recommendations.

López-Ridaura et al. (2002) stress that the fundamental systemic properties of sustainable farming systems have to be discussed further. They also state that it is very important to consider social-ecological aspects, indicator interrelations, and participatory approaches. Finally, linkages between different assessment levels are seen as very important to achieve consistency, but are not yet included in the MESMIS framework.

II. Multiscale methodological framework for the evaluation of AFS like peasant systems

The multiscale methodological framework (MMF) is a successor framework of MESMIS (López-Ridaura et al. 2005). MMF is based on five sustainability attributes, two referring to the systems’ functioning (productivity, stability) and three focusing on the systems’ behavior when facing internal or environmental changes (resilience, reliability, and adaptability). The aim of MMF is to assess the sustainability of natural resource management system by facilitating the development of a site-specific set of criteria and indicators evaluating the systems’ performance in view of each attribute at different scales. MMF has a cyclic structure with seven operational steps per cycle, strongly following a participatory approach: (1) The study area within which the farming system is located, identified, and characterized; (2) Relevant impact scales within the study area are defined and evaluation objectives for each scale are determined, both together with the different stakeholders; (3) Evaluation criteria are derived from the objectives and combined with the five sustainability attributes of natural resource management systems. With regard to their interrelations, indicators are defined for each chosen criterion; (4-7) In the remaining steps, functions for the indicators and their relations at and among each assessment scale are developed, modeled, analyzed, and discussed resulting in a relative sustainability performance measure, recommendations for best practices, and research objectives.

III. Assessment framework for AFSs based on a multifunctionality approach

The first framework specifically focusing on AFSs was developed by Lovell et al. (2010) and is based on the theories of agroecology and multifunctionality, in this way creating an integrated approach to design agroecosystems. As recommended in the works of Mullender et al. (2017) and Rasmussen et al. (2017), Lovell et al. (2010) forego new terminologies or definitions and refer to the combination of existing theories and corresponding indicators or attributes. They combine multifunctionality and agroecology, because these concepts are closely related to each other, and multifunctionality allows also to include in the assessment the impacts on a larger landscape level. Thereby, three functions, i.e., production, ecological, and cultural, are considered and for each function five attributes are determined and rated together with farmers. The farm landscape is divided according to land use and for each division all 15 attributes are scored according to a range from -2 to 2. Finally, the scores are summed and weighed relative to the size of the land use divisions. The total score indicates the multifunctional performance of the agroecosystem. This approach allows analyzing the interaction between multiple functions or attributes on various land use units. Using attributes and a scoring methodology instead of absolute indicators facilitates the applicability and the utility for farmers.

IV. Assessment framework for AFSs based on a modeling approach

The second framework that specifically assessed AFSs also uses aggregated composite performance indices. Trabelsi et al. (2016), who developed the framework, explained that classical indicators would not capture the dynamic process of agroecological transition. After a comparison of three conventional methods for assessing agricultural sustainability in practice (IDEA, DIALECTE, and RAD), Trabelsi et al. (2016) concluded that these methods are not sensitive or adapted enough to assess the performance of organic farms in transition to agroecology. Therefore, they developed an assessment framework based on a modeling approach at plot and farm level that would assist farmers in strategic decision making for a transition to agroecological practices, assessing the performance of this transition process, and simulating possible decision impacts and consequences (cf. updated description of the ESSIMAGE framework, Trabelsi et al. 2019). According to Trabelsi et al. (2016:149), “[t]he transition to agroecology is a dynamic process characterized by different relationships between the objectives, agricultural techniques, means of implementation, and impacts ...” An agricultural technique can, thereby, serve multiple objectives, have several means of implementation, and, hence, one or more impacts. To capture this complex dynamic, the assessment framework is based on interaction matrices that reflect these relationships.

Trabelsi et al. (2016) identified five impact topics, i.e., environment, crop protection, health, society, and economy, that have to be assessed before, during, and after agroecological transition. For each topic, the authors determined indicators that are subsequently weighed and composed into a performance index. Except for the economic indicators, each indicator is a function of parameters that reflect the implementation of certain agroecological techniques that influence the performance of the corresponding indicator. Thereby, the parameters that are allocated to a certain indicator depend on the considered production system. In addition, the implementation and agroecological techniques depend on the environment of the farms. Each parameter is further weighed according to its importance for achieving a certain performance level. Finally, a score is assigned to each parameter to then derive the performance index. Inverse values of parameters are used if parameters affect the indicator negatively. In comparison with the first assessment framework, the performance estimation is much more complex, time intensive, and even partially computer-based. The framework was developed mainly by scholars, although the importance of farmers’ involvement and participatory research was acknowledged. Based on the parameters used (agroecological techniques) and the inclusion of site-specific data, simulations of future performance are possible. Furthermore, the assessment framework analyzes the interaction and relationships of the multiple agroecological techniques, attributes, and objectives. Tests in practice have shown that multiple agroecological practices have to be implemented holistically to have an effect on agroecological, social, but also economic performance (Trabelsi et al. 2019). Hence, the multifunctionality of the agroecosystem is clearly integrated.

V. Assessment framework for AFSs based on the evaluation of ecosystem services

The third framework that specifically assessed AFSs was developed by Dendoncker et al. (2018) and is similar to the one of Lovell et al. (2010). Dendoncker et al. (2018) suggest a four-step approach to facilitate designing agroecological transition. Their framework is based on the theory that agroecological practices aim to optimize ecosystem services and, hence, foster resilience and sustainability of agroecosystems. The first step of the framework is to understand the current situation. This initial step includes a multilevel assessment of agroecological practices and their impact on ecosystem services. For this purpose, indicators are identified in a participatory process by scientists, ecosystem service managers (including farmers), and beneficiaries of ecosystem services. These indicators should cover a number of ecosystem services that can be categorized in provisioning, regulating, and cultural services. Furthermore, this assessment step includes a social ecosystem service valuation analysis. After having assessed the current situation, different possible scenarios, key drivers of change, and trade-offs and synergies are analyzed in a second step. Subsequently, in a third step, the stakeholders involved have to agree on the most accepted future scenario. The implementation of the chosen scenario is then part of the fourth step. Because the understanding of sustainability and the social valuation of ecosystem services might change over time the proposed assessment has to be repeated from time to time. Because the four assessment steps are based on a participatory approach, involving different stakeholders, enough time has to be assigned for it.


The majority of the frameworks analyzed in this study do not focus on evaluating AFSs and were not suitable or applicable to evaluate AFSs in an international cooperation context. This is so because they barely involve farmers, rarely include indicator trade-off and interaction analyses, and often also do not consider explicitly multiple agricultural functions nor account for local conditions in developing countries. Furthermore, there was no agricultural sustainability assessment framework review that particularly studied the suitability or applicability of the existing assessment frameworks to evaluate the multifunctional and multidimensional outputs of AFSs. In total, only five assessment frameworks were identified that are (a) explicitly designed to evaluate AFSs (cf. Lovell et al. 2010, Trabelsi et al. 2016, Dendoncker et al. 2018) or (b) related to natural small-scale farming systems that are rooted in AFSs, focusing on systemic sustainability attributes (López-Ridaura et al. 2005, Speelman et al. 2007). Below, we focus on these five frameworks.

Adaptability to local conditions and complexity of implementation

ASAFTs should be adapted to local conditions in different world regions. Most ASAFTs are, however, developed in the context of the Global North. Hence, they may be less suitable or adapted to conditions that are present in countries of the Global South. The agricultural context, i.e., infrastructure and practices, the sustainability knowledge and measures, as well as the assessment traditions are considerably different between the North and the South. For instance, although record keeping is very common in commercial farming in the Global North, small-scale farmers in the Global South rarely keep written records about their agricultural activities, inputs, and outputs. Yet, record keeping is the prerequisite of data collection and, thus, key for agricultural assessment. A study by Minae et al. (2008) showed, for example, that farm data collection, utilization, and dissemination among small-scale farmers in sub-Saharan Africa are nonexistent or of low quality. Reasons for this lack of farm data in sub-Saharan Africa are (1) the cumbersome nature of record keeping, (2) the complexity of natural small-scale farming, (3) the absent business features of most small-scale farming systems, and, finally, (5) the incompatibility between conventional data systems and the subsistence management systems in developing countries, i.e., the measures and practices of smallholders (Minae et al. 2008). However, record keeping is essential for successful and sustainable decision making at farm level and, hence, very important for small-scale farmers in developing countries (Bockstaller et al. 1997, Pope et al. 2004, Pintér 2007, Minae et al. 2008, Coteur et al. 2016).

Accordingly, ASAFTs need to be based on simple, i.e., understandable and manageable, farmer-based measures and tools to collect and evaluate farm data. Although the framework of Lovell et al. (2010) seems to be quite straightforward in usage, the other two frameworks of Trabelsi et al. (2016) and Dendoncker et al. (2018) require technical support. Measurement and calculation methods of these two frameworks appear to be time and knowledge intensive, aiming at integrating multifunctionality and dynamic processes into the evaluation. This is so, even though, in the case of Trabelsi et al. (2016; cf. also Trabelsi et al. 2019) the method is actually based partially on simple agricultural, social, and economic parameters. Similarly, the application of the MMF and the MESMIS frameworks requires institutional support for indicator sampling and evaluation, although these frameworks have been developed in a truly participative process and adapted to the needs and requirements of the Global South.

Finally, to be easily applicable by different stakeholders/users in the Global North and Global South, ASAFTs might need to be based on a dual structure offering a simplified and a complementary more comprehensive implementation, with correspondingly different levels of detail. Parameters for the simplified version could easily be collected by the farmers themselves, while the other parameters would be measured and evaluated with the help of professionals or by trained farmers with the corresponding technical support. This structure might also avoid unnecessary replication when collecting agricultural sustainability data and enhances the usefulness of the frameworks for farmers, two main concerns of practitioners (Mullender et al. 2017).

Further, to be suitable for different stakeholders and users worldwide, the applicability has to go beyond farm level and include relevance for policy analysis. Except MMF, the analyzed frameworks are not designed to evaluate which existing or new policies at a higher level could be used in order to support the implementation or scaling up of successful and promising AFSs. Although López-Ridaura et al. (2002) have already identified the importance of articulating the interrelation between different evaluation scales, only the MMF comprises a multiscale approach that allows for the analysis of the effect of agroecological policies and practices at different farm and food system levels. Higher assessment levels, i.e., regional, national, and international level, need to be considered when assessing AFSs at the farm level because these higher levels also affect farmers’ decisions and the sustainability of the activities (Russillo and Pintér 2009, Hayati et al. 2010, de Olde et al. 2016a, Latruffe et al. 2016). At these higher levels, linkages between different assessed agroecological food system dimensions, i.e., production, processing, distribution, and consumption, can be made. Furthermore, linkages to higher assessment levels allow considering and evaluating other factors like policies, economics, the local and national culture, and the environment.

The assumption that farmers will change their behavior and perform more sustainably if corresponding sustainability assessment data and information is available is misleading (Slätmo et al. 2017). In reality, farmers perceive their scope of action as very limited within the institutional context that determines their farming practices (de Olde et al. 2016a). This means that farmers’ decisions and behavior with regard to the agroecological production also depends on other system dimensions and factors beyond the farm-level that are based on choices made by other stakeholders of the food system. ASAFTs need, therefore, to address the interlinkage between lower and higher assessment levels and have common guidelines, i.e., agroecological attributes and framework standards, that facilitate the horizontal and vertical harmonization among ASAFTs at different assessment levels. Methodological approaches as used by Dendoncker et al. (2018), López-Ridaura et al. (2002), and López-Ridaura et al. (2005) suggest some guidelines on a regional level. Yet, widely accepted guidelines for agroecological assessment frameworks like the SAFA guidelines for sustainable agriculture (cf. FAO 2014) are missing. However, FAO (2019) recently published a first test version of a tool for agroecology performance evaluation (TAPE) that describes the process of development and guidelines for application that may result in such commonly used guidelines. A potential guiding approach that would facilitate the harmonization among ASAFTs might be a linkage to the SDGs (cf. AFSA 2016) or to the 10 elements of agroecology (cf. FAO 2018b), and, hence, to commonly accepted objectives of higher assessment levels (Russillo and Pintér 2009). Some assessment frameworks developed by practitioners in the field of agroecology as well as TAPE are already referring to the 10 agroecology elements by FAO, which is not the case with regard to the analyzed frameworks for evaluating AFSs.

Finally, assessment frameworks are inherently normative and value-based (Rametsteiner et al. 2011, Alrøe et al. 2016, de Olde et al. 2018). This normative aspect of assessment frameworks is often neglected but very important for the utility and acceptance of an assessment (Rametsteiner et al. 2011). Linking farm-level assessments to higher level sustainability objectives and measurements might enhance the normative transparency to a wider public and harmonize the individual ASAFTs. It might also help to overcome the limits of focusing on the production side at the farm level. At higher levels new dimensional focuses could be introduced and interlinked. Hence, research about common ASAFT features is needed to avoid any uncoordinated proliferation of ASAFTs that would decrease the credibility, salience, and legitimacy of the frameworks and impede an interplay between corresponding policies (cf. Marchand et al. 2014, Schader et al. 2014, Dabkiene 2016, de Olde et al. 2016a, 2017a, b, 2018).

Farmers’ involvement

The degree of farmers’ involvement is quite different among the analyzed frameworks for evaluating AFSs. Trabelsi et al. (2016) only mention farmers’ relevance. Lovell et al. (2010) determine and rate the agroecological attributes of the different land use units together with the farmers, while the framework of Dendoncker et al. (2018) includes farmers as well as other stakeholders of the AFSs in each step of the framework. Finally, the MMF and the MESMIS framework present methodological approaches for NGOs, farmers’ organizations and farmers to develop a framework to evaluate agroecological sustainability, following a participative bottom-up strategy. These different involvement levels are also reflected by the varying complexity and applicability of the individual frameworks. In the end, further research is needed to define an appropriate level of farmers’ involvement that would maximize the quality and applicability of an ASAFT.

The analyses conducted in this study showed that the involvement of farmers in the development process of an assessment framework has only been explicitly studied since 2015, whereas the involvement of other stakeholders was investigated earlier. Awareness for this needs to increase. In this respect, all identified frameworks for evaluating AFSs include participative elements but transparency, harmony, and contextualization are not automatically given. All frameworks show very different methodological assessment approaches: although Dendoncker et al. (2018), López-Ridaura et al. (2002), and López-Ridaura et al. (2005) describe methodological frameworks that lead to jointly developed assessment frameworks, Lovell et al. (2010) and Trabelsi et al. (2016) present final but customizable frameworks for evaluating AFSs. Although Dendoncker et al. (2018), López-Ridaura et al. (2002), and López-Ridaura et al. (2005) use a set of indicators, Lovell et al. (2010) assign scores to different agroecological attributes, and Trabelsi et al. (2016) calculate a complex agroecological index, and have their individual set of agroecological indicators and attributes. The selection of indicators and attributes is part of a participatory decision but not always explicitly explained in the frameworks. Only in the MMF and the MESMIS framework are the chosen sustainability attributes outlined clearly.

Multifunctional productivity measures

Although all five identified frameworks evaluating AFSs do explicitly integrate the principle of multifunctionality, the frameworks do not use multifunctional productivity indicators that are tailored to AFSs or non-European small-scale farming systems. Whereas Lovell et al. (2010) used a qualitative productivity indicator, Dendoncker et al. (2018) suggests conventional productivity indicators like yield, i.e., harvest per area, and quality. Conventional productivity indicators, like yields, labor demand, cost/benefit ratio, or total income are also used in TAPE and many case studies that applied the MMF and the MESMIS framework, indicating the efficiency and profitability of the assessed system. These indicators are not able to capture the multidimensional productivity of multicrop systems that are very common in agroecology and subsistence farming. Although labor is included in present cost-benefit calculations, negative impacts of farm operations such as polluting emissions or public good resource utilization are often externalized, causing unsustainable emission levels and resource use.

However, all analyzed frameworks evaluating AFSs are in their structure very flexible, a fundamental principle in agroecology where practices have to be adapted to the local environment and context (Hatt et al. 2016). In that respect, conventional productivity indicators could easily be complemented with multifunctional ones that are adapted to AFSs without changing the structure of the assessment tool. A quantitative indicator that is often considered as alternative to yields is the net income per unit area. It is used, for example, in the MESMIS framework (cf. Speelman et al. 2007) as well as the framework of Trabelsi et al. (2016) and proposed by the CSO coalition at the Second International Symposium on Agroecology in Rome (P. Rosset 2017, personal communication). However, although this indicator better captures the multifunctional productivity, i.e., the diversity of products, of (small-scale) farming, it also shows certain disadvantages. A particular challenge is the high dependency of this measure for productivity on world food market prices that are very volatile, barely representative for local markets, and based on an economic system that agroecology aims to transform. Furthermore, Trabelsi et al. (2019) stated that with a focus on financial criteria to evaluate the economic performance of a farming system, wider economic impacts on society have to be internalized as well in future agricultural assessments. Finally, alternative indicators that capture the multifunctional productivity of AFSs are still part of future research (cf. also GTAE 2018).

Indicator interaction: trade-offs and synergies

The analysis of trade-offs and interactions of sustainability components and indicators are not common among most individual agricultural sustainability assessment frameworks. In contrast, the five frameworks looking explicitly at AFSs or related systems try to address this. Lovell et al. (2010) and Trabelsi et al. (2016) used matrices as a tool to visualize the interrelations between various agroecological practices and different ecosystem services and impacts. Dendoncker et al. (2018) used a multilevel approach to better understand the impacts of agroecological processes on ecosystem services and underlying processes. In the MESMIS framework, relationships, i.e., positive and negative interactions, between different indicators are analyzed in the fifth operational step, the synthesis and integration of the results. In case of the MMF, the indicator interaction analysis is integrated in the fourth operational step by developing functions for the relationships between the individual indicators at each scale and among different assessment levels. However, the methodology used to analyze indicator interactions is not clearly outlined in both the MMF and the MESMIS framework. In comparison to the other frameworks, the indicator interaction analysis approach of Lovell et al. (2010) and Trabelsi et al. (2016) allows to estimate the effect of different practices on one ecosystem service or social-ecological objective at the same time. Trabelsi et al. (2019) showed in different assessment cases that a single agroecological technique can not lead to a significant change in the overall agroecological, social, and economic performance of a farming system, a holistic implementation of different techniques is needed.


Two frameworks, the MMF and the MESMIS frameworks, were identified to assess small-scale farming systems that are closely related with AFSs. Additionally, three assessment frameworks (cf. Lovell et al. 2010, Trabelsi et al. 2016, Dendoncker et al. 2018) were found in the literature that explicitly aimed at evaluating AFSs. All of these frameworks included an indicator interaction analysis for which in two cases innovative matrices are used to analyze the multiple impact and outcome of certain agroecological practices. This matrix tool seems to be a key element for ASAFTs. Consequently, they all considered agroecological multifunctionality at a framework level, while, at an indicator level, agroecological multifunctionality and practices are not fully addressed yet. Monetary total productivity measures that are alternatively proposed to capture the multiple agroecological products seem to be conflicting with the agroecological principles and need further reflection. New indicators that capture the multiple agroecological, social, and economic outputs, practices, and impacts need to be developed and established (cf. Trabelsi et al. 2019). However, the analyzed frameworks are quite flexible, i.e., indicators can easily be exchanged. Furthermore, three out of five analyzed frameworks are customized to local conditions in the Global North. However, they are less suitable to the situation in the Global South where the tradition of record keeping and technical equipment are missing. Therefore, a dual framework structure is proposed that combines farmers’ measurements with scientific indicators. This would also allow to fully integrate farmers in the assessment framework process, increasing the applicability and usefulness of the framework to farmers globally. Although in comparison to other existing frameworks, farmers had an important role in the five identified frameworks that evaluated AFSs, they were involved to different degrees. Thus, there is a potential for improving farmers’ involvement, especially, with regard to the important factors of credibility, salience, and legitimacy that were identified in addition to the four analyzed aspects as important elements of ASAFTs. Frameworks for evaluating AFSs need to become more transparent, harmonized, and implementation focused. This could be achieved through common ASAFTs guidelines that facilitate the horizontal and vertical harmonization among assessment frameworks and policies within an agroecological food system. Thus, we finally recommend that future assessment frameworks for evaluating AFSs should (1) emphasize indicator interactions using matrix tools; (2) focus on agroecological multifunctionality applying (productivity) indicators tailored to AFS approaches; (3) be globally adaptable to local conditions and fully involve farmers because of a dual structure; (4) consider harmonization factors, e.g., the 10 elements of agroecology; (5) be transparent; and (6) take into account vertical interactions with AFS policies.

Innovative transdisciplinary and participatory research about published and unpublished frameworks and tools for evaluating AFSs are required to explore the new field of ASAFTs and to facilitate a transition toward an agroecological world food system. Many CSOs are already working on frameworks for evaluating AFS, but to avoid any uncontrolled and inconsequential proliferation of assessment frameworks, a common platform and widely accepted guidelines have to be established with scientific assistance. Unpublished frameworks mirror local needs and are an inspirational source for any future ASAFT guideline. Therefore, new scientific methods and topics that are tailored to agroecology and the topic of ASAFTs need to be recognized and financially supported by private and public institutions. Agroecology and corresponding evaluation frameworks and tools are a promising approach to contribute to a sustainable food/farming system and, hence, to achieve the sustainable development goals. It will be fundamental to work on a commonly accepted reference framework to evaluate AFSs all over the world. Detailed research about the necessary features of ASAFTs that complement and deepen the four aspects identified in this study, i.e., adaptability to local conditions, farmers’ involvement, multifunctionality, and accounting for interactions between different functions and their measurement, is highly needed. Agroecological productivity indicators and possible indicator interaction analysis methods should be identified and studied. Moreover, the linkages between ASAFTs at the farm level and higher levels, i.e., regional, national, and international, have to be investigated (cf. also Russillo and Pintér 2009). Finally, capacity building among farmers, especially in the Global South, is required to involve them in the development process of ASAFTs and adapt the frameworks to their needs. Part of future investigations should also (1) be simple assessment measures with which farmers in developing countries but also in industrialized countries can evaluate their agroecological performance in a way that is scientifically sound and at the same time providing relevant information to the farmers; (2) be sensitive to the commonly used monetary total productivity indicator with regard to global financial markets; (3) understand the impact of record keeping and ASAFTs on farmers’ livelihoods (in particular in the Global South); and (4) test and evaluate common agroecological assessment guidelines such as TAPE, their impact, and applicability.


Responses to this article are invited. If accepted for publication, your response will be hyperlinked to the article. To submit a response, follow this link. To read responses already accepted, follow this link.


We would like to thank Mr. Michael Farrelly, Program Officer at the Alliance for Food Sovereignty in Africa (AFSA), Mr. Peter Rosset, Professor at the Department of Agriculture, Society and Environment at ECOSUR in Mexico, and Ms. Dominique Barjolle, Senior researcher and lecturer at ETH Zurich and Associate Senior Scientist at Unité Mixte de Recherches, for informative discussions and constructive inputs. AM gratefully acknowledges financial support from the Mercator Foundation Switzerland. Finally, we want to thank two anonymous reviewers for their detailed critique and constructive comments.

Data Availability Statement

The data that support the findings of this study are published, peer-reviewed scientific papers that were derived from the web of science and are readily available in the public domain. We have listed all papers in the tables that are part of the main body of our manuscript.


Alliance for Food Sovereignty in Africa (AFSA). 2016. Agroecology: the bold future of farming in Africa. AFSA and Tanzania Organic Agriculture Movement, Dar es Salaam, Tanzania.

Alrøe, H. F., H. Moller, J. Læssøe, and E. Noe. 2016. Opportunities and challenges for multicriteria assessment of food system sustainability. Ecology and Society 21(1):38.

Altieri, M. A. 1989. Agroecology: a new research and development paradigm for world agriculture. Agriculture, Ecosystems & Environment 27(1):37-46.

Altieri, M. A. 2002. Agroecology: the science of natural resource management for poor farmers in marginal environments. Agriculture, Ecosystems & Environment 93(1-3):1-24.

Altieri, M. A. 2004. Linking ecologists and traditional farmers in the search for sustainable agriculture. Frontiers in Ecology and the Environment 2(1):35-42.[0035:LEATFI]2.0.CO;2

Altieri, M. A., F. R. Funes-Monzote, and P. Petersen. 2011. Agroecologically efficient agricultural systems for smallholder farmers: contributions to food sovereignty. Agronomy for Sustainable Development 32(1):1-13.

Altieri, M. A., and V. M. Toledo. 2011. The agroecological revolution in Latin America: rescuing nature, ensuring food sovereignty and empowering peasants. Journal of Peasant Studies 38(3):587-612.

Andersen, P. S., H. Vejre, T. Dalgaard, and J. Brandt. 2013. An indicator-based method for quantifying farm multifunctionality. Ecological Indicators 25:166-179.

Bernard, B., and A. Lux. 2017. How to feed the world sustainably: an overview of the discourse on agroecology and sustainable intensification. Regional Environmental Change 17(5):1279-1290.

Binder, C. R., G. Feola, and J. K. Steinberger. 2010. Considering the normative, systemic and procedural dimensions in indicator-based sustainability assessments in agriculture. Environmental Impact Assessment Review 30(2):71-81.

Binder, C. R., and A. Wiek. 2007. The role of trans-disciplinary processes in sustainability assessment of agricultural systems. Pages 33-48 in F. J. Häni, L. Pintér, and H. R. Herren, editors. From common principles to common practice. Proceedings of the 1st Symposium of the International Forum on Assessing Sustainability in Agriculture (INFASA). 16 March 2006, Bern, Switzerland. International Institute for Sustainable Development and Swiss College of Agriculture, Winnipeg, Manitoba, Canada. [online] URL:

Bockstaller, C., P. Girardin, and H. M. G. van der Werf. 1997. Use of agro-ecological indicators for the evaluation of farming systems. Developments in Crop Science 25:329-338.

Bonisoli, L., E. Galdeano-Gómez, and L. Piedra-Muñoz. 2018. Deconstructing criteria and assessment tools to build agri-sustainability indicators and support farmers’ decision-making process. Journal of Cleaner Production 182:1080-1094.

Butler Flora, C. 2004. Community dynamics and social capital. Agroecosystems analysis. American Society of Agronomy, Crop Science Society of America and Soil Science Society of America, Madison, Wisconsin, USA.

Cassidy, E. S., P. C. West, J. S. Gerber, and J. A. Foley. 2013. Redefining agricultural yields: from tonnes to people nourished per hectare. Environmental Research Letters 8(3):034015.

Castoldi, N., and L. Bechini. 2010. Integrated sustainability assessment of cropping systems with agro-ecological and economic indicators in northern Italy. European Journal of Agronomy 32(1):59-72.

Clarivate. 2018. Web of Science core collection help. Clarivate Analytics, Philadelphia, Pennsylvania, USA. [online] URL:

Coolsaet, B. 2016. Towards an agroecology of knowledges: recognition, cognitive justice and farmers’ autonomy in France. Journal of Rural Studies 47:165-171.

Coteur, I., F. Marchand, L. Debruyne, J. Bijttebier, L. Triste, and L. Lauwers. 2014. Development and evaluation of an on-demand sustainability tool in Flanders. Pages 38-48 in T. Aenis, A. Knierim, M.-C. Riecher, R. Ridder, H. Schobert, and H. Fischer, editors. Farming systems facing global challenges: capacities and strategies. Proceedings of the 11th European IFSA Symposium. Humbolt-Unitversität, Berlin. International Farming Systems Association (IFSA) Europe; Leibniz-Centre for Agricultural Landscape Research (ZALF), Müncheberg, Germany.

Coteur, I., F. Marchand, L. Debruyne, F. Dalemans, and L. Lauwers. 2016. A framework for guiding sustainability assessment and on-farm strategic decision making. Environmental Impact Assessment Review 60:16-23.

da Silva, J. G. 2018. Scaling-up agroecology to contribute to the Sustainable Development Goals. Chair’s summary of the 2nd International Symposium on Agroecology, 3-5 April, Rome, Italy. Food and Agriculture Organization of the United Nations, Rome, Italy.

Dabkiene, V. 2016. The scope of farms sustainability tools based on FADN Data. Scientific Papers-Series Management Economic Engineering in Agriculture and Rural Development 16(1):121-128.

Dantsis, T., C. Douma, C. Giourga, A. Loumou, and E. A. Polychronaki. 2010. A methodological approach to assess and compare the sustainability level of agricultural plant production systems. Ecological Indicators 10(2):256-263.

Darnhofer, I., T. Lindenthal, R. Bartel-Kratochvil, and W. Zollitsch. 2010. Conventionalisation of organic farming practices: from structural criteria towards an assessment based on organic principles. A review. Agronomy for Sustainable Development 30(1):67-81.

de Groot, R. S. 1992. Functions of nature: evaluation of nature in environmental planning, management and decision making. Wolters-Noordhoff BV, The Netherlands. ISBN 9001355943.

de Olde, E. M., E. A. M. Bokkers, and I. J. M. de Boer. 2017a. The choice of the sustainability assessment tool matters: differences in thematic scope and assessment results. Ecological Economics 136:77-85.

de Olde, E. M., H. Moller, F. Marchand, R. W. McDowell, C. J. MacLeod, M. Sautier, S. Halloy, A. Barber, J. Benge, C. Bockstaller, E. A. M. Bokkers, I. J. M. de Boer, K. A. Legun, I. Le Quellec, C. Merfield, F. W. Oudshoorn, J. Reid, C. Schader, E. Szymanski, C. A. G. Sørensen, J. Whitehead, and J. Manhire. 2017b. When experts disagree: the need to rethink indicator selection for assessing sustainability of agriculture. Environment, Development and Sustainability 19(4):1327-1342.

de Olde, E. M., F. W. Oudshoorn, E. A. M. Bokkers, A. Stubsgaard, C. A. G. Sorensen, and I. J. M. de Boer. 2016b. Assessing the sustainability performance of organic farms in Denmark. Sustainability 8(9):957.

de Olde, E. M., F. W. Oudshoorn, C. A. G. Sorensen, E. A. M. Bokkers, and I. J. M. de Boer. 2016a. Assessing sustainability at farm-level: lessons learned from a comparison of tools in practice. Ecological Indicators 66:391-404.

de Olde, E. M., M. Sautier, and J. Whitehead. 2018. Comprehensiveness or implementation: challenges in translating farm-level sustainability assessments into action for sustainable development. Ecological Indicators 85:1107-1112.

De Schutter, O., and G. Vanloqueren. 2011. The new green revolution: how twenty-first-century science can feed the world. Solutions 2(4):33-44.

Dendoncker, N., F. Boeraeve, E. Crouzat, M. Dufrêne, A. König, and C. Barnaud. 2018. How can integrated valuation of ecosystem services help understanding and steering agroecological transitions? Ecology and Society 23(1):12.

Fernandes, L. A. d. O., and P. J. Woodhouse. 2008. Family farm sustainability in southern Brazil: an application of agri-environmental indicators. Ecological Economics 66(2-3):243-257.

Flores, C. C., and S. J. Sarandon. 2004. Limitations of neoclassical economics for evaluating sustainability of agricultural systems: comparing organic and conventional systems. Journal of Sustainable Agriculture 24(2):77-91.

Food and Agriculture Organization (FAO). 2014. Sustainability assessment of food and agriculture systems (SAFA) guidelines. Version 3.0. FAO, Rome, Italy.

Food and Agriculture Organization (FAO). 2018a. Scaling up agroecology initiative. Transforming food and agricultural systems in support of the SDGs. Proposal for the 2nd International Symposium on Agroecology 3-5 April. FAO, Rome, Italy. [online] URL:

Food and Agriculture Organization (FAO). 2018b. The 10 elements of agroecology. Guiding the transition to sustainable food and agricultural systems. FAO, Rome, Italy.

Food and Agriculture Organization (FAO). 2019. TAPE tool for agroecology performance evaluation 2019: process of development and guidelines for application. Test version. FAO, Rome, Italy.

Gasparatos, A. 2010. Embedded value systems in sustainability assessment tools and their implications. Journal of Environmental Management 91(8):1613-1622.

Gasparatos, A., M. El-Haram, and M. Horner. 2008. A critical review of reductionist approaches for assessing the progress towards sustainability. Environmental Impact Assessment Review 28(4-5):286-311.

Gerrard, C. L., L. G. Smith, B. Pearce, S. Padel, R. Hitchings, M. Measures, and N. Cooper. 2012. Public goods and farming. Pages 1-22 in E. Lichtfouse, editor. Farming for food and water security. Springer, Dordrecht, The Netherlands.

Gasparatos, A., and A. Scolobig. 2012. Choosing the most appropriate sustainability assessment tool. Ecological Economics 80:1-7.

Gliessman, S. R. 2014. Agroecology: the ecology of sustainable food systems. Third edition. CRC Press, Taylor & Francis Group, New York, New York, USA. ISBN 978-1-4398-9561-0.

Gómez-Limón, J. A., and G. Sanchez-Fernandez. 2010. Empirical evaluation of agricultural sustainability using composite indicators. Ecological Economics 69(5):1062-1075.

Goswami, R., S. Saha, and P. Dasgupta. 2017. Sustainability assessment of smallholder farms in developing countries. Agroecology and Sustainable Food Systems 41(5):546-569.

Groupe de Travail sur les Transitions agro-écologiques (GTAE). 2018. Agroecology: evaluation methods for its effects and conditions for development. In S. Berton, P. Burger, J.-B. Cheneval, L. Levard, and B. Mathieu, editors. Proceedings of the Exchange and methodological construction workshop, 14-15 December, Paris, France. Le Groupe de Travail sur les Transitions Agro-écologiques, Lyon, France.

Guzmán, G. I., E. Aguilera, R. García-Ruiz, E. Torremocha, D. Soto-Fernández, J. Infante-Amate, and M. González de Molina. 2018. The agrarian metabolism as a tool for assessing agrarian sustainability, and its application to Spanish agriculture (1960-2008). Ecology and Society 23(1):2.

Häni, F., F. Braga, A. Stämpfli, T. Keller, M. Fischer, and H. Porsche. 2003. RISE, a tool for holistic sustainability assessment at the farm level. International Food and Agribusiness Management Review 6(4):78-90.

Hatt, S., S. Artu, D. Bredart, L. Lassois, F. Francis, E. Haubruge, S. Garre, P. M. Stassart, M. Dufrene, A. Monty, and F. Boeraeve. 2016. Towards sustainable food systems: the concept of agroecology and how it questions current research practices. A review. Biotechnologie Agronomie Societe Et Environnement 20(1):215-224.

Hayati, D., Z. Ranjbar, and E. Karami. 2010. Measuring agricultural sustainability. Pages 73-100 in E. Lichtfouse, editor. Biodiversity, biofuels, agroforestry and conservation agriculture. Sustainable agriculture reviews, Volume 5. Springer, Dordrecht, The Netherlands.

Herrera, B., M. Gerster-Bentaya, and A. Knierim. 2016. Stakeholders’ perceptions of sustainability measurement at farm level. Studies in Agricultural Economics 118(3):131-137.

High Level Panel of Experts on Food Security and Nutrition (HLPE). 2019. Agroecological an other innovative approaches for sustainable agriculture and food systems that enhance food security and nutrition. High Level Panel of Experts on Food Security and Nutrition of the Committee on World Food Security, Rome, Italy.

Hodbod, J., O. Barreteau, C. Allen, and D. Magda. 2016. Managing adaptively for multifunctionality in agricultural systems. Journal of Environmental Management 183(2):379-388.

Holt-Giménez, E., and M. A. Altieri. 2013. Agroecology, food sovereignty, and the new green revolution. Agroecology and Sustainable Food Systems 37(1):90-102.

International Assessment of Agricultural Knowledge, Science and Technology for Development (IAASTD). 2009. Agriculture at a crossroads: global report. B. D. McIntyre, H. R. Herren, J. Wakhungu, and R. T. Watson, editors. Island, Washington, D.C., USA.

International Panel of Experts on Sustainable Food Systems (IPES-Food). 2016. From uniformity to diversity: a paradigm shift from industrial agriculture to diversified agroecological systems. N. Jacobs and A. Demonceaux, editors. IPES-Food, Brussels, Belgium.

Latruffe, L., A. Diazabakana, C. Bockstaller, Y. Desjeux, J. Finn, E. Kelly, M. Ryan, and S. Uthes. 2016. Measurement of sustainability in agriculture: a review of indicators. Studies in Agricultural Economics 118(3):123-130.

Levidow, L., M. Pimbert, and G. Vanloqueren. 2014. Agroecological research: conforming-or transforming the dominant agro-food regime? Agroecology and Sustainable Food Systems 38(10):1127-1155.

López-Ridaura, S., H. V. Keulen, M. K. v. Ittersum, and P. A. Leffelaar. 2005. Multiscale methodological framework to derive criteria and indicators for sustainability evaluation of peasant natural resource management systems. Environment, Development and Sustainability 7(1):51-69.

López-Ridaura, S., O. Masera, and M. Astier. 2002. Evaluating the sustainability of complex socio-environmental systems. The MESMIS framework. Ecological indicators 2(1-2):135-148.

Lovell, S. T., S. DeSantis, C. A. Nathan, M. B. Olson, V. E. Méndez, H. C. Kominami, D. L. Erickson, K. S. Morris, and W. B. Morris. 2010. Integrating agroecology and landscape multifunctionality in Vermont: an evolving framework to evaluate the design of agroecosystems. Agricultural Systems 103(5):327-341.

Marchand, F., L. Debruyne, L. Triste, C. Gerrard, S. Padel, and L. Lauwers. 2014. Key characteristics for tool choice in indicator-based sustainability assessment at farm level. Ecology and Society 19(3):46.

Mari, F. J. 2017. Die heile Welt der Standards. Über die nächste Kolonialisierung der Landwirtschaft in Entwicklungsländern. In M. Schneider, A. Fink-Kessler, F. Stodieck, J. Dettmer, C. Gabriel, E. Müller, and B. Voss, editors. Der Kritischer Agrarbericht 2017. ABL Bauernblatt Verlags-GmbH, Konstanz/Hamm, Deutschland.

Martín-López, B., E. Gómez-Baggethun, M. García-Llorente, and C. Montes. 2014. Trade-offs across value-domains in ecosystem services assessment. Ecological Indicators 37:220-228.

Méndez, V. E., C. M. Bacon, and R. Cohen. 2013. Agroecology as a transdisciplinary, participatory, and action-oriented approach. Agroecology and Sustainable Food Systems 37(1):3-18.

Meul, M., S. Van Passel, F. Nevens, J. Dessein, E. Rogge, A. Mulier, and A. Van Hauwermeiren. 2008. MOTIFS: a monitoring tool for integrated farm sustainability. Agronomy for Sustainable Development 28:321-332.

Minae, S., D. Baker, and J. Dixon. 2008. Status of farm data systems and farmer decision support in Sub-Saharan Africa. Food and Agriculture Organization of the United Nations (FAO), Sub-regional Office for Southern and East Africa, Harare, Zimbabwe. [online] URL:

Monzote, F. R. F., R. Bello, A. Alvarez, A. Hernández, E. A. Lantinga, and H. van Keulen. 2012. Identifying agroecological mixed farming strategies for local conditions in San Antonio de Los Baños, Cuba. International Journal of Agricultural Sustainability 10(3):208-229.

Mullender, S., L. Smith, and S. Padel. 2017. Sustainability assessment: the need for convergence. The Organic Research Center, Cirencester, UK and Sustainable Food Trust, Bristol, UK.

Muller, A., M. Steffens, H.-M. Krause, L. Bautze, M. Meier, and S. Stöckli. 2017. What can organic farming contribute. Rural 21: The International Journal for Rural Development 51(4):20-22.

Pimbert, M. 2015. Agroecology as an alternative vision to conventional development and climate-smart agriculture. Development 58(2-3):286-298.

Pintér, L. 2007. A strategic approach to influencing agricultural policy and practice through measurement. Pages 19-24 in F. J. Häni, L. Pintér, and H. R. Herren, editors. From common principles to common practice. Proceedings of the 1st Symposium of the International Forum on Assessing Sustainability in Agriculture (INFASA), 16 March 2006, Bern, Switzerland. International Institute for Sustainable Development and Swiss College of Agriculture, Winnipeg, Manitoba, Canada. [online] URL:

Pope, J., D. Annandale, and A. Morrison-Saunders. 2004. Conceptualising sustainability assessment. Environmental Impact Assessment Review 24(6):595-616.

Rametsteiner, E., H. Pülzl, J. Alkan-Olsson, and P. Frederiksen. 2011. Sustainability indicator development—science or political negotiation? Ecological Indicators 11(1):61-70.

Rasmussen, L. V., R. Bierbaum, J. A. Oldekop, and A. Agrawal. 2017. Bridging the practitioner-researcher divide: indicators to track environmental, economic, and sociocultural sustainability of agricultural commodity production. Global Environmental Change 42:33-46.

Reig-Martínez, E., J. A. Gómez-Limón, and A. J. Picazo-Tadeo. 2011. Ranking farms with a composite indicator of sustainability. Agricultural Economics 42(5):561-575.

Renting, H., W. A. H. Rossing, J. C. J. Groot, J. D. Van der Ploeg, C. Laurent, D. Perraud, D. J. Stobbelaar, and M. K. Van Ittersum. 2009. Exploring multifunctional agriculture. A review of conceptual approaches and prospects for an integrative transitional framework. Journal of Environmental Management 90:112-123.

Rickerl, D., C. Francis, S. R. Gliessman, C. I. Nicholls, M. A. Altieri, R. R. Janke, T. L. Dobbs, C. Buttler-Flora, T. E. Schumacher, R. M. Caldwell, L. Salomonsson, G. Lieblein, J. Helenius, and F. Kirschenmann. 2004. Agroecosystems analysis. American Society of Agronomy, Inc., Crop Science Society of America, Inc., and Soil Science Society of America, Inc., Madison, Wisconsin, USA.

Russillo, A., and L. Pintér. 2009. Linking farm-level measurement systems to environmental sustainability outcomes: challenges and ways forward. International Institute for Sustainable Development, Winnipeg, Manitoba, Canada.

Ryan, M., T. Hennessy, C. Buckley, E. J. Dillon, T. Donnellan, K. Hanrahan, and B. Moran. 2016. Developing farm-level sustainability indicators for Ireland using the Teagasc National Farm Survey. Irish Journal of Agricultural and Food Research 55(2):112-125.

Schader, C., J. Grenz, M. S. Meier, and M. Stolze. 2014. Scope and precision of sustainability assessment approaches to food systems. Ecology and Society 19(3):42.

Schindler, J., F. Graef, and H. J. König. 2015. Methods to assess farming sustainability in developing countries. A review. Agronomy for Sustainable Development 35(3):1043-1057.

Slätmo, E., K. Fischer, and E. Roos. 2017. The framing of sustainability in sustainability assessment frameworks for agriculture. Sociologia Ruralis 57(3):378-395.

Speelman, E. N., S. López-Ridaura, N. A. Colomer, M. Astier, and O. R. Masera. 2007. Ten years of sustainability evaluation using the MESMIS framework: lessons learned from its application in 28 Latin American case studies. International Journal of Sustainable Development & World Ecology 14(4):345-361.

Sukhdev, P., P. May, and A. Müller. 2016. Fix food metrics. Nature 540(7631):33-34.

Šūmane, S., I. Kunda, K. Knickel, A. Strauss, T. Tisenkopfs, I. des los Rios, M. Rivera, T. Chebach, and A. Ashkenazy. 2018. Local and farmers’ knowledge matters! How integrating informal and formal knowledge enhances sustainable and resilient agriculture. Journal of Rural Studies 59:232-241.

The Economics of Ecosystems and Biodiversity (TEEB). 2018. Measuring what matters in agriculture and food systems: a synthesis of the results and recommendations of TEEB for agriculture and food’s scientific and economic foundations Report. A. Müller, P. Sukhdev, and S. O'Neil, editors. United Nations Environment Programme, The Economics of Ecosystems and Biodiversity, TEEB for Agriculture & Food (TEEBAgriFood), Geneva, Switzerland.

The Integrated Assessment Society (TIAS). 2020. Integrated assessment. TIAS, Berlin, Germany. [online] URL:

Trabelsi, M., E. Mandart, P. Grusse, and J.-P. Bord. 2016. How to measure the agroecological performance of farming in order to assist with the transition process. Environmental Science and Pollution Research 23(1):139-156.

Trabelsi, M., E. Mandart, P. Le Grusse, and J.-P. Bord. 2019. ESSIMAGE: a tool for the assessment of the agroecological performance of agricultural production systems. Environmental Science and Pollution Research 26(9):9257-9280.

van Asselt, E. D., L. G. J. van Bussel, H. van der Voet, G. van der Heijden, S. O. Tromp, H. Rijgersberg, F. van Evert, C. P. A. van Wagenberg, and H. J. van der Fels-Klerx. 2014. A protocol for evaluating the sustainability of agri-food production systems—a case study on potato production in pen-urban agriculture in The Netherlands. Ecological Indicators 43:315-321.

Van Cauwenbergh, N., K. Biala, C. Bielders, V. Brouckaert, L. Franchois, V. Garcia Cidad, M. Hermy, E. Mathijs, B. Muys, J. Reijnders, X. Sauvenier, J. Valckx, M. Vanclooster, B. Van der Veken, E. Wauters, and A. Peeters. 2007. SAFE—A hierarchical framework for assessing the sustainability of agricultural systems. Agriculture, Ecosystems & Environment 120(2-4):229-242.

Van Huylenbroeck, G., V. Vandermeulen, E. Mettepenningen, and A. Verspecht. 2007. Multifunctionality of agriculture: a review of definitions, evidence and instruments. Living Reviews in Landscape Research 1(3):1-43.

Zahm, F., P. Viaux, L. Vilain, P. Girardin, and C. Mouchet. 2008. Assessing farm sustainability with the IDEA method—from the concept of agriculture sustainability to case studies on farms. Sustainable Development 16(4):271-281.

Address of Correspondent:
Angelika Hilbeck
Institut of Integrative Biology
CHN G 26.1
Universitätstrasse 16
8092 Zürich
Jump to top
Table1  | Table2  | Table3  | Table4  | Figure1