Valeria Fanghella, Matteo Ploner and MassimoTavoni

Energy saving in a simulated environment: An online experiment of the interplay between nudges and financial incentives

Journal of Behavioral and Experimental Economics , Volume 93, 101709 - August 2021

Though nudges are gaining attention as complements to financial incentives, evidence of the interplay between these two policy instruments is lacking. Here, we discuss and evaluate how combinations of financial policies and nudges affect behaviors. Through a framed online experiment, we assess the effect of combining financial incentives (monetary reward) with nudges (goal setting and feedback). We introduce an innovative incentive-compatible energy-saving task: participants optimize their virtual energy consumption on a simulated washing machine. Our findings do not show evidence of synergies between traditional and behavioral interventions. On the contrary, the nudge seems to divert participants’ attention from the financial incentive.

DOI   PDF  

Jacopo Bonan, Pietro Battiston, Jaimie Bleck, Philippe LeMay-Boucher, Stefano Pareglio, Bassirou Sarr, MassimoTavoni

Social interaction and technology adoption: Experimental evidence from improved cookstoves in Mali

World Development , Volume 144, 105467 - August 2021

Easy-to-use and low-risk technologies, which require little investment and potentially provide health and environmental benefits, often have low adoption rates. Using a randomized experiment in urban Mali, we assess the impact of a training session in which information on an improved cookstove (ICS) is provided along with the opportunity to purchase the product at the market price. We find strong effects from our invitation to the session on ICS ownership and usage while no discernible effects on product knowledge or household welfare. We find that some diffusion occurs beyond the intervention and provide evidence on the role of social interaction, mostly through imitation.

DOI  

Jacopo Bonan, Cristina Cattaneo, Giovanna d’Adda and MassimoTavoni

Can social information programs be more effective? The role of environmental identity for energy conservation

Journal of Environmental Economics and Management , Volume 108, 102467 - July 2021

Social information programs are widely used to nudge behavioural change. Their effectiveness strongly depends on household and individual traits. The existing evidence in economics and psychology points to the role of environmental values and identity in determining pro-environmental behavior and the impact of social information. In a large field experiment on household energy conservation, we combine electricity metering and survey data to test whether the impact of a social information program can be strengthened by leveraging environmental values and identity. We experimentally augment social information messages with an environmental self-identity prime. The self-identity prime does not strengthen the effectiveness of a social information program on average. Nonetheless, we find suggestive evidence that priming environmental self-identity can be effective if targeted to specific sub-groups.

DOI  

Jacopo Bonan, Cristina Cattaneo, Giovanna D’Adda and Massimo Tavoni

The interaction of descriptive and injunctive social norms in promoting energy conservation

Nature Energy - November 2020

Behavioural interventions that leverage social norms are widely used to foster energy conservation. For instance, home energy reports combine information on others’ behaviour (descriptive feedback) and approval for norm compliant behaviour (injunctive feedback). In a randomized controlled trial, we investigated how descriptive and injunctive feedbacks interact to affect electricity use, and evaluate the effects of additional normative feedback presented in the form of descriptive or injunctive energy conservation norm primes. We found that consistent descriptive and injunctive feedback boosts the effectiveness of social information in inducing energy conservation. When descriptive and injunctive feedback are in conflict, conservation behaviour is a function of the relative strength of the two types of feedback. Additional normative feedback produces smaller gains when it reinforces existing information of the same type. These results suggest complementarities between different types of normative messages rather than superiority of any one kind of feedback.

Giovanna d’Adda, Yu Gao, and Massimo Tavoni

Making energy costs salient can lead to low-efficiency purchases

Under review

We conduct a large-scale field experiment with an online retailer to evaluate the impact of energy cost information on purchasing decisions of large household appliances. We vary the presence of energy cost information and its level of aggregation. We find that increasing the salience of energy costs leads to less efficient purchases - especially when providing lifetime energy costs. Search and navigation data on the retailer’s website confirm that the treatments increase attention paid to inefficient products. This is consistent with customers over-estimating energy savings from high-efficiency products. Results have implications for designing information campaigns aimed at promoting energy efficiency.

PDF  

Oreane Edelenbosch, Davide Rovelli, Antoine Levesque, Giacomo Marangoni and Massimo Tavoni

Long term, cross-country effects of buildings insulation policies

Technological Forecasting and Social Change , Volume 170, 120887, ISSN 0040-1625 - September 2021

Building codes are an effective policy instrument to reduce energy consumption, but their impact depends on local building construction, renovation and demolition cycles, affected by economic and demographic development. In this research a unique global building stock model, with country level detail, is developed to understand the impact of building codes on global energy scenarios. The model shows that the majority of buildings standing in 2050 will be built after 2015, mostly outside of the OECD. In these regions despite growing space cooling demand due to projected economic development, insulation levels of new buildings remain low. New construction policies could thereby have a significant impact. In Africa and China the model shows that if all new buildings would be near zero-energy buildings in 2050 this would save respectively 64% and 43% of space heating and cooling energy demand. In OECD countries, on the contrary, the slower stock turn-over results in renovation policies being more effective, but also more vulnerable to delays. Delaying policy implementation by only 10 years drops global annual emission savings in 2050 by approximately 1 Gt CO2, showing the necessity of a fast and ambitious ramp up of building codes for achieving the Paris climate agreement.

DOI  

Oreane Edelenbosch , Luciana Miu, Julia Sachs, Adam Hawkes and Massimo Tavoni

Calibrating agent based models to real data affects the timing and speed of the energy transition

Under review

The building sector has a high potential to reduce energy consumption. Achieving this depends on household’s choices, which are known to be highly heterogeneous. Agent-based models are tools used to describe energy choices, but require data demanding calibration. Here we combine a novel, cross-country European household-level survey -including socio-demographic characteristics, energy-saving habits, energy-saving investments, and metered household electricity consumption- with a global agent based energy choice model. Cluster analysis reveals that households who demand and consume energy in very similar ways cannot easily be mapped to standardly used socio-demographic classes or attitudes. However, the data also shows interesting patterns both between and within the clusters. Most noticeably, income, consistently, has the largest effect on demand, dwelling efficiency and energy-saving investments. Dwelling improvement potential also incentivizes energy efficiency investments. We use this cluster analysis to calibrate agents of the residential sector of an agent based model, including also the within cluster variations and uncertainty. Including these various sources of heterogeneity affects the timing and speed of the transition, two key indicators in the context of climate change mitigation. The results reinforce the need for grounding agent28 based models in real data, to take real advantage of their capabilities and contribute to a better understanding of energy transition dynamics.

PDF  

Johannes Emmerling, Laurent Drouet, Kaj-Ivar van der Wijst, Detlef Van Vuuren, Valentina Bosetti, and Massimo Tavoni

The role of the discount rate for emission pathways and negative emissions

Environmental Research Letters , Accepted Manuscript online - 20 August 2019

The importance of the discount rate in cost-benefit analysis of long term problems, such as climate change, has been widely acknowledged. However, the choice of the discount rate is hardly discussed when translating policy targets –such as 1.5 and 2°C– into emission reduction strategies with the possibility of overshoot. Integrated assessment models (IAMs) have quantified the sensitivity of low carbon pathways to a series of factors, including economic and population growth, national and international climate policies, and the availability of low carbon technologies, including negative emissions. In this paper we show how and to what extent emission pathways are also influenced by the discount rate. Using both an analytical and a numerical IAM, we demonstrate how discounting affects key mitigation indicators, such as the time when net global emissions reach zero, the amount of carbon budget overshoot, and the carbon price profile. To ensure inter-generational equity and be coherent with cost-benefit analysis normative choices, we suggest that IAMs should use lower discount rates than the ones currently adopted. For a 1000 GtCO2 carbon budget, reducing the discount rate from 5% to 2% would more than double today's carbon price (from 21 to 55 $/tCO2) and more than halve the carbon budget overshoot (from 46% to 16%), corresponding to a reduction of about 300 GtCO2 of net negative emissions over the century.

DOI   PDF  

Oreane Edelenbosch, Valentina Bosetti, Enrica de Cian, Andrea Castelletti, Laurent Drouet, Johannes Emmerling, Matteo Giuliani, Celine Guivarch, Adam Hawkes, Giacomo Marangoni, Jean Francois Mercure Robert Pietzcker, Neil Strachan, Bas van Ruijven, Evelina Trutnevyte, Charlie Wilson and Massimo Tavoni

Behavioral assumptions in global energy models need unpacking

Under review

Energy demand reduction is critical to comply to the Paris Agreement climate policy ambition, downsizing not only direct emissions but also the upstream supply emissions. Such a transition will depend on changed consumer behaviour. Integrated assessment models, widely used to analyse long-term mitigation pathways, tend to focus on supply side solutions. The rules affecting behavioral change are less defined, than the “rational” cost-optimization that are typically used in the supply sectors. We argue that in fact the models do include behavioural mechanisms, but that they are often implicit and not communicated in a transparent way. To understand the role of behavioral change in these long-term pathways, the first step is to unpack these embedded behavioral assumptions, which could morevover provide insights in to the large, across model, demand sector emission ranges. While numerous efforts have been performed to improve transparency on the supply side, this has to date not happened for behavior affecting demand. This perspective takes a first step by discussing key behavioral model assumptions along four dimensions: mechnisms, durability, actor and scope. We emphasize that the evaluation of those behaviors that are impactful, have a society wide effect and can be affected by policy levers, should be prioritzed.

Giovanna d’Adda, Yu Gao, Russell Golman and Massimo Tavoni

It’s So Hot in Here: Information Avoidance, Moral Wiggle Room, and High Air Conditioning Usage

Under review

Environmental policies based on information provision are widespread, but have often proven ineffective. One possible explanation for information’s low effectiveness is that people actively avoid it. We conduct an online field experiment on air conditioning usage to test the theory of moral wiggle room, according to which people avoid information that would compel them to act morally, against the standard theory of information acquisition, and identify conditions under which each theory applies. In the experiment, we observe how exogenously imposing a feeling of moral obligation to reduce air conditioning usage and exploiting natural variation in the cost of doing so, given by outside temperature, influences subjects’ avoidance of information about their energy use impacts on the environment. Moral obligation increases information avoidance when it is hot outside, consistent with the moral wiggle room theory, but decreases it when outside temperature is low. Avoiding information positively correlates with air conditioning usage. These findings provide guidance about tailoring the use of nudges and informational tools to the decision environment.

PDF  

Yu Gao, Giovanna D'adda and Massimo Tavoni

How Long Does It Last -and Why? A Field Experiment on Information Provision and Energy Efficiency in China

Under review

We study the short and long-term eects of information provision on ecient lighting through an online eld experiment in China. Providing information on monetary and environmental benets of LED light bulbs causes an immediate increase in willingness-to-pay (WTP). This eect decays hyperbolically over time: WTP decreases by half after three months, and only 20 percent of the initial impact is left after ten months. However, when information provision leads to knowledge acquisition, WTP for ecient light bulbs doesn't drop over time. Monetary and environmental information have similar impacts, though environmental information is less eective if outdoor air pollution is high.

PDF  

Matteo Fontana, Massimo Tavoni and Simone Vantini

Functional Data Analysis of High-Frequency Load Curves Reveals Drivers of Residential Electricity Consumption

PLoS ONE , 14(6): e0218702 - June 2019

Smart energy meters generate real time, high frequency data which can foster demand management and response of consumers and firms, with potential private and social benefits. However, proper statistical techniques are needed to make sense of this large amount of data and translate them into usable recommendations. Here, we apply Functional Data Analysis (FDA), a novel branch of Statistics that analyses functions to identify drivers of residential electricity load curves. We evaluate a real time feedback intervention which involved about 1000 Italian households for a period of three years. Results of the FDA modelling reveal, for the first time, daytime-indexed patterns of residential electricity consumption which depend on the ownership of specific clusters of electrical appliances and an overall reduction of consumption after the introduction of real time feedback, unrelated to appliance ownership characteristics.

DOI   PDF  

Jacopo Bonan, Pietro Battiston, Jaimie Bleck, Philippe LeMay-Boucher, Stefano Pareglio, Bassirou Sarr and Massimo Tavoni

Social interaction and technology adoption: Experimental evidence from improved cookstoves in Mali

World Development , Volume 144, 105467 - August 2021

Using a randomized experiment, we assess the impact of a training session where information on a more efficient cooking stove (ICS) is provided along with the opportunity to purchase it at market price. We find positive direct and spillover effects of being invited to the session on ICS ownership in 6 to 9 month following our treatment. We also find a positive impact on usage, measured both through self-reporting and with monitoring devices, along the extensive and intensive margin. Our training session appears to have no impact on the knowledge about ICS and any measure of welfare. Within our session, we furthermore randomly assign participants to receive information on a peer’s actual purchase or previous ownership and find that women are more likely to adopt the product if the information they receive is on a peer who purchased the product and whose opinion is respected. This allows us to investigate the various mechanisms of social interaction potentially at play and provide evidence supporting imitation effects, rather than social learning or constraint interaction.

DOI   PDF  

Giovanna d’Adda, Arianna Galliera and Massimo Tavoni

Urgency and engagement: a field experiment of the impact of message content on consumption awareness

Journal of Economic Psychology , Available online, 102275 - 21 April 2020

We study how to foster engagement in the energy sector, a market where signals about consumption are opaque and infrequent. We investigate the impact of an intervention aimed at fostering natural gas self-reading, which allows utilities to bill customers on the basis of their real and not of their estimated consumption. Since utilities all over the world are relying heavily on users to submit their meter readings to achieve billing accuracy, understanding how to engage consumers is an important policy issue. In our study, messages that induce a sense of urgency are two times more effective than the generic messages in encouraging self-readings, consistent with previous research on the urgency effect. Our findings suggest that the increased sense of urgency moves to action customer with both high and low levels of prior engagement, but that the effect on the former is stronger.

DOI   PDF  

Valeria Fanghella, Matteo Ploner and Massimo Tavoni

Nudges and traditional policies: experimental evidence of substitution effects for energy consumption

Under review

Though nudges are gaining attention as complements to traditional policies, evidence of the interplay between these two policy instruments is lacking. Here, we discuss and evaluate how combinations of traditional tools and nudges may affect individuals’ attitude towards energy consumption. Through an online experiment (N=566), we assess the impact of an energy policy mix which combines a traditional instrument (tax rebate) and a nudge (goal setting and feedback). We introduce an innovative incentive-compatible task: participants are asked to optimize their energy consumption on a virtual washing machine. Our findings do not show evidence of positive synergies between the traditional and the behavioral tools. Specifically, adding the economic incentive to the nudge neither strengthens the effect of the latter nor increases its medium-term impact. Our results suggest that a policy mix characterized by multiple stimuli can overwhelm individuals and eventually backfire.

PDF  

Yu Gao, Giovanna d'Adda and Massimo Tavoni

Adopting LEDs changes attitudes towards climate change: experimental evidence from China

Environmental Research Letters , Accepted Manuscript - 29 March 2019

The adoption of low carbon technologies needs to go hand in hand with an increased awareness of climate change and its consequences and solutions. Attitudes toward climate change are influenced by a variety of factors, most notably educational attainment and exposure to climatic events attributable to climate change. However, less is known about the effect of technology adoption on climate change beliefs and support for mitigating measures. Through a longitudinal, incentivized field experiment with Chinese households, we assess attitudes toward climate change before and after adopting efficient lighting technology. The results show differential patterns of attitudinal change: while belief in the reality of climate change and willingness to adopt energy-efficient appliances increase, support for energy taxes does not. We attribute the attitudinal change to the adoption of LED light bulbs. Further evidence suggests that experience with efficient technology, rather than knowledge acquisition, drives this change. These results highlight the importance of action-initiating behavioral intervention to complement educational programs aimed at improving knowledge.

DOI   PDF  

Valeria Fanghella, Giovanna d’Adda and Massimo Tavoni

On the Use of Nudges to Affect Spillovers in Environmental Behaviors

Frontiers in Psychology - 29 January 2019

Environmental self-identity is considered a promising lever to generate positive spillovers across pro-environmental behaviors: existing evidence shows that it is positively correlated with pro-environmental choices and that it can be easily manipulated, by reminding individuals of their past pro-environmental actions. However, it remains unclear whether it can be successfully used for environmental policy making. In two online, incentive-compatible experiments, we manipulate participants’ environmental self-identity and test whether this leads to increased donations to an environmental charity. Additionally, we investigate the interaction between self-identity priming and two commonly used behavioral policy tools: social information (Study 1, N = 400) and goal commitment (Study 2, N = 495). Our results suggest caution in leveraging environmental self-identity to promote pro-environmental behaviors, provide indications on how to target policies based on self-identity primes, and offer novel evidence on the interaction between different behavioral policy tools.

DOI   PDF  

O.Y.Edelenbosch, David L McCollum, Hazel Pettifor, Charlie Wilson and Detlef P. van Vuuren

Interactions between social learning and technological learning in electric vehicle futures

Environmental Research Letters , Volume 13, Number 12 - 23 November 2018

The transition to electric vehicles is an important strategy for reducing greenhouse gas emissions from passenger cars. Modelling future pathways helps identify critical drivers and uncertainties. Global integrated assessment models (IAMs) have been used extensively to analyse climate mitigation policy. IAMs emphasise technological change processes but are largely silent on important social and behavioural dimensions to future technological transitions. Here, we develop a novel conceptual framing and empirical evidence base on social learning processes relevant for vehicle adoption. We then implement this formulation of social learning in IMAGE, a widely-used global IAM. We apply this new modelling approach to analyse how technological learning and social learning interact to influence electric vehicle transition dynamics. We find that technological learning and social learning processes can be mutually reinforcing. Increased electric vehicle market shares can induce technological learning which reduces technology costs while social learning stimulates diffusion from early adopters to more risk-averse adopter groups. In this way, both types of learning process interact to stimulate each other. In the absence of social learning, however, the perceived risks of electric vehicle adoption among later-adopting groups remains prohibitively high. In the absence of technological learning, electric vehicles remain relatively expensive and therefore is only an attractive choice for early adopters. This first-of-its-kind model formulation of both social and technological learning is a significant contribution to improving the behavioural realism of global IAMs. Applying this new modelling approach emphasises the importance of market heterogeneity, real-world consumer decision-making, and social dynamics as well as technology parameters, to understand climate mitigation potentials.

DOI   PDF  

K. Ricke, L. Drouet, K. Caldeira and M. Tavoni

Country-level Social Cost of Carbon

Nature Climate Change , 8 - Sep.2018

The social cost of carbon (SCC) is a commonly employed metric of the expected economic damages from carbon dioxide (CO2) emissions. Although useful in an optimal policy context, a world-level approach obscures the heterogeneous geography of climate damage and vast differences in country-level contributions to the global SCC, as well as climate and socio-economic uncertainties, which are larger at the regional level. Here we estimate country-level contributions to the SCC using recent climate model projections, empirical climate-driven economic damage estimations and socio-economic projections. Central specifications show high global SCC values (median, US$417 per tonne of CO2 (tCO2); 66% confidence intervals, US$177–805 per tCO2) and a country-level SCC that is unequally distributed. However, the relative ranking of countries is robust to different specifications: countries that incur large fractions of the global cost consistently include India, China, Saudi Arabia and the United States.

Felix Creutzig, Joyashree Roy, William F. Lamb, Inês M. L. Azevedo, Wändi Bruine de Bruin, Holger Dalkmann, Oreane Y. Edelenbosch, Frank W. Geels, Arnulf Grubler, Cameron Hepburn, Edgar G. Hertwich, Radhika Khosla, Linus Mattauch, Jan C. Minx, Anjali Ramakrishnan, Narasimha D. Rao, Julia K. Steinberger, Massimo Tavoni, Diana Ürge-Vorsatz & Elke U. Weber

Towards Demand-Side Solutions for Mitigating Climate Change

Nature Climate Change , 8, pages 260–263 - April 2018

Research on climate change mitigation tends to focus on supply-side technology solutions. A better understanding of demand-side solutions is missing. We propose a transdisciplinary approach to identify demand-side climate solutions, investigate their mitigation potential, detail policy measures and assess their implications for well-being.

DOI   PDF  

Jacopo Bonan and Laura Pagani

Junior Farmer Field Schools, Agricultural Knowledge and Spillover Effects: Quasi-Experimental Evidence from Northern Uganda

The Journal of Development Studies , Published online - Sep. 21st 2017

We analyse the impact of a junior farmer field school project in Northern Uganda on students’ agricultural knowledge and practices. We also test for the presence of intergenerational learning spillover within households. We use differences-in-differences estimators with ex-ante matching and find evidence that the programme had positive effects on students’ agricultural knowledge and adoption of good practices. The project also produced spillover effects in terms of improvements of household agricultural knowledge and food security. Overall, our results point to the importance of adapting the basic principles of farmer field schools to children.

DOI   PDF  

Jacopo Bonan, Stefano Pareglio and Massimo Tavoni

Access to modern energy: a review of barriers, drivers and impacts

Environment and Development Economics , 1-26 - July 2017

Universal access to modern energy services, in terms of access to electricity and to modern cooking facilities, has been recognized as a fundamental challenge for development. Despite strong praise for action and the deployment of large-scale electrification programs and improved cookstove (ICS) distribution campaigns, few studies have shed light on the barriers to, the enablers of and the impacts of access to energy on development outcomes, using rigorous methodologies. This paper reviews this recent strand of research, trying to fill these gaps. The authors focus on the demand-side and household perspective. Their main outcomes of interest are electricity connection and ICS adoption for the analysis of barriers, time allocation, labor market outcomes and welfare for the impact analysis. They provide evidence of significant wellbeing impacts of electrification and mixed evidence for cookstoves

DOI   PDF  

Giovanna d'Adda

Relative social status and conformism: Experimental evidence on local public good contributions

Economics Letters , Available online - May 2017

Through an artefactual field experiment conducted in Colombia, where participants make repeated contributions to a local natural conservation project, I test a novel way to identify high status individuals within a community, and show that status, so defined, is correlated with public good giving. Both absolute and relative status have a statistically significant and economically relevant influence on behavior. In particular, the same individual, when matched with a lower status partner, donates more and conforms less to the partner’s action. The results indicate that contributions to local public goods can be enhanced by interventions that make relative status more salient.

DOI   PDF  

Giovanna D'Adda , Valerio Capraro and Massimo Tavoni

Push, Don't Nudge: Behavioral Spillovers and Policy Instruments

Economics Letters , Volume 154, Pages 92-95 - May 2017

Policy interventions are generally evaluated for their direct effectiveness. Little is known about their ability to persist over time and spill across contexts. These latter aspects can reinforce or offset the direct impacts depending on the policy instrument choice. Through an online experiment with 1486 subjects, we compare four widely used policy instruments in terms of their ability to enforce a norm of fairness in the Dictator Game, and to persist over time (i.e., to a subsequent untreated Dictator Game) or spill over to a norm of cooperation (i.e., to a subsequent Prisoner’s Dilemma). As specific policy interventions, we employed two instances of nudges : defaults and social information; and two instances of push measures: rebates and a minimum donation rule. Our results show that (i) rebates, the minimum donation rule and social information have a positive direct effect on fairness, although the effect of social information is only marginally significant, and that (ii) the effect of rebates and the minimum donation rule persists in the second game, but only within the same game type. These findings demonstrate that, within our specific design, push measures are more effective than nudges in promoting fairness.

DOI   PDF  

Johannes Emmerling and Salmai Qari

Car Ownership and Hedonic Adaptation

Journal of Economic Psychology , Available online - 19 March 2017

Using panel data from the UK, we study the long-term effect of purchase decisions of automobiles on individuals’ happiness. We find a significant and sizable decrease in individual happiness in the years after a car purchase. We develop a model of hedonic adaptation that can explain these results. Applying the model to the data indicates a strong degree of habit persistence of around 80%, and that within five years after a purchase, around one third of the happiness increase is dissipated due to hedonic adaptation.

DOI   PDF  

Valentina Bosetti, Elke Weber, Loïc Berger, David V. Budescu, Ning Liu & Massimo Tavoni

COP21 climate negotiators’ responses to climate model forecasts

Nature Climate Change , doi:10.1038/nclimate3208 - 06 Feb. 2017

Policymakers involved in climate change negotiations are key users of climate science. It is therefore vital to understand how to communicate scientific information most effectively to this group1. We tested how a unique sample of policymakers and negotiators at the Paris COP21 conference update their beliefs on year 2100 global mean temperature increases in response to a statistical summary of climate models’ forecasts. We randomized the way information was provided across participants using three different formats similar to those used in Intergovernmental Panel on Climate Change reports2,3. In spite of having received all available relevant scientific information, policymakers adopted such information very conservatively, assigning it less weight than their own prior beliefs. However, providing individual model estimates in addition to the statistical range was more effective in mitigating such inertia. The experiment was repeated with a population of European MBA students who, despite starting from similar priors, reported conditional probabilities closer to the provided models’ forecasts than policymakers. There was also no effect of presentation format in the MBA sample. These results highlight the importance of testing visualization tools directly on the population of interest.

DOI   PDF