January 16 2014 — The Productivity Commission this week released a staff working paper that examines methods available for assessing costs and benefits for environmental values. At 145 pages, it’s not a lightweight read but it makes some important points about the methods used to guide policy and development decisions.
So what’s it worth knowing there are endangered frogs in Kakadu? And could you put a dollar value on that? This is the issue which confronts decision makers and stakeholders when assessing the costs and benefits of any major policy or development decision where there is an environmental value at stake.
The Productivity Commission has just released Environmental Policy Analysis: A Guide to Non Market Valuation, a staff working paper that investigates the workings of the tradeoff between environmental benefits and costs, and how things that cannot be easily measured in dollars can best be given a recognisable value (a quick note – staff working papers are not signed off by the commissioners but are used as supporting research for future work commissioned by the government.)
The problem in a nutshell
It is taken as a baseline understanding in government policy that if the environment is to be protected, the community must therefore pay for it. Where there is no actual up-front cost, for example not proceeding with an activity like fracking in an aquifer, the “cost” is imputed through such things as income foregone (the “opportunity cost”) and royalties not received.
Where there is an actual cost, such as repairing the legacy of past pollution, the figures are more straightforward, and yet again must be borne by the community. In both scenarios, jobs and economic development are categorised as a competing interest with environmental protection, based on a fundamental assumption that we cannot always have both.
This analytical framework therefore creates a situation where a value must be assigned to environmental benefits and costs in order to effectively carry out cost–benefit analyses to inform policy decisions.
“Governments are often faced with decisions about whether to impose costs on the community to improve the condition of the environment (or prevent its deterioration),” write authors Rick Baker and Brad Ruting. “How such policy tradeoffs should be made is a matter of considerable debate. Some stakeholders favour prioritising environmental outcomes above other considerations, while others argue that jobs and economic development should come first. The former approach effectively assigns an infinite value to environmental outcomes, while the latter assigns a value of zero.”
How do you value the air?
The working paper provides an up-to-date guide on valuing environmental outcomes, and gives an overview of the various methods that have been used (with varying degrees of success) to attempt to put a clear value on environmental costs and benefits including amenity, water quality, air quality, endangered species and native forests.
The pros and cons of each method are examined, with Baker and Ruter integrating extensive academic research and case studies into their comprehensive analysis.
One case study example that shows a more clear-cut application of cost-benefit assessment methods is the decision to protect New York’s main water catchment.
“Historically, the Catskills catchments supplied New York City with high quality water with little contamination due to the natural filtration processes of the ecosystems on the banks of streams, rivers, lakes and reservoirs. However, increasing housing development and pollution from vehicles and agriculture threatened water quality in the region. By 1996, New York City faced a choice: either it could build water filtration systems to clean its water supply or the city could protect the Catskills catchments to ensure high-quality drinking water.
“A decision was taken to protect the Catskills catchments. It was estimated that the total cost of building and operating a filtration system was in the range of US$6 to US$8 billion. In comparison, the total cost for protecting the water provision service of the Catskills through land purchases and regulations to control development and land use in the catchments was estimated at US$1 to $1.5 billion. The catchment protection option also produced nonmarket environmental benefits, but quantifying them was unnecessary for reaching a decision.”
However, as the authors point out, many cases are not so clear-cut, as they involve places or issues remote from the individual’s ordinary experience.
Baker and Rutling conclude there are methods for non-market assessment that can, if handled effectively, be used to inform policy and project assessment.
The two basic types of methods they outline are revealed preference and stated preference methods, and they give a thorough explanation of the key features required for a sound valuation study for cost–benefit assessment purposes.
So which works better?
Highlights from the report include:
The validity of revealed preference methods is widely accepted, but there are many circumstances where they cannot provide the estimates needed for environmental policy analysis. Stated preference methods can be used to estimate virtually all types of environmental values, but their validity is more contentious.
The evidence suggests that stated preference methods are able to provide valid estimates for use in environmental policy analysis. However:
- there are many elements that practitioners need to get right to produce meaningful results
- value estimates are likely to be less reliable when respondents are asked about environmental assets that are especially complex or relatively unfamiliar to them
Benefit transfer involves applying available value estimates to new contexts. Its accuracy is likely to be low unless the primary studies are of high quality and relate to similar environmental and policy contexts. These seemingly obvious cautions are often not observed.
Because nonmarket valuation methods can generally provide objective estimate of the value that the community places on environmental outcomes, they offer advantages over other approaches to factoring these outcomes into policy analysis.
The case for using nonmarket valuation varies according to circumstances. It is likely to be strongest where the financial or environmental stakes are high and there is potential for environmental outcomes to influence policy decisions.
Where nonmarket valuation estimates are made they should generally be included in a cost–benefit analysis. Sensitivity analysis should be provided, as well as descriptive information about the environmental outcomes of the proposed policy.
How non-market valuation can improve decision-making
According to Baker and Ruting, there are several barriers to nonmarket valuation achieving its potential to improve environmental policy. One important barrier is that a cost–benefit framework is often not applied. Where this occurs, nonmarket valuation is unlikely to gain traction. Concepts such as sustainability and the precautionary principle understandably play a major role, and it is often thought that these are incompatible with applying a cost–benefit framework. But the extent of any incompatibility is unclear. For example, some interpretations of the precautionary principle are fully consistent with cost–benefit analysis, while others are not. Greater guidance on how to apply these concepts could help to resolve these issues.
There are also proactive steps that could be taken to realise more fully the potential of nonmarket valuation, including:
- paying greater attention to the quality of studies and developing a more widespread understanding of what constitutes a highquality study
- better aligning the research effort into nonmarket valuation with policy needs, including building up a bank of value estimates to support benefit transfer
- developing greater knowledge about nonmarket valuation within relevant government agencies
Consultation with stakeholders is important for informing judgments. However, the absence of nonmarket value estimates means that the decision maker may not be informed in an objective and unbiased way about the strength of preferences across the community. This brings the danger that decisions may be unduly influenced by lobbying from prominent stakeholders. The cost of a poor decision should be weighed up against the cost of a nonmarket valuation study.
Even when nonmarket values are not estimated, it would seem worthwhile to “push” scientists and engineers to do this. Such a description would ideally include quantitative elements, such as the area of native vegetation that would be expected to be in an improved condition as a result of a policy, and a measure of the degree of improvement (relative to what would have occurred in the absence of the policy).
Models currently used for valuation
Some critics of non-market valuation methods advocate the use of valuation by scientific and/or policy experts (Hausman 2012). In practice, experts (such as environmental scientists and environmental managers) often do make environmental policy decisions based explicitly or implicitly on their own valuations. Sometimes this is done using analytical tools and models that incorporate values (or weights). For example:
- multicriteria analysis has been used in Queensland, Western Australia and some other jurisdictions to weighup environmental and other outcomes of policy options (a process that can effectively place an implicit value on nonmarket outcomes)
- the Investment Framework for Environmental Resources (INFFER) has been used by a number of Catchment Management Authorities and other regional bodies in several states to prioritise environmental investments
- Marxan is a decision support tool that has been applied to a range of conservation planning problems in Australia and elsewhere (for example, it was used to assist in the rezoning of the Great Barrier Reef)
- EnSym is a decision support tool designed to help prioritise natural resource investment, which has been used by the Victorian Department of Environment and Primary Industries (DEPI) to evaluate the relative cost effectiveness of bids in environmental tenders, and other policies and programs
- NaturePrint is a model that integrates and analyses information about biodiversity values, threatening processes and ecosystem function that is used by DEPI to evaluate the relative biodiversity value of locations across Victoria
Case Study: NaturePrint
NaturePrint has been incorporated into Victoria’s native vegetation clearing regulations and it appears that it has enabled the regulations to more cost effectively contribute to the Victorian Government’s biodiversity conservation objectives (PC 2013). One way that it does this is by allowing environmental offsets to be determined on a “value for biodiversity” basis rather requiring offsets to be of the same vegetation type. This can substantially reduce the cost to developers of providing offsets, without compromising biodiversity outcomes.
Case study: Multi criteria analysis (MCA)
MCA is an alternative to cost–benefit analysis that is often used where non market outcomes are important. MCA is simpler to apply than cost–benefit analysis. There are many variants of MCA, but it usually involves defining policy objectives, determining a set of criteria to measure performance against each objective and assigning weights to criteria. Typically, some criteria relate to market factors (such as the cost of funding a project) and some to non market outcomes. Each policy or project option is given a score for each criterion and these are weighted and added up to give an overall score.
A range of analysts have argued that MCA is seriously flawed (Dobes and Bennett 2009; Pannell et al. 2013). Criticisms include that:
- while a major motivation for choosing MCA is to avoid assigning dollar values to environmental outcomes, the method usually does implicitly assign dollar values
- implicit values from MCA are a consequence of the framing of the policy problem and the way that a particular MCA is done, meaning that two analyses may produce very different values for the same outcome
- the adding up of weighted scores can lead to errors because there are situations where they should be multiplied (for example, “benefit if successful” should be multiplied by, not added to, “probability of success”)
- implicit values are usually determined by a single individual or small expert group, and therefore do not represent community preferences.
Case Study: Investment Framework for Environmental Resources (INFFER)
INFFER is a tool for developing and prioritising projects to address environmental issues, such as reduced water quality, biodiversity, and land degradation. Like MCA, it is designed to be simpler to use than cost–benefit analysis. However, unlike MCA, it is based on the principles of cost–benefit analysis (Pannell et al. 2012).
INFFER involves scoring the value of the environmental asset in question relative to a table of well known environmental assets. This score is converted to a dollar value, based on estimates of the value of the well known assets. A range of other inputs are then used to estimate the change in value expected to result from the project. These include estimates of the impact of the project on the asset’s value (if successful) and information on various types of risks, including those related to technical feasibility and adoption of desired practices by landowners. The change in value is used in calculating the benefit–cost ratio of projects.
Like MCA, INFFER is open to criticism because it uses values that are not based on community preferences (although informal means, such as community workshops, have sometimes been used to inform value estimates). However, it has a range of advantages compared to MCA, including that it avoids logical errors in determining project rankings and explicitly considers relevant sources of risk.
People (and environmental values) are complicated
As Baker and Ruting state, there is potential for the relative environmental values set by experts to depart substantially from those of the community. For example, the community may place a substantially higher value on an environmental asset that is close to a population centre because of the recreational opportunities it provides, but this proximity may not be factored into expert valuations that focus on environmental condition. This potential may be lessened to some degree through informal approaches to factoring community preferences into expert valuations. For example, some applications of INFFER have used community workshops for this purpose (Pannell et al. 2012).
Overall, there are both advantages and disadvantages in using expert valuation rather than nonmarket valuation in costeffectiveness analysis of environmental policies. Experts may be able to develop consistent sets of values at relatively low cost compared to using nonmarket valuation, but these values may depart significantly from those held by the community. At present, there are many circumstances in which it is not feasible (for cost reasons) to use nonmarket valuation, and so expert valuation has a role to play. In time, if a strategic approach was taken to conducting nonmarket valuation studies so as to support benefit transfer, this role might be diminished (some steps have already been taken in this direction — for example, by van Bueren and Bennett (2004) and Greyling and Bennett (2012)).
Whether or not expert valuation is used, scientific and other expertise is essential for predicting the environmental and other outcomes of policy proposals. This involves specifying the likely condition of environmental assets over time, with and without the proposed policy. In doing this, policy risks need to be considered — for example, INFFER pays particular attention to various risks that, if not accounted for properly, could cause outcomes to be greatly overstated. Careful estimation of outcomes is an important complement to nonmarket valuation. There would be little point in devoting considerable time and expense to obtaining a robust nonmarket value estimate that is calibrated to a substandard outcome estimate, which might be wrong by several orders of magnitude.
The potential of the process
Baker and Ruting conclude that nonmarket valuation should be used in combination with good practice policy principles, such as those set out in the Australian Government’s Best Practice Regulation Handbook. For example, valuation should only be used in situations where a sound reason for considering government action (such as the existence of market failure) has been established. Where nonmarket valuation estimates are made they should generally be included in a cost–benefit analysis. The likely accuracy of all components of the analysis should be explained and sensitivity analysis used to demonstrate how the results change under alternative assumptions. It is important to describe the nonmarket outcomes (what the policy would achieve relative to what would have occurred in its absence) as well as providing their estimated value.
Cost–benefit analysis is an information aid to decision making, not a substitute for it. The analysis needs to be presented clearly to allow for proper scrutiny, including of the basis for nonmarket valuation estimates.