Chris Nixon* and John Yeabsley§

5.1. Introduction

Optimising economic activity within an economy requires constant attention as economic circumstances and attitudes to economic growth change.[1] Therefore, efficient and effective regulation of all these activities is one of the major preoccupations of government.

The purpose of this paper is to discuss the tools and techniques available and used to pre-test and then evaluate the efficiency, effectiveness, and overall performance of proposed and implemented regulations. Along the way we: examine the idea of an economic model; show different approaches and when they are best used; and explain why quantitative tools are useful and valuable in the production of practical solutions to real world regulatory issues.

5.1.1 A stylised setting

The development of regulations takes place in a political environment, as that is the mechanism that we have assigned to control the coercive power of the state that underpins public regulation. Earlier work by Dr Mark Prebble shrewdly points out[2] the way that this matters for decision-making, particularly about complicated issues. In his view, politicians effectively act politically, all of the time. Thus, the key environment for deciding about regulations is one that is dominated by the views of politicians, acting politically.

One crucial feature of such an environment is the attitude the key participants have toward risk.[3] In discussing this issue two commentators[4] have summed up a simple view:

... the political market place, unlike its economic counterpart, does not easily tolerate diversified portfolios. After all, this process is in place to make decisions, not to have it both ways.

This intolerance has obvious consequences; it is likely to make outright experimentation difficult without extensive structural pre-arrangement and/ or preparatory consultation to lower expectations. Indeed, the political scene is seemingly characterised by an ideal of decisive action. In a world where risk is unavoidable, and likely to be present in any attempts to change the rules around human behaviour – the nub of regulation - this has implications. In particular, the natural way to cope with risk in general policy interventions would be a judicious mix of pre-testing and post implementation monitoring leading to review and remediation.[5] That way the various types of risk that are relevant in policy implementation can be investigated under different circumstances.

Further, if it takes some considerable time before a policy is known to work since testing via case law[6] is inevitably 'thin' in a small country, policy makers hesitate to judge the last government[7], or the problem is unique to New Zealand; then post-monitoring is unlikely to be politically favoured, as it leaves the question of whether a regulation is effective hanging. In this situation, therefore, the only design ploy naturally available is to place great weight on the quality of the pre-testing.

Without the mandate to extensively pilot and test the intervention in actual operational conditions, regulatory designers need to use as much ingenuity as possible to check its functioning and weaknesses in laboratory conditions – bench trials – typically using models.[8] This type of approach also has the advantage of potentially being able to trial a number of variants without the costs (and publicity) associated with full scale field testing.

The standard form that such bench trials take is to use the analytical resources that economics and associated 'policy' and public management disciplines have built up over the years, as the basis for simulation methods to gain greater insight into the likely value and workings of the proposed regulations.

5.1.2 Analytics

Thus, to craft new regulation and lower its risk level, as well as possibly establishing standards by which to monitor its performance, requires the use of simulation, via models of some kind. These statistical and mathematical approaches attempt to replicate regulatory execution and influence, without going into actual implementation.

Creating the appropriate tools to do this is not easy, and requires: a detailed understanding of the problems that need to be addressed; use of economic (and other) theories that explain stakeholder behaviour; and the use of appropriate data to calibrate the expected effects back to the real world. Combining all three elements to produce an apt analysis in a complex, dynamic world is demanding, particularly when, questions are not well defined, theory only partially assists framework development, and local data is scarce.[9]

Statistical, mathematical, and other analytical approaches can be useful tools to untangle the economic (and other) pressures in operation, since the impact of regulation in many instances cannot be fully understood through 'common sense', casual observation or by 'back of the envelope' calculations. These 'looser' techniques all suffer the same potential failing: they lack a comprehensive framework and thus can overlook significant issues. Without a systematic approach to complex problems and the analysis of the range of options available we cannot make sense of the past, test theories that describe the critical relationships, or attempt realistically to predict future activity.[10] A well-formulated and executed approach can provide insights to shed light on the efficiency and effectiveness of particular regulatory functions.[11]

5.2. Commissioning analytical work

To increase the quality of an intervention requires a systematic understanding of the steps required to achieve the desired outcome. The previous discussion has stressed the difficulty of actually doing this. The problems are the sheer number of potentially significant factors that have to be taken into account, in a setting where the consequences are complicated and the resources typically limited – if not sparse. The variety of settings and challenges rules out a single comprehensive road map. What is required is a mechanism that aids the shrewd practitioner by flagging issues, and areas that have been known to cause problems.

One effective approach that fits this prescription is to develop a checklist. It would support accurate execution of what can be a complex task. According to McLaughlin,[12] the research into checklist development puts emphasis on the human factors that guide behaviour. He cites a number of examples in the aviation, medical, and space research where the use of checklists has dramatically reduced errors to the point where governments have mandated their use in specific instances.

It must be recognised that the application of a checklist here is subtlety different to the run of those discussed in the literature. In most of the cited uses the role is to provide a certainty in process to ensure that all the necessary operational steps are taken. Here the logical function is to ensure that all potential complicating factors have been addressed and examined. In a sense it is being used in a 'meta' way; at a level of analysis one step higher than the normal operational examples.

While it is a valuable mechanism (as it sums accumulated learning to catch the regulation designer's attention), a checklist will not solve all problems. Applying a checklist approach to regulatory matters has the potential to enhance or complicate processes. The critical issue is one of execution: checklist design. Badly-designed approaches can constrain performance as much as well-designed checklists improve efficiency and effectiveness.

As argued above, in regulatory design, reducing risk is of paramount importance. To mitigate risk, depending on the regulation being considered, politicians and policymakers will choose to work somewhere on a continuum between checking the logic of their own intuition and sample testing the possible impact of the proposed regulation.

5.2.1 Steps to analytical support

There are a number of steps required to be addressed when considering what policy analysis/research might be considered to test the problem which regulation is trying to fix.

These include:

1. What are we looking for in a policy intervention framework?

The policy framework should aim to maximise welfare over time with respect to risk and cost. In an ideal world the market takes care of these outcomes. In real world situations, however, market failure can occur which necessitates regulatory intervention.

In general terms, less regulation is preferred to more. This reflects the general starting point of our laws that people should be free to engage in activities unless they are prohibited for some good reason.[13] Also, more specifically, good regulatory design should signal the importance of innovation for economic growth, and the wider goal of maintenance and enhancement of New Zealand's standard of living. This is why, for example, New Zealand has signed up to international commitments such as GATT (WTO) Uruguay Round agreements and FTAs (e.g. with Australia, ASEAN, China and Singapore).

2. Problem definition

The fundamental questions that need to be addressed when designing any regulatory regime are based on simple principles in relation to regulatory policy design. These are reflected in many publications specific to New Zealand conditions and from practical experience in developing and advising on policy in a range of fields.[14] The important questions are: what are the policy objectives, and the problem definition (this is covered in more detail in section 5.3.2 below), and, will the proposed regime advance those objectives, in principle and in practice? And, overall, will the results be economically sound?

3. The politics of the intervention

The politics comes first. The workings of regulation, for all their economic and social impacts, are decided by the politics. The consequences are widespread. Policymakers must first do the political analysis on the problem at hand; this involves a degree of interaction with a spread of advisors and experts, with the decision frame driven by the politics. The political implications will, typically, largely determine the sensitive issues to be worked through as part of the policy development, and as put to politicians, influence the final outcomes. Other factors in which the political influence will be felt will be the scope of commissioned research, especially in terms of the potential reaction to its results, and there will be keen awareness of interested stakeholders with political power. Seasoned advisors and analysts will select appropriate process strategies to deal with these situations.

4. How big is the problem?

Proportionality Policy is not only a practical art, but one that is very much a dynamic process, seeking to reflect the pressures of the current situation. Thus, for instance, the choice of methodologies to gather information and data will not necessarily be standard. While it is important that the evidence for a particular solution or approach is as strong as possible, policymakers are willing to expose themselves to 'evidence error' in order to inform better policy making advice.[15] This is a difficult balancing act between following a particular regulatory approach and assessing the evidence. How much margin of error can be tolerated also needs to be considered. Brookshire[16] sets out a framework (shown in Figure 1) that matches the appropriate strength of evidence to regulatory objectives.

If the objective is to gain more information about a policy or develop an initial assessment then a relatively low level of data or level of evidence is required (i.e. gaining knowledge or developing a pilot). Higher degrees of evidence are required if a national policy decision is being taken, or money is being paid out for damages incurred. In such cases, a compelling assessment that supports any particular approach may be required.

Figure 1 ─ Continuum of decision settings

Source: Brookshire[17], quoted in Pearce et al[18]

However, pursuing this 'proportionate' approach – seeking to match the strength of the backing to the scale of the consequences - depends on an assessment of the state of the evidence, which in itself can be subjective. Care is required – possibly through experience and learning – to judge how much evidence is required given (expected) risks, resources and timeframes. Further, thought needs to be given to how much evaluation might be required to show whether a policy is working in a way that the designers intended.

5. What are the consequences of the problem and what evidence do we have that it exists (including data)?

Does the problem exist or is it just a passing phenomenon – a statistical 'cluster'? What evidence does the policymaker have that it is not such an artefact? What types of solutions are required? That is, does it require ones that change political perceptions, change economic behaviour, or institutional approaches, or a combination of all three?

Policymakers should be clear as to how the problem (which the regulation is attempting to fix) has manifested itself over time and the stakeholders who are affected by the problem. This assists in building the evidence base that can inform the problem definition, reinforce any research undertaken, and deal with the political and institution issues as they may crop up.

6. What needs to be valued?

One of the more potentially challenging practical exercises is valuing the costs and benefits of regulatory changes. These include direct costs for the Government and for participants and indirect costs. The main form of indirect cost is what economists call opportunity cost. That is, if the regime results in some otherwise undertaken valuable activity not being pursued, the loss of these potential contribution is a cost of the regime. Such a situation may occur through target and boundary design or execution problems where the operation of a prohibition catches not only those 'evils' that were intended, but also 'innocent bystanders,' who were not seen as part of the ban. It can also arise where the regime would, in theory, allow the activity, but, in practice, the cost and delay involved in obtaining the necessary permissions is: seen as prohibitive by the promoter; or greater than the cost of doing something else, which is actually less socially valuable, and which is thus done instead; or greater than the cost of doing the proposed regulated activity in another country, under a different regime.

7. What are the alternatives to addressing the issue matched against the chosen approach?

As part of the initial enquiry into the proposed policy, understanding the available set of alternatives is an important step in developing the most cost effective approach to solving the perceived problem, whether to be approached either by the use of regulation or through other means. A typical preliminary method is to compare and contrast the relevance of the alternatives to the preferred regulatory approach, setting out the advantages and disadvantages of each alternative, and any other issues that might impact on addressing the problem.

8. Have the critical assumptions of the chosen intervention been tested?

Crucial to any analytical appraisal is to understand the robustness and likelihood of the assumptions that underpin any regulatory approach. This knowledge gives the policymaker an understanding of the risk and/or uncertainty associated with the preferred option. And informing politicians of the key risks and uncertainties is one of the most important jobs for a policy advisor since politicians make the decisions and bear the consequences. Failure to provide the politicians with an appropriate assessment of the risks can seriously compromise the regulation system as it undermines the division of labour between advisors and politicians, by leaving them unprepared for setbacks. The inevitable result is a loss of trust and this, in turn can lead to scaling back or even cancelation of regulation developments.

9. Can the critical values be monetised? How should we approach the non-economic factors? Is there potential for unacceptable hazards or risks that need to be addressed?

Because the key comparison in public policy is always with resource costs, politicians grow used to dealing with decisions involving money and/or monetised values. Policymakers therefore are comfortable analysing issues where the key items are amenable to being valued in money – preferably via an established market price. It is less straightforward to incorporate matters that are difficult to value e.g. species extinction. While the specifics of the situation can be used to put the decision in usefully stark terms,[19] it may be sufficiently unstructured on other occasions as to lead to faulty design of policy that generates unintended outcomes or creates externalities that impinge upon the durability of the intended regulation. Complicating matters further, costs of regulation are typically easy to put into money, up-front and real, while the benefits can be more defuse and generally take longer to accrue.

While there are well-established methodologies[20] to estimate total economic value, they are (relatively) expensive, may require specialised researchers, and take time, as they usually involve the collection of new data. Because of all of these factors, these methods would not normally be used in forming a typical piece of regulation. Therefore, to improve the assessment process (and by so doing develop more durable regulation), ways need to be found to better reflect non-economic factors in the decision process (e.g. as discussed above, a cost benefit analysis could be inverted.)

10. Another side entails behaviour: how do stakeholders attempt to minimise the impacts of a particular problem in a similar situation?

Estimating the costs stakeholders are prepared to incur to mitigate the problem provides a lower bound to the solution's value (stakeholders' willingness to pay.) A key consideration is that the poorer the stakeholder, the less willing they are to spend on mitigation, therefore, how important are distributional considerations? What intervention characteristics are appropriate? How should catastrophes be analysed?

Understanding the economic size, industry characteristics, political influence and the way business is conducted in the sector is also important in developing effective and efficient policy. How the agents affected by regulation respond – including the way society seems to perceive multiple fatalities (catastrophes)[21] - is an important factor to be considered both in the short and long term.

For example, in British high hazard industries the Health and Safety Executive (HSE)[22] have developed an approach to high hazards that tries to reflect the difficult to parameterise catastrophic risks involved. This reflects the thinking that has been elegantly summed up by Noll: [23]

… the same factors that cause citizens to behave inefficiently in preparing for and responding to disasters also will cause inefficiencies in policies concerning catastrophic events. Among the more important expected pathologies of disaster policies are:

(1) a failure to balance benefits and costs at the margin for different types of disasters, due to cognitive pathologies such as availability and overconfidence;

(2) a tendency for policy to underinvest in protection and overinvest in response; and

(3) a scepticism for policies based on sound insurance principles for spreading costs and assessing claims.

The approach is to create a cost benefit analysis that informs a stakeholder whether further risk reduction is 'reasonably practicable', where 'reasonably practicable' means that costs are appropriate to the benefits.

In other words, as a formula the test is:

Costs/ Benefits > DF

Where DF is the 'disproportionate factor' included to indicate when a risk reduction regulation is not worth implementing. DFs vary from one to ten depending on the assessed risk factors. High hazard industries typically have a DF of ten meaning the costs of a safety improvement have to be ten times the benefits before it can be considered as disproportionate.

11. What sort of trade-offs are needed?

Who is affected can be important when the regulatory costs and or benefits are not homogenous in their affects. Often under new regulations it is possible that some stakeholders will win and others will lose. Understanding the values that are being traded-off in such a case is of crucial importance. This concern passes through into the analysis. Therefore, the policymaker needs to identify the main areas of impact to make decisions about the focus of the quantification. The regime will need to deal with various types of (possibly overlapping) issues, and strike balances between them.

The list includes: technical/scientific issues, environmental issues, cultural and ethical issues, other social issues and economic issues. If one of these aspects is omitted then the potential for unintended consequences increases. For example in trade agreements New Zealand will typically trade a loss of sovereignty for increased economic growth (through further economic integration). For environmental protection, it is widely accepted that the pursuit of economic growth by itself may increase the risks of environmental damage and that some restraints are needed on the types of activities that may be undertaken, and the manner in which they are undertaken.

12. What sort institutional implementation process do we expect? Will the interventions be effective, given who is charged with the task?

This requires an examination of the proposed substantive rules, procedures, and institutions. It involves asking whether the substantive rules accurately capture the policy objectives; and if the institutions and processes involved will, in practice, apply those substantive rules, in an appropriate and timely way, so that the objectives will be actually achieved. It usually also entails probing the realism and adequacy of the monitoring, accountability and sanctions regimes. How will learning occur? And is there a bias to openness and transparency to use public interest to underline the workings of the institutions?

13. Are there outstanding technical and data issues that need to be resolved?

The degree to which technical issues have been pre-solved can impact on the risks and uncertainty associated with the proposed regulation. Unresolved technical issues can lead to flawed approaches since the details of the regulatory approach may depend on precise details of technical specifications. If the technical specifications are inappropriate it can jeopardise the intended performance of the regulation.

14. What other laws impact on the area where intervention is intended?

Understanding how the proposed regulation might impact on or interact with other laws/regulation must be understood for efficient regulation to take place. This may entail a wider system examination.

15. Are the timelines appropriate?

Stakeholders must be given adequate time to respond to any regulatory changes. Holiday times and the end of the financial year should be avoided, particularly where small players are involved.

16. How will the regulation be evaluated?

Detailing how a policy can be evaluated is necessary to understand how the regulatory process is working. Regulation does not happen in a vacuum therefore as the regulatory approach is put together evaluation should be designed. An important feature of evaluation often overlooked is the gathering of data about the situation ex ante – before the regulation comes into force. A panel of respondents may be a useful device. They can be revisited at intervals to provide updates on the regulatory impacts and costs.

This is not a trivial exercise since we expect that an evaluation will pinpoint areas of weakness in the selected policy and set the direction of future regulatory design work.

5.3. Approach to quantification

5.3.1 What is a model?

A model is a description of economic relationships, which although abstract, seeks to capture the salient elements of the real world. There are, of course, many different views about what a model is in economics, but all have the simplification of reality as a core ingredient. Samuelson and Nordhaus[24] describe a model as a formal framework for representing the basic features of a complex system by a few central relationships. Models can take the form of graphs, mathematical equations or as computer programmes. Begg, Fischer, and Dornbusch[25] state that a model or theory makes a series of simplifications from which it deduces how people will behave. It is a deliberate simplification of reality.

Any real world problem will conceptually involve a large number of variables with a large set of, often complex, relationships between them. The practitioner aims to draw out the main points of interest without the full complications of unnecessary issues. In this way, it is hoped to gain meaningful insights into the relevant aspects of the problem (i.e. in this case, the effectiveness and efficiency of regulation). The potential cost of this approach is that in the process of abstraction some important factor is eliminated from the analysis as seemingly irrelevant. Understanding what should be left in and what should be tossed aside in a modelling process requires close coordination between policymakers and those modelling the regulatory responses.

5.3.2 Questions, theory, and data

The building blocks of modelling are questions, theory and data. They also influence the type of model best suited to the task, as it depends on the interaction between the nature of the question(s), the type of theory being applied, and the raw data available.

Ideally, economists should present theories that can shed light on the questions asked and assist in the organisation of data. Without any of these building blocks Leamer explains:[26]

[If] … we forget the questions, and use only theory and data, we often imagine that theories are either true or false, and we fantasize that the data will point clearly toward one or the other logical possibility. Then we engage in pointless formal hypothesis tests, which we routinely ignored if they are successful.

When we forget the theory, and use only the questions and the data, we do economic journalism, which can be interesting but also misleading if the data are organized in a way the is incompatible with accepted theory.

When we forget the questions and the data, we do mathematics not economics.

When we forget the data and use only the questions and theory, we do what economists usually do: we manipulate.

5.3.2.1 Questions

Defining the questions revolves around the problem attributes. A research question greatly assists in understanding how to tackle the problem. Developing this research question will entail consideration of the features of the subject matter, what the project/regulation is aiming to achieve, why is it important to achieve it and how will we know if have got there?

According to Lipowski the characteristics of useful research questions are to:[27]

… generate new insights into old problems … [and] … challenges researchers to see matters from a new perspective and to learn something new.

To develop the triggers for such unique insights, many publications have suggested the requirements for a research question.[28] In general, three things are necessary to formulate a good research question[29]:

· it should start with words such as 'how' or 'what'

· it should contain exploratory verbs, such as 'explore' or 'describe'

· its focus should initially be on one area of interest.

The research is designed to provide a service that cannot be gained by any other method, so understanding the practicalities of data collection for the evaluation of the proposed regulatory intervention may also inform how the research question is structured. Inconsistency between the evaluation method and the data provision can lead to difficulties, not the least of which is not being able to tell accurately whether or not success has been achieved.

Framing of a research question must also consider constraints on the process, such as the resources available and the timeframe.

5.3.2.2 Theory

Theory motivates the modelling framework. It assists in describing the key forces at work and detailing how system components and agents relate to each within a defined system of relationships. Its purpose is to describe and capture the drivers for the behaviour being exhibited.

Theory is incorporated in the modelling assumptions. These assumptions act as 'external validity' back to the problem that is being investigated, as part of the process of testing the model's degree of useful approximation to current reality (and thus whether or not it can act as a guide to the likely effects of changes to regulation.) for example, consumer rationality is an assumption in many models.

A theory can be as simple as assuming the demand for kiwifruit to be inversely related to price. This generates testable hypotheses like, that the lower the price of kiwifruit the more consumers will buy (all other things being equal). Or it can be more elaborate, such as a theory that ties overall output of the economy to the behaviour of different sectors and their interaction in markets like labour and capital, which motivates a modelling structure using many complex equations.

5.3.2.3 Data

Data is used in two vital ways in a typical model. First it will provide the basis for the construction of the sections of the relationships – through the estimation of the key parameters. Then, once the model is complete, another part the data will be drawn on to compare, contrast and test the relevant economic theory, trends in economic behaviours, and predict the behaviour of economic variables. Without data economists rely on the logic and manipulation of models – pure reason not empirical evidence.

This is a particular concern in small jurisdiction where the fixed costs per person of collecting relevant data are higher (sometimes much higher) than large countries. But for statistical validity, Statistics NZ, in some cases, carries out similar size surveys as their counterparts in Australia and the United States.

The expense of generating data means that a key issue is benefit transfer i.e. the portability of data from one jurisdiction (country) to another as evidence to support an intervention. How this is done, and the data and information used, requires care e.g. New Zealand is a water-rich country and while water quality is relatively high, it is fragile. To import water values from other jurisdictions that have water deficits or quality concerns will be drawing information from an unrelated situation, and therefore the values associated with water are inappropriate.

Despite this risk, useful evidence and indeed, many successful foreign regulatory approaches have proven to be appropriate and portable to New Zealand. For example, many of the decisions made in the 1980s to deregulate the New Zealand economy were taken from standard OECD policy prescriptions.[30]

5.4. Frame of reference

Figure 2 sets out the organising frame of reference to illustrate how different types of models can gauge the impact of regulatory responses. It also demonstrates how they can potentially be used. The level of economic growth is the outcome of a variety of indicators: social, environmental, cultural, economic and political factors. These factors interact with each other and directly shape the institutions (including the regulatory structure) that set rules under which economic growth is driven. How these factors impact on economic growth depends on the details of each area being examined.

Figure 2 - Understanding the causes of economic growth and the role of quantitative methods

Source: Adapted from Wold H[31]

5.4.1 Description of techniques

Below we examine quantitative methods that could be potentially used in further understanding the efficiency, effectiveness, and overall performance of any particular regulation or regulation change.

5.4.1.1 Cost benefit analysis

CBA can be viewed as similar to the financial analysis undertaken by businesses in determining the relative profitability of different investments. Instead of maximising profits, CBA is concerned with social well-being, the net sum of the economic costs and benefits borne by all those affected by the decision being considered. A form of applied welfare economics[32], CBA recognises that private appraisals need not coincide with social appraisals, because of effects external to the private decision-maker. There may be costs (opportunity costs) that do not figure in private finance flows or benefits outside the observable market transactions, so CBA takes a society-wide perspective. Costs can be regarded as negative benefits (e.g. noise is a loss of tranquillity) and benefits as negative costs (e.g. avoided future damage is the benefit of coastal defence works).

(a) Historical development

Cost benefit analysis is now widely used in the development of formal regulatory appraisals. However, this is only a comparatively recent phenomenon. According to David Pearce[33] its origins can be traced back to Jules Dupuit (1844, 1853), who set out the foundations of marginal analysis, defined the way full costs and benefits should be measured, and introduced the concept that benefits must outweigh costs.

While the theoretical foundations were further developed in the nineteenth and early twentieth centuries, it was not until the Flood Control Act of 1936 in the United States that CBA was applied with any commitment.[34] After World War Two, a Green Book was produced [35] and the Rand Corporation set out in a number of papers CBA principles.[36]

The use of CBA in New Zealand started with the appointment of Bryan Philpott as professor of agricultural economics at Lincoln College in 1959. His colleague John Ward produced the first publication of the newly formed Agricultural Economics Research Unit[37] arguing that:[38]

… the application of this type of [CBA] analysis to development projects in this country would be of great value, not only because it might result in more correct decisions than if these were made in an arbitrary fashion, but also because it would ensure a more thorough consideration of all aspects of the proposed development than appears to be the case at present.

By the 1960s and 1970s most of the theoretical development on CBA had been done. Though debates about where and how it might be appropriately applied continued. The choice of discount rate, intriguingly, became a particular focus of discussion both in New Zealand and overseas.[39]

(b) Matching 'like' with 'like'

Care needs to be taken in advocating the extent to which CBA can be the answer to key information requirements in any particular regulatory case. In principle, if a CBA is comprehensive and uses 'correct' prices and its results are acted upon, it should ensure efficient resource allocation, if its assumptions are well-founded. In practice, a more realistic expectation is that it should reduce the probability of inefficient resource allocations through poor selection of regulations, ex ante.

The focus of CBA is on the incremental change in benefits and costs should a regulation proceed. To be comprehensive, it needs dollar values for incremental changes in all situationally salient attributes affected by the regulation under consideration. Hence, non-market valuation methods have been developed to meet the need to apply consistent, defensible dollar values to effects that are outside the scope of market transactions (i.e. economic externalities).[40] In practice, they are not widely used and the value of some regulatory outcomes is often determined implicitly by political judgement. In such situations a cost benefit assessment can still be informative by indicating how big the unquantifiable outcomes would need to be to change the result of the analysis.

Not all CBA's need to be fully quantified. If the quantified items in an analysis suggest a large net benefit, there is no need to expend resources on attempting to quantify and value the unquantified aspects, if they are unlikely to be large enough to overturn the result.

The usefulness of CBA where benefits/costs are difficult to value hinges on using 'correct' prices. The use of methods that estimate people's willingness to pay (WTP) for new regulation and their willingness to accept (WTA) compensation have gradually gained ground over the years. According to Pearce:[41]

· benefits are gains in wellbeing and costs are losses in wellbeing

· a benefit is measured by how much an individual is willing to pay to secure that gain or how much they are willing to accept in compensation to forgo that gain

· a loss is measured by how much an individual is willing to pay to prevent the loss or how much they are willing to accept the loss

· if benefits exceed costs then the project is potentially worthwhile. Only 'potentially,' since there may be other projects or policies of higher merit and there is always a limited budget

· a discount rate is required since benefits and costs are likely to stretch out over time.

(c) Current use in New Zealand

With the Regulatory Impact Statement (RIS) needed in most instances for regulatory proposals in New Zealand the use and importance of CBA has increased. According to Treasury[42]:

A RIS provides a high-level summary of the problem being addressed, the options and their associated costs and benefits, the consultation undertaken, and the proposed arrangements for implementation and review.

The use of a CBA to inform the RIS can be valuable component of the RIS process.[43] As part of the revised Regulatory Standards Bill, any Bill being presented will require the explanatory note to disclose: [44]

· whether any regulatory impact statement were prepared to inform the government's policy decisions that led to the proposed legislation, if any statements were prepared, where they may be accessed

· whether an independent assessment was made of the quality of analysis and presentation for any of these regulatory impact statements and, if so, give a brief description of the assessment; and

· whether more recent estimates of likely costs and benefits of the proposed legislation have been made since any relevant regulatory impact statements were completed, and, if so, indicate where this information may be accessed.

The proposed Revised Regulatory Standards Bill sets out in clear terms the relationship between a RIS and CBA, with the RIS being the initial estimate and CBA being a more detailed analysis emerging as the costs and benefits become clearer. As noted below in the criticisms of CBA section and reflected in the Treasury discussion document, independence verification of assumptions and approach is becoming an increasingly important component of the CBA process, since the stakeholders, particularly central agencies, are looking for assurance that the CBA has been conducted in an impartial way.

We have put together two CBAs to illustrate how they can assist public policy. The first of these examines the removal of parallel importing restrictions on New Zealand given a single market with Australia, an increasingly globalised world, and to further understand the implications for future trade agreements.

Parallel importing is typically one of three things: a good exported, then imported back into the originating country; unauthorised imports competing with domestically produced goods; or unauthorised imports competing with authorised imports.[45]

Four sectors were identified as being vulnerable to parallel importing: books, music and sound recordings, DVDs, and Computer Software. Nearly all evidence unequivocally pointed to the removal of restrictions as being positive for the New Zealand economy with little impact on domestic industries competing with parallel imports. Further, the importance of parallel importing restrictions has declined as new electronically based business models have been developed.

The second CBA dealt with the likely impact of patent extension. A patent is a crude instrument that grants an exclusive right, in the form of a patent, to sell (as taken in this case) a pharmaceutical product. The patent holder is likely to reap significant profits since they are protected from imitators who can marginally price.[46] These profits are said to be designed to create incentives to innovate further. The impact of patent extension increases the length of time the patent holder has to make such protected profits.

Analysis of the costs and benefits shows that patent extension is unlikely to benefit New Zealand. As an illustration, we briefly examined the Australia United States Free Trade Agreement[47] and the Korean United States Free Trade Agreement.[48] We found that patent extension is likely to be costly, possibly between $100 and $200 million over three years.[49] However, the exact cost will depend on other (trade agreement related) changes that could potentially be made to the New Zealand drug buying agency PHARMAC.[50] The more that PHARMAC is constrained in its monopsony buying role the more costly patent extension could become.[51]

(d) Criticisms of CBA

Shaprio[52] and Kotchen[53]point out that CBA has become the analytical model of choice in regulatory analysis in the US and most OECD nations. Latterly, the debate has not been about whether CBA is used or not but how the CBA approach can be improved.[54]

In the US setting, the main concern is that institutions conducting regulatory impact statements are not independent. That is, a CBA is merely a cover for political goals. While many of the other criticism of CBA have dissolved, this criticism, first made by Olsen, has been re-emphasised, Shaprio who predicts that:[55]

 … questions about the independence of those conducting analysis will dominate the debate and the resolution of these questions may result in institutional structures that further cement cost-benefit analysis as part of the regulatory process.

Other concerns about CBA mirror critiques of economics more generally. These revolve around the use of efficiency as an appropriate criteria and the reliance on monetisation of all costs and benefits.

Kotchen sets out a number of these criticisms. These include:[56]

· distributional equity questions are not well addressed in CBA. In weighing up the costs and benefits most policies will have winners and losers. Critiques argue that while winners could compensate losers, a CBA only requires that benefits exceed costs. Since compensation rarely happens, using CBA can be at odds with distributional equity;

· costs and benefits are measured in terms of willingness to pay. Those who have less income have less ability to pay and therefore have less influence on the CBA outcome;

· dealing with intergenerational issues is problematic. This focuses attention on the appropriate discount rate – the higher the discount rate the lower weighting future generations are assigned in the calculation. This issue is important in longer run questions like climate change and regulatory interventions in childhood to prevent later life chronic conditions; and

· many things are difficult, if not impossible, to monetise. For example, some believe that preserving some flora and fauna is priceless; so some species have a right to live regardless of the costs and benefits.

It would be difficult to find a technique that is immune to some criticism and CBA is no exception to this rule. However, CBA is the dominant public policy tool in many OECD nations. Many of the criticisms are more directed at making refinements to the CBA approaches than at doing away with it altogether. Kotchen makes the point that CBA is "a decision tool rather than a decision rule"[57] and one which sets out a systematic framework to aid regulators.

5.4.1.2 Economic impact analysis

(a) History of impact analysis

Related to CBA is economic impact analysis (EIA), which seeks to enumerate the effects across all sectors of the economy of a change in investment or output of one particular sector. Such economic impact analysis is often associated with the calculation of economic multipliers derived from input output models,[58] which describe the total economic effect on output, incomes, value added and employment arising from a direct effect on the target sector, taking into account indirect and induced effects in other sectors.

However, an impact assessment does not represent a decision framework like CBA as it is confined to tracking transactions through the economy, rather than the effect on economic well-being (economic surpluses) generated by a particular set of resource use options.

Economic impact analysis, as commonly encountered is based on tracing impacts through inter-sectoral transactions in an input-output table, is not equipped to identify such surpluses. Nor does it indicate the efficiency of resource use, as the impacts calculated from such models allow for no constraints on expanding inputs of labour and capital into the different sectors e.g. if one sector increases its share of labour, typically it bids labour away from other sectors by offering higher wages. An input-output model allows the expansion without offering high wages; therefore it is likely to overestimate the expansion of the sector since it will not be constrained by having to offer higher wages.

(b) Current use

Without careful use, impact analysis is a very unsatisfactory tool for regulatory analysis since it is not a good tool for understanding the difference between the full 'with' and 'without' any particular policy. Impact analysis can only track economic activity stemming from a single activity in isolation from other aspects of the economy; therefore there is a danger of it generating meaningless numbers which do nothing to inform the investment or regulation decisions or the choice between competing options. As an example, NZIER commented on an impact (multiplier) study done for the Hamilton car race, where it was claimed that it would generate many millions of dollars for the city.[59] What the council report on the study did not say was how much of this money would be spent anyway, 'without' the car race happening – the counterfactual.[60] By quoting the very large (gross) benefits, a misleading impression was given on the impact of the car race for the city.[61]

5.4.1.3 Computable General Equilibrium Modelling

CGE modelling has gradually replaced other modelling approaches and become the main economy-wide investigation process.[62] CGE explicitly models resource constraints, thus the flow-on effects through the wider economy are usually smaller, compared with impact analysis. This comes as resources in the model move from one use to another (reallocated to different sectors) and so the true cost is included as well as the possible gain; growth becomes less bullish, contraction less bearish, and the pattern of impacts across sectors can be quite different. CGE models also have potential to calculate economic surplus measures - at least for the marketed sectors - providing a rather different picture of overall welfare effects across the community. But for all that, its main benefits are that, it considers all markets within the economy; it is empirically supported, and is grounded in standard economic concepts.

According to Dixon the factors that set CGE modelling apart include:[63]

· explicit economic behavioural theory is built into the model (e.g. households are utility maximises and firms are profit maximisers or cost minimisers);

· cost functions allow a change in demand for one sector to be reflected in changes in prices for the inputs used by that and all other sectors, followed by 'knock-on' responses to these changes including income effects and further rounds of subsequent adjustment and response. The significance of this is that a change in one sector would change the prices of resource inputs and the pattern of supply and demand in all other sectors; and

· numerical results are produced (they are computable). The parameters are linked to a numerical database.

(a) History

Johansen[64] developed the first CGE model for Norway using input output tables, household prices, and income elasticities produced by Frisch.[65] However, it was not until the late 1970s that CGE modelling began to be used in policy.[66] Dixon suggests that the main drivers for this were the oil shocks of the early 1970s and the ability of CGE to handle increasing amounts of data.[67] In the last 30 years the most significant change has been the development the ORANI model in Australia[68] and latterly the Global Trade Analysis Project (GTAP).[69]

(b) New Zealand experience

New Zealand CGE began with Byran Philpott in the 1970s. His Research Project on Economic Planning (RPEP) lasted for some 30 years.[70] Philpot's students Stroombergen[71] and Nana[72] have also developed CGE modelling programmes alongside NZIER.[73]These models have been applied to a wide variety of public policy issues from climate change[74] to tax policy.[75]

As an example for this project, the NZIER used CGE to examine the 1995/96 electricity reforms i.e. when Contact Energy was split from ECNZ and a wholesale electricity market was established. They found that initially this move created benefits for consumers of power and industries for which it was an intermediate input. These gains were lost with the unintended consequences of the 1998/1999 reforms which further split ECNZ into three SOEs based on geographical locations, and implemented significant regulatory changes.

The size of the gains forgone were significant. If the gains from the 1995/96 had been maintained then consumption would have been $4.4 billion higher and wages would have been 1% higher.[76]

5.4.1.4 Multi-criteria analysis

Multi-criteria analysis (MCA) is not a single methodology, but rather a suite of techniques for assessing different levels of weighting for regulation options that deliver varying combinations of outcomes. Some multi-criteria analyses take a form very similar to CBA except that instead of reducing all the diverse components to a common monetary valuation[77], weights are scored against criteria that are not explicitly economic. Their efficiency implications are therefore cast in terms of maximising 'weight-scores' per unit of expenditure, rather than an economic return. Assigning weights has a degree of subjectivity to it and requires close scrutiny to ensure that all relevant items receive their due weight. Other MCAs have a CBA component or economic impact component embedded within them, but weighted with respect to other criteria such as distributional outcomes, strategic fit and so on. Because of the variety of approaches to weighting, it is more difficult than with CBA to compare results across MCAs to identify which achieves the highest overall return.

(a) Current use

MCA has been used in The Netherlands to fill a perceived policy gap. Pearce refers to the Environment Agency in the Netherlands using multi-criteria techniques to rank projects, since efficiency was seen as not providing the right outcomes desired by policy process. Pearce comments:[78]

This offers one explanation of the apparent 'gap' between the current widespread practice of CBA and the fact that decisions appear not to be heavily influenced by CBA. If decision-making is rational, and efficiency was the only goal, then CBA should show up as being influential. But efficiency is not the only goal, nor are decisions always made rationally. It would therefore be surprising if CBA dominated actual decisions.

In the New Zealand Law Foundation's Regulatory Reform project's discussion about possible interventions in the consumer credit market the implicit question is whether or not regulation should be based on rationality.[79] Possibly, MCA could be used to determine consumer attitudes/values to credit protection i.e.: do consumers want more or less protection?

With carefully crafted questions MCA could examine consumer preferences and trade-offs between:

· the status quo i.e. the current information paradigm approach which makes it more difficult to police the oppressive or unfair contracts ex post, but gives consumers choice about the products to buy; or

· the development of safety paradigm means that the credit products are standardised somewhat. All oppressive or unfair contracts are banned. Thus there is a smaller and simpler range of credit products on the market. But less policing of unfair contracts is needed after the fact.

A WTP approach could also be used to further understand consumer preferences. This has the added advantage of being able to be compared with other costs and benefits in a CBA, unlike a MCA approach.

5.4.1.5 Econometrics

Econometrics is based on upon the development of statistical methods for estimating economic relationships, testing theories, and evaluating and implementing government and business policies.[80]

(a) History

According to Griliches and Intriligator the development of econometrics has been driven by the increased availability of data and attempts to examine that data in a systematic way.[81] An early example is what is known as Engel's Law where a study of household expenditure showed that the proportion of total expenditure spent on food tended to fall as incomes rise. This 'Law' has been tested in many countries and over various time periods.[82] In the first part of the 20th century sufficient econometric studies were being completed to found the Econometric Society in 1930 and the journal Econometrica in 1933.

(b) Current use

A common application of econometric models is forecasting macro-economic variables such as unemployment rates, labour market behaviour, interest rates, inflation rates, and gross domestic product. While not as high profile, econometric methods can be used to examine aspects of health, education and social welfare programmes. So the NZIER have been examining the impact of maternal smoking using a micro simulation model.[83] The parameters that are used in the model are econometrically estimated either from New Zealand or overseas studies. Once fully specified the ambition is to be able to rank various possible smoking policy interventions to gauge their relative effectiveness.

A common area for econometric analysis is the labour market. Stillman et al for example, examines the relationship between individual labour market outcomes, household income and expenditure, and inequality and poverty in New Zealand. The paper finds that income and expenditure declined sharply between 1983 and 1993 and rose sharply between 1993 and 2003. Levels of poverty increased between 1983 and 1993 and remained stable between 1993 and 2003.[84]

5.4.1.6 Cost effectiveness

Cost-effectiveness analysis compares the costs of alternative regulations which yield a single, common output. Such analysis can be discounted to indicate which option has the lowest present value cost over a defined period.

As cost-effectiveness is concerned with achieving a single, defined output, the issue of weighing up multi-attribute outputs does not arise and therefore the approach avoids the issue of valuation. Values can be inferred from cost-effectiveness analysis (e.g. the value per unit energy saved of different energy efficiency options; the value per statistical life saved of different safety programmes), but its single focus leaves to one side the challenge of valuing non-market effects, on the benefits side at least (although they may come into the cost side).

In the case of the value per statistical life saved, which is a non-market value, all that cost-effectiveness analysis indicates is whether a particular safety measure is more or less effective in saving lives per dollar spent than alternative known safety measures. This allows a (social) judgement call to be made about whether the given measure represents value for money, but it does not provide an indication of whether society would be better off spending its resources on that measure or on other things. To make that assessment involves a further social valuation judgement.

Although cost-effectiveness analysis has its uses, it has a narrow focus and is begs the question in respect of valuation of non-market benefits. Decision frameworks that do use non-market valuation are those that try to assess options across multiple outcomes simultaneously, namely cost-utility analysis, cost-benefit analysis and multi-criteria analysis.

Cost effectiveness is usually employed once a policy or course of action has already been decided upon. The aim is to deliver the investment or policy in the most efficient way possible. Its main risk is that the cost of even the most effective way of achieving the selected goal may be unacceptable if it were sensibly compared to a sound social valuation.

5.4.1.7 Cost-utility analysis

Cost-utility analysis occupies a position between cost-effectiveness analysis and cost-benefit analysis, as it compares options or regulations in terms of their cost-effectiveness in achieving gains in a multi-dimensional index or scale. Valuation across a number of different dimensions of output is implicit in the construction of the index. Cost-utility analysis becomes effectively a cost-effectiveness analysis of options for maximising the constructed index score obtained from available resources. It does not indicate whether society's well-being is enhanced to the maximum extent possible, except in terms of the scale selected. The most common example of cost-utility analysis in assessment of health outcomes, where Quality Adjusted Life Years combine effects on patient longevity, mobility and independence into an index that allows comparison of different types of health intervention.

NZIER has used a cost-utility analysis approach to examine options related to national state of the environment (SoE) reporting.[85] Two options were examined: options relating to the institution responsible for national SoE reporting and options concerning the long-term quality and availability of data.

For the custodian of statistics role the analysis found that the Parliamentary Commission for the Environment (PCE) had the highest scores for independence and credibility and the Ministry for the Environment (MfE) had the lowest. However, the cost of the PCE taking on the custodian role was $10 million above the status quo. A further option involved Statistics New Zealand (SNZ) taking over control of environmental statistics. The cost of SNZ taking over was $4 million above the status quo.

Reporting of SoE data was seen as the least beneficial of the policy objectives. Local authorities are focused on data for regional reporting purposes rather than any secondary collection use for national reasons. Stakeholders preferred a 'whole of system' approach to data quality and consistency. If intervention were to be found warranted the preferred approach was an RMA based option such as the use of a National Environmental Standard.

A cost utility approach can be useful since it sets out the preferences of stakeholders in any particular area. However, its primary weakness is that it does not provide a measure of the economic efficiency of options. This means that it will not be possible to determine if the benefits outweigh the costs.

5.4.1.8 Other approaches

Commercial entities considering a course of action or regulation generally prepare a 'business case' to assess whether it is likely to be worthwhile. Central to this consideration is likely to be a discounted cash flow analysis, which enables the initial capital outlays at the start of the regulation to be compared with the stream of revenue flows over the lifetime of the regulation. Generally, only the entity's own private revenues and expenditures are considered in such an analysis – externalities do not count directly, although some analyses may consider them indirectly, for instance in allowing for an expected value of fines and penalties should an externality-causing infringement of rules and laws be detected.

A related approach is pay-back analysis (also known as break-even analysis), which calculates simply the time required for a regulation to recoup its initial (setup and/or capital) costs. For instance, the payback on plant and machinery refurbishment would be a function of the costs of refurbishment and the additional revenues generated from use in subsequent years. The financial return or revenue stream is critically dependent on the forecast of future use. Such analysis is of limited use for comparing options as it does not directly take into account differing time profiles in the costs incurred and returns earned by different regulations. Setting a target payback for regulation approval discriminates against regulations with a long-term stream of benefits and high up-front costs, as well as regulations likely to gain from value shifts over time. In short, payback analysis can overlook regulations with the highest long-term returns, as it seeks to avoid having to deal explicitly with the intertemporal effects usually captured in a discount rate.

5.4.1.9 Role of behavioural economics

Behavioural economics seeks to use the basic foundations of economics as found in introductory textbooks and integrate these ideas with the realities of life and behaviour as exposed by psychology. Just as standard economic analysis takes into the account market failures that arise from transaction costs, externalities, imperfect competition, and information asymmetry, behavioural economics attempts to take into account market, court, and government failures that arise from the fact that people are not perfectly rational, calculating, or interested in utility maximisation.

Kahneman and Tversky were instrumental in the development of behavioural economics, although it has been long recognised that the behaviour of individuals, firms, regulators have important implications for behaviour.[86] According to Earl there are three themes which are prevalent (1) decision heuristics: people often make decisions based on 'rules of thumb' logic. (2) framing: the collection of anecdotes and stereotypes that add up to a recipe for success, and (3) mis-information, include mis-pricings and non-rational decision making.[87] An example of a rule of thumb approach is for a consumer to buy the top selling new car every five years.

Behavioural economics explicitly attempts to relax the assumptions set out in orthodox economic models and attempt to draw policy conclusions. Spiegel gives the following example: if you want people to save for retirement it is important to understand that people are easily overwhelmed by information and so are likely to simply opt for the status quo. If you want people to enrol in the pension plan, then automatically enrol them. Only after they have enrolled should you allow them to opt out: "You must push them in the right direction".[88]

The real issue for behavioural economics is how much further has a greater understanding of human behaviour got policymakers relative to standard (usually simpler) economic approaches? While policymakers have been able to integrate some aspects of human behaviour into models that assist policymakers, a checklist has not emerged from behavioural economics that should be used in general policy formulation. For this to happen, would require more explicit testing of the rules of thumb being used and increased transparency.

While behavioural economics may be useful for developing insights into firms and other institutions, generalising it to the population as a whole is questionable since there are likely to be a whole raft of behaviours acting in unison or against each other. Not only is the modelling of those behaviours extremely challenging and complex but the data required to drive these models in a small jurisdiction such as New Zealand is not always available.

5.5. Implications

In most developed countries, the use of quantitative methods to guide regulatory management has gone through cycles; from occasions when it has been used heavily to those when its employment has been sparing.[89] Currently, quantitative methods, particularly CBA, are being used heavily and exerting increasing influence.

The following implications set out what we might be striving for, how quantitative methods can assist, and the obstacles that need to be considered.

5.5.1 Quantification can assist in developing an optimal portfolio of regulation

From an economic perspective the aim of all regulation is to increase net wealth across the community, by assisting markets to work better or to correct some market failure or social injustice. In other words, regulation is intended in some way to improve community well-being, social welfare, net benefits or some similar measure. This is consistent with the stated purpose of good regulatory management set out by the Treasury.[90] A good regulation is one that achieves its performance aims, is effective, and improves efficiency, which normally requires an integrated assessment as provided by a Regulatory Impact Analysis, or Cost Benefit Analysis to identify whether a regulation is worth having.

An optimal level of regulation in New Zealand is one which maximises societal well-being from use of resources. In principle, this would result by starting from a (virtual) situation of no regulations and then proceed to add successive regulations across activities as long as each new one adds incremental net benefit. (Some way would have to be found to ensure all possible regulations were being tested.) This would result in an optimal portfolio of regulation for New Zealand.

Complicating matters further, the world is 'dynamic,' with features constantly changing (including tastes and technologies). Moreover, the measurement of 'net benefit' may be seen wildly differently by different groups of people. For instance, there may be a tension between those doing the regulation and those being regulated, if they assess the net benefits of the restriction quite differently. Pragmatically, such differences are part of the stuff of political decisions in a nation state. So having explicit transparent consideration of costs and benefits of each regulation and regular frequent reviews as circumstances change with new legislation, new technologies, and new market conditions provides citizens with assurances that social processes are sound – even if their outcomes are not what would be favoured.

For a quantitative approach to accurately reflect how the costs and benefits fall, the scope of an analysis needs to be defined broadly to include welfare effects on producers, consumers and affected third parties, including general public interests such as effects on residents' health and safety and on more abstract notions valued by citizens such as the state of the natural environment.

An optimal portfolio of regulation requires steps being taken, on an on-going basis, to:

· remove regulations whose costs exceed the value of likely benefits, or whose function has been taken over by other changes in the regulatory environment;

· replace components of existing regulations with less costly alternatives, as indicated by cost effectiveness analysis of options over time.

5.5.2 Data is one of the main obstacles to quantification

Improving the efficiency of regulation is important for retaining the New Zealand economy's capacity to adapt to changing circumstances, innovate to increase its productivity, and maintain its competitiveness in a changing world. It is part of New Zealand's ability to remain a viable part of the international economy.

Efficiency, effectiveness and overall performance of regulation needs to be seen in the wider context of full costs and benefits that regulations are intended to deliver.

As this book points out, this is not a straight forward or simple task. Demonstrating efficiency, effectiveness, and performance can be approached by asking the right questions, ensuring we chose to approach their answers via the right method and theory, and seeking to have the right data to understand and test the questions.

In a small jurisdiction the main issue is acquiring and maintaining a data set that can assist in understanding the regulatory questions being asked and serve in the evaluation of regulation effectiveness.

5.5.2.1 Benefit transfer is not a panacea

The high fixed costs associated with data gathering means that policy makers look to other tools to gauge success. Foremost among these is benefit transfer. Benefit transfer is:[91]

… a practice used to estimate economic values by transferring information available from studies already completed in one location or context to another. This can be done as a unit value transfer or a function transfer.

It is normally associated with environmental values e.g. the EVRI database[92] has over 2,000 international studies that provide values, methodologies, techniques and theories on environmental valuation. The aim is to provide valuation using the benefit transfer approach.

The problem with benefit transfer occurs when we strike situations that are unique to New Zealand e.g. cultural values associated with Maori and/or Pacific people. This means there is a mis-match between values estimated overseas and the possible values in New Zealand or for some reason the economic behaviour is different. Applying other jurisdictions estimates to underpin regulations can create continuing unintended costs and erode expected benefits.

The underlying problem is a version of the issue that was sought to be addressed by benefit transfer in the first place. New Zealand's understanding of its relative individuality is almost as lacking as its regulation supporting data. It would seem, though, that a relatively high level stock take of the degree of individuality of the key parameters most often required for regulatory analysis could be undertaken at reasonable cost. This would provide validation of areas where benefit transfer was potentially possible, and a list of those where it was more problematic. Further refinement could follow.

5.5.2.2 Problems with classifying benefits with no data

When there are no observable arms-length market prices or it is difficult to observe how much individuals, stakeholders, or government are prepared to pay to correct problems, or retain existing states, then other methods are required to understand and size up economic value. This is particularly important when dealing with issues that are unique to New Zealand. To illustrate this issue we have used freshwater as an example. One way of understanding value is by classifying the different types of value to arrive at a total economic value.

Figure 3 sets out the approach to total economic value associated with freshwater.

Use values are readily classified into commercial uses (values identified in a market) and indirect use values (e.g. parks) where resources are directly used but no use price is observable in the commercial market.

Non-use values can include:

· An option value where the resource is preserved for future use

· Existence value where the resource is preserved or improved e.g. Zealandia is an attempt to recreate native flora and fauna

· Other non-use values could include bequest values, existence values and cultural values.

Total economic value demonstrates that not all value or benefit can be observed trading in the market. That is, non-market issues need to be considered, too, since we are interested in whether the total social benefit outweighs the total social cost.

The main problem is that we have no data to support a credible valuation for non-market use. Benefit transfer cannot be used because value estimates come from water-scarce regions and therefore provide unrealistic values. New Zealand faces a different threat; we are not water-scarce, but potentially could, in the future, have a water quality issue. Therefore, to inform policy and further understand how New Zealanders view quality and the possible trade-offs required to reach an optimal level of water quality is highly valuable information.[93]

Figure 3 - Total Economic Value - Fresh water example

Source: Adapted from Serageldin[94] & Harrison Grierson[95]

In New Zealand, there are very few studies that examine the national perspective of river values. It is national studies rather than regional studies which are of interest because people tend to be more attached to resources that are close by, inferring that the existence values decline with distance.[96]

Sharp and Kerr refer to two studies that have taken a national perspective:[97]

· Kerr[98] which is a study of the Kawarau River values. Kerr's study suggested that New Zealand households would pay $197 ($242 in 2010 dollars) per household to prevent the Kawarau River hydro-electricity development; and

· Greer and Sheppard[99] study of funding for biological control of clematis vitalba. This study suggests that New Zealanders were willing to pay $7 ($8.61 in 2010) per household to prevent the spread of clematis vitalba.

This suggests that the existence value (and other non-use values) are somewhere between $14.7 million and $398 million.[100] This is a wide variation and gives some indication of the lower and upper bound, although the estimates are dated. The real problem is that these calculations favour various actors, who for their own reasons play up the extremes of these estimates to support their respective cases. Further, because we have only two dated studies with a wide variation, we are not confident that the midpoint will give us a good approximation.

Water quality will be a major issue for New Zealand over the next thirty years and there is a danger that policy will be made without reference to New Zealanders attitudes to water quality.

5.5.2 Frequency of use

While these evaluation techniques are well established and understood, in many sectors they are not universally used. In a recent study by Hitchiner and Gill[101] of cost effectiveness in government they found "that more than half of the departments made no meaningful attempt to address the requirement for cost-effectiveness measures". Further, there was a perceived lack of leadership from the centre in relation to cost effectiveness.

The current drive by the New Zealand government to focus on value for money has generated renewed interest in developing good estimates of the costs and benefits of a proposal. For example, the Treasury are currently reviewing their guidelines for regulatory standards and have clearly spelt out the need for CBA, and where possible independent verification of the costs and benefits.

5.6. Conclusions

Improving regulatory quality must be a key objective of any government. Economic tools can assist the improvement process by testing the efficiency, effectiveness and overall performance of a regulatory initiative. Whatever the chosen economic method used, the successful use of economic tools is based around the policy questions that attempt to address the regulatory problem, theory that describes the economic relationships between the stakeholders, and data that can be used to test the theories used. The absence of any one of these building blocks can potentially lead to sub-optimal outcomes or misleading information on the success or otherwise of regulatory performance.

A number of tools can be used to shed light on the effectiveness and efficiency of regulatory approaches. These tools can demonstrate how environmental, social, cultural and economic factors impact on economic growth and show how policy and political factors can improve or constrain growth. Some of these economic tools are geared to examine specific micro (e.g. CUA and choice modelling) economic problems, while other can be used to examine the whole economic system (e.g. CBA and CGE).

The most widely used of these tools is CBA. CBA is an aggregator of impacts that can demonstrate how regulatory initiatives affect stakeholders both positively and negatively. Its strength is its analytical rigor, monetisation of all costs and benefits so that like can be compared with like, and the ability to come up with a 'bottom line' conclusion. It is the dominant decision making tool in sectors such as transportation, health, law and order and the environment. While not as widely used, and often more data-demanding, CGE modelling, choice modelling, econometric techniques and CUA are all usefully employed to illustrate the efficiency and effectiveness of regulatory design.

The benefit of using these economic tools is to develop a systematic approach to regulatory decisions and implementation matters so that societal well-being is optimised from the viewpoint of the state's limited resources. In principle, this means promoting regulation that adds a net benefit and removing regulations that do not provide a net benefit.

CBA and the use of other economic tools are not without their critics, who raise specific issues. These include questions about the independence of those doing the analysis; concerns that the technique does not address distributional equity; worries that poorer people have less impact on CBA outcomes since CBA is based on willingness to pay; fears that intergenerational issues are handled less well since discounting means a dollar today is more important than a dollar tomorrow; and claims that it is too difficult to monetise values in some areas (e.g. intrinsic environmental values such as species preservation).

Despite these criticisms most OECD countries use economic tools to gauge the effectiveness of regulatory actions. Many of the criticisms of CBA, for instance, are more concerned with improving CBA process rather than doing away with the approach all together.

This practical point, taken together with the arguments that a significant effect of analytical approaches is to stiffen up the quality of the debate, therefore suggests that CBA and other economic tools are likely to be more, rather than less important, in the future. What they require is more support to improve the quality of their application and overwhelmingly, systematic moves to improve the supply of relevant data to provide the raw material for the analyses.