Chapter 12 - Weathertight Buildings and Performance-based Regulation: What Lessons can be Drawn from a Complicated and Evolving Situation?

James Zuccollo[*], Mike Hensen[§] and John Yeabsley[#]

12.1 Introduction

This chapter follows on from the Stage One chapter of the New Zealand Law Foundation Regulatory Reform Project, "Regulating the Building Industry – A Case of Regulatory Failure".[1] That chapter examined four aspects of the failure of weathertightness of some homes constructed under the performance-based building regulations.

Rather than push further into the arguments in Brent Layton's chapter, which are indeed, reasonably self-contained, in this chapter we consider the wider lessons from the 1990's change in the regulation of the building sector.

The new regulations in the 1990s can be seen as a stress test of a new regulatory regime. This is because of: the scope and scale of the change; the explicit confidence placed in market actors to deliver the goals of the regulation; and because it affected an asset that was of significant value to its owners.

· The key issues we cover in this chapter are these aspects of the change:

· the type of rule – prescriptive- to performance-based;

· heavy reliance on expert knowledge, judgment and "prediction" that designs would meet performance standards in the sign-off process;

· decentralised implementation (by local authorities) with weak feedback loops, including unclear accountability for identifying and analysing performance failures;

· underestimating both the complexity of the risks (connected, rather than independent) and unknowns and the limitations of standard risk allocation mechanisms (insurance and tort) to correctly identify and price what turned out to be the main drivers of weathertightness failure; and

· the fragmented nature of the design of elements of the regulatory rules.

We then assess the way the failure occurred and draw out what the discussion contributes to the wider regulatory project.

12.2 Performance-based regulation

12.2.1 Change objective

We begin our discussion by considering the problem that performance-based regulation is intended to solve, and how to characterise performance-based regulation so that we can describe how the shift from prescriptive- to performance-based regulation might affect the allocation of risk and uncertainty between the regulator and parties affected by the changes. This discussion draws heavily on comments by Peter May who has written extensively on the subject of performance-based regulation.

May states that:[2]

The notion that regulations should be based on achievement of specified results rather than on adherence to particular technologies or prescribed means has been widely accepted as a basis for improving social and environmental regulation.

He also comments on:

· the high cost and capricious enforcement practices[3] of prescriptive regulation that encouraged the swing toward performance-based regulation;

· the need for performance-based regulation to confront the issue of how to balance tight controls in pursuit of consistency versus allowing discretion to promote flexibility and innovation;[4] and

· the appeal of performance-based regulation being as much about introducing a new regulatory regime (presumably as a "circuit breaker" to change behaviour of market participants), as it is about regulating for results.

These comments about the drivers of performance-based regulation are closely reflected in Peter Mumford's description of the objectives of the changes to the Building Act 1991:[5]

The key objectives of the 1991 reforms were to: encourage innovation in the building and construction industry through the adoption of a performance-based building code; encourage an improvement in service quality provided by local authorities through competition from private building certifiers; and provide a more coherent and accessible building control regime through a single building code and one-stop-shop.

In the next part of this chapter we outline a framework for characterising performance-based regulation that allows us to consider how regulations can be changed to enable innovation while ensuring that the innovative processes deliver the performance standard that is required.

12.2.2 Characterising performance-based regulation

Peter May suggests that the term "performance-based regulation" is broad and involves many possible combinations of trade-offs between prescription and flexibility. (The building regulations effectively operated prescriptive and performance-based elements side-by-side and allowed actors in the market some choice about how to combine elements from these regimes.)

May notes that it is difficult to define "performance-based regulation" in concrete terms, and he therefore opted to consider the conceptual basis of performance regulation. The key operational concept of performance-based regulation is to focus the regulation on the achievement of results rather than the compliance with a method or process. The focus on outcomes rather than process raises three questions about how to express the desired outcomes:

(1) How to define the outcomes, ie the goals or intent of the regulation?

(2) How to set a standard for the desired level of achievement that is, how do we translate the goals of the regulation into measurable outcomes or outputs?

(3) How the performance of the outcomes delivered under the regulation can be accurately and effectively measured against the standards set for the regulation? In particular, can the performance of outcomes be observed directly or does it have to be assessed indirectly?

We suggest that a further dimension needs to be added to this matrix – the potential for catastrophic failure or loss due to some or all of the markets of interest.

Our argument for the inclusion of this element is to provide:

· A counter-weight for the tendency of decision makers and market participants:

· to rely on past experience as a guide to the outcome of innovations, rather than clearly assess how much of their past experience is rendered partially irrelevant by the nature of the innovation;[6] and

· move toward "optimism bias"[7] when assessing the likelihood of future outcomes with actual experience.

· A checkpoint for regulators and decision makers to ensure they have considered the potential for type 2 errors (deviations from expected performance that indicate the expected underlying model of performance is incorrect or incomplete); how to recognise and adapt to the type 2 errors; and the consequences of a catastrophic failure of the product or service.

These questions can be answered with varying degrees of comprehensiveness and precision, as they all give rise – conceptually – to a range of possible situations. These can be illustrated, as in the following table.[8]

Figure 12.1: Scope of performance-based regulation – the four dimensions

Figure 12.1: Scope of performance-based regulation – the four dimensions

Source: Peter May, adopted by authors

The framework provides several relevant insights for the establishment of performance-based building regulations:

(1) to operate effectively, performance-based regulations need to achieve a minimum standard across three dimensions:

(a) the definition of outcomes;

(b) the desired level of achievement; and

(c) a method of measuring performance against the regulation; and

(2) cope adequately with the risk of catastrophic failure;

(3) performance-based regulations can operate with a range of combinations of detail and objectivity about outcomes; the variation in objectivity and detail across these dimensions suggests different allocations of uncertainty and expectations of judgment between the regulator and the parties being regulated;

(4) to be effective on establishment, performance-based regulation needs to be based on:

(a) a mix of the outcome definition, standards and performance measurement that can be understood and applied by the parties being regulated;

(b) a shared understanding about the allocation of uncertainty and risk that innovations will deliver the performance outcomes set in the regulations; and

(c) correct assumptions about the capability and incentives for the parties to exercise judgment on how to resolve issues where the regulations no longer prescribe a method or solution.

This framework suggests that performance-based regulations can be arranged on a continuum that indicates the extent of dependence of their success on the expertise of the participants in the regulated market and on the complexity of the product or service being offered. Performance-based regulations with qualitative definitions of standards and achievement levels, as well as reliance on estimated rather than observed results are likely to depend heavily on well-informed participants or simple products/processes to be successful.

The framework also implies that there is a potential for the design of performance-based regulations to evolve over time in response to the following:

· A recognition of gaps between the initial assumptions about the expected and actual stakeholder interpretation of performance outcome definition, standards and measurements; and

· A deeper understanding about the capacity of participants to change components and processes while still demonstrating that performance standards will be met, or providing reliable assurance that they will be met.

12.2.3 Reliance on expert knowledge – re-allocating risk and uncertainty

The above "unknowns" refer to lack of knowledge about whether a new solution will actually meet a performance standard. In the absence of this "knowledge", explicit decisions need to be made on how to monitor the compliance[9] of new solutions and how to recognise and share the risk of failure. These decisions become complicated when: the new solution contains multiple elements; the solution is required to last for a long time; there are limited opportunities to fast-track testing of the new solution before implementation; and the potential cost of failure is large relative to the stakeholder's capacity to bear that loss.

There is an extensive literature on paradigms for risk and reliability, and also on integrated approaches to managing these issues. There is also an extensive literature[10] on how, where there is uncertainty, [it can be argued that] there are inefficiencies associated with the exclusive use of negligence liability and that ex ante regulation can correct the inefficiencies.

A survey of these techniques is beyond the scope of this chapter. However, in the discussion of these issues we have compared the approach to risk and reliability management in the implementation of the performance-based building regulations to risk and reliability paradigms as discussed by Richard Robinson and Gaye Francis.[11] Here we consider:

(1) the factors affecting stakeholder recognition and the response to "new information" on the premature failure;

(2) approaches to allocating responsibility and cost of unforeseen failure among stakeholders after the event; and

(3) alternative models that could be used to increase stakeholder and regulator awareness of, and responsiveness to, the risk posed by "unknowns".

12.2.4 Expected vs actual implementation

The implementation of the performance-based regulation was the result of a long process starting with the Office of the Review of Planning and Building Controls (ORPBC) in 1982, followed by the Building Industry Commission (BIC) which reported in 1988 and culminating in the passage of the Building Act 1991 and the establishment of the Building Industry Authority (BIA).

Peter Mumford analyses the evolution of thinking about the implementation of the regulation in some detail and summarises the comparison of the expected and actual models of implementation as follows:[12]

1. Model A is one where essential requirements couched in performance terms are mandated, and a regulator determines, through a process of citing specific compliance documents, approaches that are deemed to comply with the essential requirements. Compliance documents may be based on international standards, the standards of other jurisdictions, or industry codes, but the key point is that there has been some form of standardisation process that is subject to regulatory oversight. …

2. Model B is one that, again, couches essential requirements in performance terms, but provides for both standardised solutions and alternative ways of meeting those requirements. The alternative route allows designers, constructors, and manufacturers to develop unique approaches to meeting the essential requirements, and then these are subject to some form of approval.

… What was delivered was an extreme version of Model B [in so far] as it was particularly enabling of alternative ways of meeting the performance requirements. Not only was quality control weak but, relative to what was envisaged by the BIC: (i) there was less explicit focus on the need to meet community expectations for buildings (ii) there was more focus on cost reduction as a goal (iii) there was less emphasis on the need for a comprehensive foundation of acceptable solutions, and (iv) there was less emphasis on acceptable solutions providing the benchmark for alternative solutions.

12.2.5 Old models and new questions

The major problem for experts in this transition and the regulators they are advising is how to distinguish a type 1 from a type 2 error. A type 1 error is an expected divergence between the expected model of performance of a system and the actual performance. Type 1 errors are consistent with the expected model being an accurate representation of what should happen and represent an accepted level of variance in performance. Type 1 error rates can be systematically reduced through adjusting accountability regimes and over time could become insurable because the drivers of the error rate can be understood using the expected model. A type 2 error is a divergence between the expected model of system performance and actual performance that indicates the expected model of performance does not reflect actual experience because the model is wrong or incomplete.

Type 2 errors are difficult to recognise because they appear as "outliers" and cannot be easily explained by the model of expected performance. Type 2 error rates are not responsive to tighter accountability based on closer attention to realising the expected model and are unlikely to become insurable because their occurrence cannot be predicted using the expected model. Type 2 errors can be managed through "incomplete" contracting – the explicit acceptance that there are unknowns affecting building performance that cannot be usefully specified at the time the contract is made.

With hindsight, the regulations did not create an effective process for either managing the uncertainty or creating a shared understanding among the actors in the market about the difference in uncertainty about the performance of alternative solutions, and therefore had the potential for unexpected failure of the alternative solutions compared to the acceptable solutions.

The lack of an effective process to manage uncertainty could be described as:

· an "implementation failure"because the feedback loops necessary to inform the evolution of complex regulatory change were not in place;[13]


· an "accountability failure" because:[14]

Prescriptive regulation seeks bureaucratic accountability through adherence to prescribed rules while performance-based regulation seeks accountability for results. The case of "leaky buildings" in New Zealand that is considered here illustrates what can happen when accountability structures are deficient. Flexibility was achieved without sufficient accountability for the performance for the particular building systems in question.

A common root cause for both of these descriptions of failure was a type 2 error – the lack of recognition of the uncertainty by the actors about whether new "alternative solutions" would meet the performance standards and the length of time required to determine whether they would meet the performance criteria. In particular, the "weathertightness failure" was the consequence of two type 2 errors:

· models of moisture penetration for monolithic cladding, and design changes such as the use of sealants instead of flashings, overestimated the capacity of installers and also the length of time (durability) for which these solutions could reasonably meet the moisture penetration performance standard;

· models of the durability for timber framing did not consider the possibility and consequences of moisture penetration (through the walls) on the structural integrity of the timber framing.

We see this type of problem as difficult to effectively regulate, without constraining the potential for innovation and change that is the objective of performance-based regulation. The analysis of the history of regulation of weathertightness is often framed as a comparison of performance-based versus prescriptive regulation. However, it is interesting to note that of the type 2 errors ("known unknowns") that contributed to weathertightness failure – the factors contributing to moisture penetration were enabled by performance-based regulations that encouraged "alternative solutions", but the use of untreated timber was (separately) approved as an acceptable solution for the building code – the prescriptive part of the regulations.[15]

12.2.6 Importance of accountability in performance-based regulation

The change objectives of performance-based regulations imply a challenging change in accountability structures for actors moving from prescriptive- to performance-based regulation. Accountabilities shift from ensuring compliance with an established process (a relatively simple check), to forming a view on whether a new process will deliver performance outcomes (a more complex professional judgment). Assessment of the performance of decision makers has to consider the quality of their decisions based on both what they could have been expected to know at the time they made their decisions, and how effectively and efficiently they learned from the outcomes of their decisions. Accountability arrangements have to include compliance enforcement, judgment of likely future performance and effective learning loops. Also, the shift in accountability arrangements needs to be clearly understood by all the participants in the decision making process, that is, the decision maker, advisers informing the decision maker, actors affected by the decision, and those responsible for the decision.

An authoritative statement on accountability provides a useful summary of the key components of accountability arrangements under a learning paradigm:[16]

· Detailed ex ante specification would be limited to areas where there was clear agreement that no change was intended or sought over the reporting period. In other areas, specification would focus mainly on process, and the emphasis of accountability might shift to ongoing explanation of how change was being managed and problems solved, as opposed to ex post detailing of outputs produced.

· There would be clear distinctions drawn between "first learning loops" (for control and correction), and "second learning loops" (for strategic adjustment and system change). Measurement systems would be clearly viewed as primarily for learning, and secondarily for control or accountability. (Bill Ryan describes this as "managing to learn about outcomes".)

· Unintended consequences would be a central part of the learning system, ie monitoring and evaluation activity would focus on all impacts, not just intended ones, and would include regular scanning for external indicators of unintended effects.

· There would be better connections between the centre (upstream and high level policy) and the periphery (downstream and joint work and service delivery), through stronger collaborative mechanisms and shared information bases.

· The emphasis on good governance (as opposed to principal-agent accountability), with an increasing margin of self-governance, as a major instrument of control, would get stronger. For example, there would be increased reliance on a strong ethical base, appointment of senior people with sound judgment and a sense of what matters, and inclusive processes of decision making.

· Where possible, there would be "self-regulation" built into the system (eg where appropriate behaviour and learning automatically increased access, responsibility, or resources (as in some Internet-based systems currently, which use rating systems to automatically provide greater access and powers to those who are judged by other participants to be the most valuable contributors).

· There would be greater use made of a Supplementary Estimates-type process to adjust funding more fluidly. As such, the process would need to be streamlined; for example, funds in possible significant change areas could be ring-fenced, and debate on them delayed.

· Overall, whatever the specific accountability arrangements are, they should encourage learning and adaptability, tolerate mistakes, and be capable of capturing lessons.

Accountability can be the "Achilles' heel" of performance-based regulation,[17] as failure to incorporate adequate accountability mechanisms into such regulations can contribute to a crisis such as was evident in the leaky homes debacle. Peter May sees three levels of accountability as the major contributing factor to the crisis.

First, is a shortfall in legal accountability, resulting from the imprecise specification of the goal of durability of structures. Second, fault with bureaucratic accountability, where bureaucratic controls for ensuring adequate construction were unreliable. Third, lapses of professional accountability, such as the lack of licensing requirements for builders undermining regulatory oversight.[18]

While prescriptive-based regulation attempts to achieve accountability through adherence to prescribed rules, performance-based regulation seeks accountability through results.[19] The leaky homes crisis allowed for flexibility without adequate accountability and "placed too much faith on self-correction of the marketplace as a means of control and too little emphasis on accountability for results".[20]

12.2.7 Importance of establishing learning loops

Policy implementation frequently becomes complex, not only when the problem addressed is complex or wicked. In complex implementation, effective organisational and individual practices facilitate learning by experimentation. Practices centre on detecting anomalies and then explicitly incorporating reflections on them in on-going design, implementation, monitoring and evaluation activities.

Figure 12.2: Experimentation and Learning Model of Policy Design and Implementation

Figure 12.2: Experimentation and Learning Model of Policy Design and Implementation

Source: Eppel, Turner and Wolf [21]

12.2.8 Scope and scale of change

Shifting from prescriptive- to performance-based regulation in the building industry enabled a shift from a standard (limited) set of building styles and costs to innovative new building processes. The performance of the building solutions and allocation of accountabilities under the old prescriptive approach had been tested by time and experience at both the system and component level. The process of pre-checking (reviewing the plans) was an exercise in considering whether the proposals met approved methods of construction. This was a relatively simple task.

The performance of the new "alternative solutions" was approved on the basis of assessment of the designs. This required the employment of appropriate experts[22] who could make the desired assessment. But the typical regulatory system seems to have persisted without such a change; using a set of underlying assumptions presumably based on systems that they were familiar with.

In addition, another possible checkpoint was omitted: the performance of alternative building solutions was not physically tested (before widespread adoption).

This shift to reliance on expert opinion would have changed the level of uncertainty about the performance of alternative solutions, even if it was appropriately resourced.

12.3 Leaky homes context

12.3.1 History of weathertightness failure

As a brief background, the leaky homes crisis[23] is often linked to the 1991 switch from prescriptive building codes to performance-based regulation that allowed greater innovation in building techniques. The change in regulation permitted more rapid innovation, but the aim of the legislation was to encourage productivity gains in the sector, which were seen as being stifled by the limiting nature of the prescriptive building codes. One of the new ideas was to construct houses in a Mediterranean style with untreated framing timber. The choices of Mediterranean style design (including monolithic cladding) and untreated framing timber were made at separate times, as independent decisions, for different reasons. Rather than a single clear-cut change (which might have been an opportunity for a holistic review of the potential for significant unexpected risks), building design and practice drifted through a series of gradual, incremental changes. These progressively eliminated the lines of defence that made conventional houses resistant to moisture damage, delivering the standard of weathertightness that stakeholders had come to expect. These decisions seem to have been based on expectations rather than evidence of their potential effects on weathertightness.

12.3.2 Evolution of leaky building problems

The timeline in Appendix A to this chapter indicates that the leaky building crisis developed in two distinct stages. In the first stage, the use of monolithic cladding increased the risk of moisture penetration and retention within the wall cavity. To be weathertight, monolithic cladding needed to be carefully installed and maintained to tolerances that were more demanding than those for weatherboard cladding. If these installation and maintenance standards were not met, then over time a house with monolithic cladding would not meet the moisture-penetration performance standard set in the regulations. In terms of the potential for regulatory intervention in the first stage, this meant that, not only were (ex ante) reviews of the plans of limited use to determine the final performance of the complete building, but because of the time required for moisture to penetrate and accumulate, it is unlikely that this potential for failure would be apparent in most cases immediately after the cladding was placed on the building, that is, when the building was inspected on site.

In the second stage, the use of kiln-dried timber in structural framing hastened the onset of structural failure. The move toward kiln-dried timber was gradual, and driven by a combination of the desire to reduce the time required for framing timber to dry before the building could be lined, and also to reduce the need for remedial work required if framing warped as it dried. Final approval of the use of kiln-dried timber appears to have been on the assumption that the moisture-penetration standard was being met, and without any clear understanding of the rot-prevention properties of the previous boron treatment.

Overlapping these two stages were style changes in the design of houses to favour two-storey buildings with narrower eaves, and the use of sealants rather than metal flashings over windows and doors. These design changes increased the exposure of the walls to rain and also the likelihood of leaks around windows and doors.

· The sequence and timing of decisions suggests that:

· cladding, framing and building design were considered as independent elements rather than as interdependent components;

· decision makers did not have the opportunity to make controlled tests of the proposed changes in standards or use of materials before or after these changes were made;

· overseas experience did not seem to influence local decisions on standards or use of materials; and

· experts were divided on the interpretation of evidence of failure of houses to meet performance standards – particularly with respect to the root cause of the failure to meet performance and the appropriate response.

To place the changes in building regulations in the characterisation framework suggested by May:[24]

· Standards for performance were defined in general terms, ie moisture penetration should not be such as to cause decay of the building or make the building unhealthy to occupy.

· The desired level of achievement was described in qualitative rather than quantitative terms, ie the potential effect of moisture penetration rather than rates of moisture penetration or volumes of accumulation.

· The measurement of the standards was predicted on the basis of a qualitative model of how the system was expected to perform if each of the components were to be installed as required when the building was constructed.

12.3.3 Estimate of leaky building problems

In hindsight, this method of construction was not suited to New Zealand's wet climate.[25] The problems are estimated to affect between 22,000 and 89,000 homes,[26] which has led some commentators to conclude that the problem is a result of a systemic failure of regulatory oversight.[27] The size of the problem is due to both the failure to anticipate the consequences of the approved changes, and the delay in reacting to indicators that the approved changes were not meeting the performance standards[28] of the regulation. Consequently, numerous authors have turned to the question of what regulatory oversight failure allowed the problems to occur and how this failure might be avoided in the future.[29]

Many of those homes were later rendered structurally unsound as leaks in the cladding allowed the timber frames to become wet enough to rot. At the time, however, stakeholders did not imagine the potential for this type of catastrophic failure and, more importantly, the combination of the factors that could lead to this failure. Even those who pointed out the risks of the design changes at the time they were considered did not fully identify the potential for building component failure, let alone the potential contribution of that component failure to building failure.

Several other factors contributed to the severity of the leaky homes crisis. These factors included a lack of detailed evidence-based understanding about the vulnerability of building systems to moisture damage; low awareness of overseas experience from building innovation; a loss of institutional memory about past cladding failures; and a slow response to evidence that new designs were not meeting regulatory performance objectives.

12.4 Identifying the regulatory failure

The residential building industry poses several challenges to performance-based regulation.

The changes in the building industry were not simple; in particular, they did not constitute a well-designed "experiment"[30] able to be controlled and assessed to analyse cause and effect. The hypothesis that the experiment was supposed to test, that is, the comparative durability of houses constructed using the new processes, was not clearly stated, and the experiment was radically altered part-way through by the decision to allow the use of untreated framing timber.

In our view, regulating the building industry posed the following practical challenges for regulators and industry participants:

(1) Industry participants find it difficult to define some standards. For instance, moisture penetration can now be measured using a meter, but the actual "safe" level is still in question, as the quantitative standard needs to take account of the features of the situation: the building system in question, as well as the various individual components; these all contribute to, or detract from, the delivery of the performance that has to be assessed against the standards set in the regulations.

(2) It is very difficult to construct a practical independent test of how different building systems perform over time – this means that the building systems enabled by the regulations are tested in "production" (by allowing their techniques to be employed) rather than in a controlled environment with limited exposure and comprehensive examination of performance.

(3) Inspection processes focused on constructing buildings to a standard that was expected to meet the performance requirements of the regulations but there was almost no testing of the performance of buildings after construction to either:

(a) confirm that the building system as a whole was meeting the performance standards set in the regulations, or

(b) identify the contribution of individual components to the achievement of building performance standards.

These factors led to what the Yates Committee report described as a systemic failure:[31]

Changes to the building control regime brought about by the Building Act, and too greater reliance [on] market competitiveness have, we believe, contributed to the systemic failure of the building industry. It is a systemic failure in the sense that, although the framework for the regulation of building work in New Zealand may, in part, be adequately designed, a wide range of participants have not complied with it. The system of procedural and technical controls also appears, in part, to be faulty in design and therefore inadequate in preventing undesirable outcomes such as the leaky buildings crisis.

However, New Zealand has experienced weathertightness failures with claddings under prescriptive regulations – therefore a shift to performance-based regulations was not necessary for the regulation to permit construction methods that were not weathertight.

It is very difficult to establish a counterfactual for the leaky homes due to the complexity of the industry and the building system, but the following thought experiment is instructive:

How different would the situation be if the profile of failure and loss from leaky homes met the following conditions?

(1) If monolithic cladding and other leak-prone design and construction processes were used, but the industry continued to use treated timber; houses would still have leaked, but it may have taken longer for the risk of structural failure to appear.

(2) If monolithic cladding and other leak-prone design processes had, in fact, been more or less as weathertight as other cladding systems, but the use of untreated timber had been approved.

(3) If the regulators had been assiduous in their roles — had specific standards against which they tested; made sensible assessments of the alternative styles; and increased their vigilance onsite — in all likelihood the increased riskiness of the methods and the difficulty of executing the methods effectively, plus the heightened possibility of leakiness developing as the buildings aged, suggests that there would still have been an upsurge in leaky homes.

What can we take out of these assessments?

(a) The process of developing and setting the regulations did not have a logic that mirrored the complexity of the building sector being regulated. In particular, the interconnected nature of the building styles and risk of component failure were treated as separate;

(b) Building science and expert knowledge was incomplete in that the building components were not considered as a system, and the implications of removing various lines of defence against moisture and rot were not understood;[32]

(c) The outputs of the sector were not explicitly monitored for "feedback" purposes;

(d) There seemed to be no central responsibility to react to the emerging picture that there was a systemic problem; and

(e) In the face of evidence of failure the regulator chose to continue the current approach rather than adopt a precautionary approach.[33]

12.5 The problem of durability

In the following sections we ask the following questions: What mechanisms could be employed to ensure the durability of legislation? How might these mechanisms have avoided the consequences we observe in the case of leaky homes?

12.5.1 Features of residential housing construction

The market for newly constructed residential houses has several features that make it a high-risk area to regulate.

(a) Owners are likely to seek compensation

Owners are likely to seek compensation for performance failure in residential housing because, for most owners, the value of the asset is a large part of their wealth. While they insure against normal risks the way weathertightness failures occurred precluded insurance cover. They do not typically have the financial capacity to repair major premature failure of the building and expect houses to have a long lifespan. Those expectations are based on both their previous experience, and on the quality assurance implied by compliance with local building regulations.

(b) Suppliers have limited ability to pay

Suppliers tend to have limited ability to pay compensation because of the small size of many construction firms. Building and design firms tend to be small businesses with modest reserves and an average business lifespan that is relatively short. Further, many may not be operating by the time homeowners begin claiming compensation. Construction defects can take a long time to emerge and may be aggravated by other factors such as poor maintenance and unrepaired damage, which lead to more complex legal actions.

(c) Alternative risk-assessment is not readily available

Alternative methods of assessing and re-allocating risk are not readily available to homeowners since information on the performance of different building methods is not widely available or easily accessible to lay people. The majority of homeowner insurance policies in New Zealand do not provide cover for gradual damage. Aside from the limited protection offered under the Master Builder's association guarantee scheme home warranty insurance is not available to New Zealand homeowners.

(Australian homeowners have access to home warranty insurance. However, the low profitability of this market and difficulty in obtaining reinsurance appear to have forced these schemes to be provided by state governments and limited the scope and duration of the cover.)[34]

12.5.2 Specific building industry issues

When considering the problem of constructing durable legislation in the context of the building industry, it is helpful to review the problems that the regulator faced. There are two issues highlighted by Mumford[35] and Layton,[36] in particular, which were not adequately addressed by the Building Act 1991:

· builders had inadequate knowledge of the uncertainty that they faced when building with new materials and techniques; and

· consumers had little information about the possible risks of the techniques used to construct their homes.

The first problem is one of uncertainty, as opposed to risk. Uncertainty can imply either that, the range of potential outcomes from a course of action is unknown, or that, although the range of outcomes in known the likelihood of those outcomes cannot be estimated. Risk implies both that the range of outcomes from a course of action is known, and that the relative likelihood of one outcome, or a range of outcomes, can be predicted with an acceptable degree of reliability.

While risk can sometimes be valued and re-allocated through mechanisms such as insurance and reduced by ensuring that accountability structures are robust, uncertainty cannot be managed in this way. Also uncertainty is difficult to include in decision making processes because it requires allowance for "unknown" events that are difficult to imagine or they may be "assumed not to apply" in the "mental model" used by decision makers to think about complex systems.

Over time, experience with processes and materials may provide enough information for the identification of cause of failure and an assessment of the probability of the range of time to failure for a process or material. However, this is not an automatic or simple process particularly for complex systems with a wide range of pathways to failure affected by multiple factors.

The decisions on the changes to construction materials and processes were not based on a well structured experiment to test for moisture penetration or rot resistance. The consideration of the building as a series of components rather than a system, ignored the uncertainty of how changes in one component might affect the performance of others. The lack of any objective local or overseas trial of the changes in building techniques meant that decision makers were dealing with a situation about which they had little information. In risk evaluation terms they were lacking adequate knowledge about both the probability of failure, and the scale of the consequences. It is the product of these two values that is the usual impact measure.[37]

The likely problems with the techniques were not fully known and the potential for failure of individual components and the consequences of these failures for other components was not available to builders. (In addition, the problems would have taken some time to appear and their causes were complex. Therefore, when builders were made aware of the problem they may have suggested a need to replace a faulty component, rather than change the system.) The second problem is one of asymmetric information: consumers were not aware that using a new building technique entailed greater risks of weathertightness failure. In general, they were under the impression that the building consent authority certifed that the building was weathertight and did not take the risks of failure into account. Layton suggests that this may be related to the slightly ambiguous role of local authorities and their lack of expertise.[38]

In combination, these two factors suggest that market participants had inadequate information to manage the risks and uncertainty that they faced. This, along with the failures of accountability at different levels, meant that market participants were left exposed to experiencing failure. Consequently, there may have been overinvestment in new housing designs, and under consumption of warranties, guarantees, sureties and expert advice to mitigate the information problems. That does not, of itself, suggest that the government should have intervened or that there was a market failure. However, the possibility that the lack of information could lead to large losses and consequential failure of the legislation should have been considered during the design of the regulations.

12.5.3 Accountability risks

As discussed, the key issues in accountability are: who is accountable and what are they accountable for; and against what level of knowledge, understanding and insight their accountability is assessed.

Accountability provides an ex ante incentive for people to perform to a standard provided they can be "reasonably" expected to know or discover how to achieve that standard with the knowledge and resources they have available. These relationships drive the system performance that is sought. The allocation of accountabilities, particularly with respect to accountability for monitoring and control of the regulatory system, changed before implementation.[39] The Building Industry Council (BIC) envisaged a central authority:

· responsible for monitoring the building control regime including the performance of the territorial local authorities; and

· a decision maker on interpretation approval and monitoring of the control system.

However, the decision making role for the central authority was not carried through into implementation and the monitoring role was narrowed. On implementation effectively:

(1) Designers and builders were responsible for ensuring that their work was consistent with the standards required by the code both on construction and through the life of the building.

(2) Territorial authorities were responsible for the administration of the Act and the Building Code and to judge whether designs would meet the performance standards within the code and confirm that the building matched the design.

(3) The Building Industry Authority (BIA) (the central authority) limited its role to an oversight of the system and did not accept responsibility for the administration of the Building Code.

This configuration of accountability meant there was no strong central accountability for quality control of the regulatory system for two reasons:

(a) Monitoring and feedback loops to check the ongoing achievement of performance standards for innovative solutions were not designed into the system. Instead, achievement of the performance standard was assumed to follow on from the approval of the design and construction of the building in line with the design.

(b) The central authority did not accept a role in aggregating and analysing the results of feedback.

If the accountability of the central authority had included administration and monitoring of the regulatory system it is likely that there would have been a faster response to the evidence of leaks. However, it is unlikely that changing the accountability of the central authority would have automatically filled key gaps in building science knowledge or triggered a system rather than a component based analysis of innovations.

12.5.4 Implementation risks

As was clearly illustrated by the problems of weathertight buildings, implementation of a legislative concept is equally as difficult as the design. Layton has detailed the problems of implementing the performance-based approach across 70 individual territorial local authorities.

Mumford points to the lack of experience in implementing such designs as a key flaw in the shift of regulatory goals. May[40] similarly points to inadequate expertise in implementation of the performance standards.

The May framework for performance-based legislation set out at the beginning of this chapter[41] suggests that the design of the performance-based regulations needs to be compared to the sophistication of the market participants and the complexity of products being regulated. Lack of information or uncertainty may prevent detailed definition and measurement of performance standards when performance-based regulation is introduced.

However, the logic of this situation should signal two (process) challenges for regulators:

(1) An ongoing exposure to uncertainty about the effectiveness of the intervention, and potentially failure of the regulations to deliver the required performance standards – with potentially, at least, catastrophic failure being one outcome.

(2) Coupled with these risks is an obvious responsibility to try to remedy the lack of information that is creating the uncertainty. The natural response is to seek to ensure that the monitoring[42] of the regulation is sensitive to evidence that performance standards are not being met. This is a non-trivial task and raises questions along the following lines for the regulator:

(a) How should the innovation benefits (for example, cost savings in construction) be compared with potential costs (more demanding maintenance or shorter component life) and risks or uncertainty about the expected level of performance?

(b) What learning loops were in place?

(c) How did these compare to ideal models for learning loops?

(d) What could have been solved by better implementation?

(e) What was the size and shape of the residual implementation problem?

12.6 Designing durable regulations

Performance failures often trigger calls for the introduction or revision of regulation to translate the "lessons learned" from the failure into new regulations that in hindsight would have prevented the failure, or at least, dispersed the cost. This section outlines three approaches that regulators could use to design "durable regulations". We use the term "durable regulation" to describe a regulation that includes a framework for addressing the risk of large losses concentrated among groups of individuals. Each of the approaches uses a different set of methods to ensure better recognition of the uncertainty attached to innovation by all stakeholders, incentives for monitoring of actual performance and change in processes if actual performance consistently fails to meet expected performance.

12.6.1 Regulation affects uncertainty and risk

We argue that reduction of uncertainty and reallocation of risk are direct consequences (and sometimes core objectives) of regulation.

Regulations define processes and standards, or roles and responsibilities, in the exchange of goods and services. They narrow the range of possibilities of what can be exchanged, how it can be exchanged and what buyers, sellers and potentially other stakeholders can expect from that exchange.[43]

As discussed previously, uncertainty and risk present fundamentally different challenges to the regulator. For reallocation of risk the regulator may be able to justify intervention on the basis of a better understanding of the risk or hedging opportunities not available or apparent to participants in the market. In this situation the regulator can facilitate better management of the risk than the market participants. For reallocation of uncertainty, the regulator is unlikely to have a better understanding of the potential outcomes than the market participants. Intervention might be based on diffusing the cost of failure across a larger group for reasons other than correcting a market failure.

Prescriptive regulation reduces uncertainty and allocates risk by requiring compliance with a set of standards and processes that are "known" to produce a specified range of performance outcomes. Providers of goods and services are responsible for complying with the processes and standards. End users can select a product or service with reasonable certainty that it will meet a pre-defined standard. The regulator is effectively providing a recipe to reduce uncertainty and reduce the risk of poor performance.

Performance-based regulation establishes the roles, responsibilities and reasonable objectives for providers, end-users and other stakeholders, but does not specify how providers should construct their goods and services. The regulator is reducing uncertainty and risk by specifying a common set of expectations for outputs, but leaving the participants to decide how they will meet the performance expectations.

Prescriptive regulation is exposed to failure if the assumptions on which the standard processes are based, change or are found to be incomplete. For example, the Christchurch earthquake highlighted the risk posed by a poor understanding of geotechnical conditions to the performance of houses that comply with prescriptive building codes. Whereas performance-based regulation is exposed to failure if the testing and verification of the performance of alternative solutions cannot keep up with the pace of innovation either because of the range of innovation or the time required for performance problems to be recognised. Irrespective of the type of regulation, the decision to regulate usually includes a reallocation of risk of product or service performance failure. However, these risks are not always visible at the time the regulation is implemented.

12.6.2 Difficulty for regulators

It is difficult for regulators to design a framework for dealing with unanticipated downside risks that will be supported by a government in response to a "crisis" (instead of an ad hoc change to the regulations and socialisation of losses.) The approach needs to be accepted as "fair and reasonable", both at the time the regulation is implemented, and when addressing future risks that were not specifically envisaged when the regulation was implemented.

As a starting point for discussion, we consider three alternative frameworks for addressing large concentrated risks for performance-based regulation:

(1) qualified approval for products or services using innovation;

(2) regulation as a managed experiment; and

(3) social insurance.

The decision makers' management of risk can be assessed against both what is known about the probability distribution for the risk and the risk preferences of those represented by the decision makers. However, response to uncertainty cannot be assessed in this way. The decision maker can only respond to uncertainty in two ways: take no action because of uncertainty; or make a decision that ignores uncertainty.

The approaches can be arranged on a continuum based on the extent to which risk is addressed through design of regulation (in anticipation of an event) versus provision for compensation. It is important to note that these options address the allocation of risk. They are not effective in managing or re-allocating uncertainty.

A drawback of both the qualified approval and managed experiment approaches is that they rely on prescriptive regulation to manage risks in performance-based regulation. This can increase the cost of innovation and slows the adoption rate of innovation. This is why implementing robust accountability structures in performance-based regulation is so important, as robust accountability structures allow for risk to be reduced and opportunities for innovation to be increased.

Social insurance allows for more risk taking within the bounds of durable regulation, but may also encourage such risk taking – a phenomenon known as "moral hazard". The following sections briefly describe each of these approaches and the circumstances in which they are likely to be most effective.

12.6.3 Qualified approval as a signal of risk

The most "light-handed"[44] approach to the problem is to attempt to remedy the information problems. Of the two informational problems identified in the building industry at the time, the most straightforward was asymmetric information, which has well known causes, consequences and solutions.

Layton identifies the greater knowledge of the builders, relative to the purchasers of a home, as a source of information asymmetry. When purchasers cannot assess the integrity of the home prior to purchase, then they are unlikely to pay for the home as if it were of great integrity. That leads to developers cutting corners because they know that they will not be rewarded for their efforts with a higher sale price. This process is known as "adverse selection" and leads to a disproportionately higher number of (unknown to be) low-quality homes on the market. While it can be mitigated by using experts to check quality, such checks are expensive, and not necessarily definitive, while many buyers are struggling to make their purchases affordable.

The standard solution to this problem is for the seller of the home to provide warranties or guarantees against failure, thus signalling the high quality of the home.[45] The way the mechanism is implemented in the case of buildings is by the assignment of liability for failure to the parties with the best knowledge of the building's quality — the builder, architect and building inspector. This is the economic basis for accountability: it is not driven by blame, but by information and control of the risks. However, in the case of leaky homes, the problem took so long to surface that by the time the problems surfaced the liable commercial parties (builders and architects) had largely dissolved. That left only the building consent authorities to defend proceedings and they were primarily composed of territorial authorities.

To supplement the market mechanism, the government could provide a signal of its own: a qualified approval rating would be used to signal the regulator's assessment of the extent to which a building meets the performance standards described in the regulation. The assessment provides the purchaser with a clear indication of the expected reliability and lifespan of the product and reduces the potential for dissatisfied purchasers to later claim performance standards "promised" in the regulation were not delivered.

This assessment could consider a number of factors such as the construction techniques and materials used, as well as the availability of evidence on the reliability and lifespan of the product. To make the rating easier to interpret the performance standards could be ordered into a hierarchy. The foundation of the hierarchy would be the minimum requirements for safety and reliability for a defined period. Higher levels in the hierarchy could show ratings of expected lifespan based on evidence of performance either in New Zealand or overseas.

This type of approach is best suited to relatively simple products with a limited number of performance measures and a short lifespan (compared to the pace of innovation). For more complex products, the rating approach becomes cumbersome to apply and difficult for purchasers to translate into a useable comparison of the risk of different product offerings. They are not well suited to the building industry because of the complexity and interdependencies in building systems. So, even if implemented, they might be of little assistance or be confined to subsectors of the total market.

12.6.4 Explicit regulatory experiments

A precautionary approach[46] to dealing with the uncertainty of losses would be to limit exposure until the risks are known. Mumford recommends viewing regulation as an experiment to encourage better recognition of the need for continued monitoring and consequential adjustments to ensure that regimes continue to be effective and efficient.[47] For example, the regulator could allow 1,000 houses to be consented in a particular new design. The outcome for those houses could be regularly reviewed until the regulator was confident that they had a good idea of the risks involved in building that type of house. They could then decide on how to proceed, given society's risk preferences.

The word "experiment" carries connotations of measurement and opportunity to learn. These connotations suggest a managed approach to both regulatory risk and evidence-based evolution of the regulation in response to changing circumstances. Applying the analogy of an experiment to regulation raises several practical issues for regulators establishing the experiment including the following:

(a) how to define a control and trial for the experiment;

(b) how to set and manage the target level of participation in the trial;

(c) how participants would choose to join the trial or remain in the control group;

(d) how long the experiment would need to run to allow measurement of results; and

(e) how to establish feedback and learning loops.

A key question for regulators is how to design and manage a more effective and efficient experiment[48] to test the effect of a change in regulation. The challenge for regulators is how to apply an "experimental" approach to changing regulation for complex long-life products because of the complexity and difficulty in defining and controlling the experiment, and the time required to complete the experiment.

(a) Defining a control and a trial

Ideally, defining a control for an experiment requires being able to operate the old and new regulatory regimes side-by-side for a defined period. Operating regulations as an experiment may require the scope, size and speed of the trial to be defined as part of the regulation. This implies that the regulator needs to have a prior view about the type of changes that could occur because of the regulation, and which of these changes are risky enough to require trialling. The trials need to be set up to give reliable and usable measures of how well the innovation is meeting the performance objectives of the regulation. In practice, the distinction between a control and a trial may need to be blurred to accommodate the potential combination of multiple innovations. There may be multiple trials running at any given time and each trial may contain cohorts of innovation that have been underway for different periods of time.

The purpose of the trial is to limit the adoption of a particular change to a defined group until the efficacy of the change can be tested. Limiting the adoption rate caps the downside risk of failure, but also prevents the evolution of the innovation over time. Data gathered from the trial allows uncertainty about the innovation to be translated into a measure of the risk of failure.

(b) Choosing to be in the control or the trial

A participant's choice of whether to join the control or the trial group will depend on his or her perception of the benefits of each and their appetite for risk and uncertainty. The issue for the regulator is how to ensure that participants recognise the asymmetry of information between the control and the trial and make an informed decision about the level of uncertainty or risk they are accepting. The regulator will need to determine in advance what, if any, losses incurred by the trial group will be socialised and over what group these risks will be socialised.

(c) Duration of the experiment

The duration of the experiment needs to balance the trade-off between the time required to gain confidence that the performance of the innovation is understood, and the cost of deferring the realisation of benefits from the innovation. The required duration of the experiment limits the applicability of this approach to products where the cycle of innovation is considerably shorter than the time required for the innovation to fail.

(d) Establishing feedback and learning loops

Establishing learning loops requires agreement on how and when to measure the results of the experiment, and then perform a comparison of the results with expected results. This leads to an analysis of the root causes of any differences between actual and expected performance. The more complex the system that is the subject of the experiment, the more complicated it is to set up the measurement processes; let alone agree the root causes of any gaps between actual and expected performance.

Establishing feedback loops is even more difficult than establishing learning loops as each feedback driven modification of components or processes, changes the experiment not just for the component directly affected by the feedback loop, but also other related components. Therefore, any feedback driven modification also requires consideration of how the model for expected outcomes needs to be changed and whether or not the modification triggers a new experiment.

12.6.5 Insurance against risk

Both the signalling and experimentation approaches seek to control, or manage, the level of risk such that no socialisation of losses is required. The drawback of these approaches is that they correspondingly limit the gains from innovation, while also failing to eliminate the risk of socialisation. An alternative approach is to acknowledge that the government will sometimes be forced to act as a large, mutual insurance cooperative. That induces moral hazard, as previously described, and increases the incidence of risky behaviour. A possible response is to formalise the system of insurance and induce or compel people to insure against the events that might induce the government to step in.

(a) Inducing private insurance

The least invasive such measure would be to induce people to take out private insurance against the possible losses. However, it is doubtful whether a large enough proportion of the population could be induced to take out insurance to avoid the risk of socialisation. That view was taken by the then Department of Building and Housing in their advice to Cabinet.[49] They say that private insurers in New Zealand are uninterested in providing such cover because:

(1) there is a lack of information about the level of risk;

(2) there is a long period of cover required; and

(3) there is a lack of regulatory certainty, with regular changes to the legislation governing the sector.

Because of these factors, private insurers are reluctant to enter the market for providing comprehensive home warranty insurance. Part of their reluctance is also likely to be due to the adverse selection that would arise in the market. It is likely that the premiums would be high enough to deter people who are very confident of the quality of their home from purchasing the insurance. That leaves only the risky participants in the market and pushes up premiums further. Consequently, it becomes untenable for the insurance company to profitably provide insurance unless it can distinguish between high and low-risk participants. This is borne out by the products that do exist on the market, which have significant limitations and exclusions.[50]

(b) Compelling private insurance

The next step in the continuum of compulsion would be to compel insurance companies to provide such insurance and require people to purchase it. This approach would overcome the adverse selection problem by preventing anyone from opting out of the market. However, without good information on the riskiness of market participants, there would still be a moral hazard in the market: home buyers and builders may act in a riskier fashion than if they did not have insurance. These risks can be mitigated to some extent by the use of incentives in the insurance contract – co-payments in the event of a claim, for example – but not eliminated.

The costs associated with a functioning compulsory insurance market are largely related to the loss of efficiency that occurs due to compulsion. There are some people for whom it is not worthwhile to purchase insurance, and some people for whom it is not profitable to sell them insurance, yet all must obtain it. For each of those transactions there is a loss of social welfare, which is a cost that must be weighed against the costs of ad hoc socialisation. However, ad hoc socialisation also imposes many such costs since it is effectively insurance without the premium.[51] That means it does not charge the people who incur the risk for the cost of providing the cover, which significantly increases the moral hazard of the implicit scheme.

There are also risks to the insurance market. If companies do not consider it profitable to enter the market at all then it may end up being extremely thin. There is also the risk of companies entering the market and underpricing the cover to gain market share, in the belief that the government will socialise losses if the insurance company ends up being unable to cover them. This situation occurred with AMI Insurance following the Christchurch earthquakes and may have generated moral hazard in the insurance market itself.[52]

(c) Social insurance

The final option is a social insurance scheme that involves the government establishing its own, formal scheme to insure against the risks of failure. That allows the legislation to internally cope with losses and ensures its durability in the face of realised risks. However, it also engenders significant moral hazard and may have negative effects on the existing market. The Cabinet paper points out that the effect on the market depends upon how the scheme is implemented:

(1) The use of a compulsory levy on building consents would be likely to drive out existing schemes and would therefore meet with industry resistance. However, if the current schemes are not sufficient to compensate for the problems of the weathertight homes, then this may not be particularly costly from a social perspective.

(2) A competing guarantee scheme (presumably with a requirement to have some cover) would likely result in adverse selection driving all of the high-risk customers to the government-run scheme, given that the government would implicitly cover these participants through ad hoc socialisation anyway, that may not be an additional cost relative to the status quo.

The advantage of social insurance over ad hoc socialisation is twofold: First, the people who create the risk pay for the consequences of it, which is more equitable. Second, the existence of a payment for taking on risk reduces the incidence of risky behaviour, which diminishes the moral hazard imposed on the government and, consequently, on the taxpayer.

Notwithstanding the comments in the Cabinet paper, the New South Wales Home Warranty Insurance Scheme[53] has been in place for a considerable time. During this time, the risk of claims has been underwritten alternately by private underwriters and the State of New South Wales. The recent return to state underwriting seems to have been driven by a combination of falling profitability of insurance in this market and two rounds of contraction and restructuring in the reinsurance market.[54] The scheme provides cover to homeowners for the cost of repairing structural defects identified within a set period (six years for structural defects and two years for non-structural work) after completion of the work. A detailed review of the interaction between the building warranty approach and the development and enforcement of building standards in New South Wales, was beyond the scope of this chapter.

12.7 Summary

The weathertightness failures in New Zealand prompted a revision of building legislation. It was also followed by a series of court cases as those who faced significant costs sought to find others who might be liable, and thus assist with the funding of their losses. A range of commentators have examined the case. Our interest has been in high-level systemic implications for regulation as this proved to be a significant example of the limits of conventional regulation.

Key lessons learned from the experience with the legislation include the following:

(1) The regulation of complex systems requires regulators to make judgments about interactions that neither the regulators nor their expert advisers fully understand – a clear distinction between uncertainty and risk should be made in deciding how to regulate complex systems.

(2) Researching the interactions in complex systems may not be feasible in the time frame within which regulators are expected to act, or with the resources available. Other ways need to be found to address this type of uncertainty such as looking and reviewing precedents in other jurisdictions.

(3) Performance-based regulations are easier to understand and enforce when the indicators of performance are directly observable and closely connected to what the regulation is intended to achieve.

(4) Accountability (risk is high if accountability is poor):

(a) legal accountability – durability of structures should be more precise;

(b) bureaucratic accountability – requirement for greater emphasis on specification of performance standards, stronger monitoring of building inspection practices and tighter licensing for professionals who certify building compliance; and

(c) professional accountability – tightening of standards by professional associations, licensing boards and peer reputations.[55]

These lessons raise the question of what mechanisms could be added to regulations to make them more effective in identifying and preventing potential performance failures, ie make them more durable.

We have considered three options

(1) light-handed signalling by the government to increase the information available in the market;

(2) explicit experimentation by the government that limits exposure to risk; and

(3) government provided or induced insurance schemes to cover such losses.

Each would work in some circumstances, but the particular nature of the building industry suggests that government-provided social insurance would have been the most likely to succeed in this case.

More generally, the question of durability is one that should be taken into account in the design of regulations. Policy makers should ask themselves what is likely to happen if things turn out badly, and whether their regulatory scheme is robust to those outcomes. That is not simply a case of maximising net public benefit, but also of avoiding instances in which large losses create vocal lobby groups. Accounting for the effect of future political responses may not create the most efficient regulation in a static sense, but it has the potential to increase both the dynamic efficiency and to make regulation more equitable in the long-run.

The experience of the leaky buildings suggests that in addition to assessable risk there may also be uncertainty about the outcomes from applying performance-based regulation to markets. The framework for characterising performance-based regulation suggested by Peter May provides a useful checklist for situations in which performance regulation may be subject to high levels of uncertainty. In particular, where the performance standards and achievement levels need to be defined in qualitative terms and the performance cannot be directly measured, the regulations are heavily dependent on the sophistication of market participants for their success.

Appendix: Leaky Buildings Timeline

New Zealand's leaky homes crisis is widely regarded as an extremely expensive regulatory failure. PriceWaterhouseCoopers estimated the financial liability resulting from the problems to be $11.3 billion,[56] and previous reviews have concluded that it represents a failure of the underlying, performance-based legislation. It has been described by some authors such as Peter May as an example of how not to implement performance-based regulation. This chapter builds on Layton's previous chapter and questions those conclusions by proposing an alternative interpretation of the problem, and the lessons it provides for future policy work.[57] In this section we briefly review the background to the leaky homes crisis and Layton's analysis of the crisis.

A12.1 Causes of the crisis

As a brief background, the leaky homes crisis is often attributed to the 1991 switch from prescriptive building codes to performance-based regulation that allowed innovation in building techniques. While the change in regulation permitted more rapid innovation, there were also several other factors that contributed to the severity of the leaky homes crisis. These factors included a lack of detailed evidence-based understanding about the vulnerability of building systems to moisture damage, low awareness of overseas experience from building innovation, loss of institutional memory about past cladding failures, and a slow response to evidence that new designs were not meeting regulatory performance objectives.

The aim of the 1991 building reforms was to: encourage innovation in the building industry through the adoption of a performance-based building code; drive improvement in the service provided by local authorities; and provide a more coherent building regime.[58]

One of the innovations was to construct houses in a Mediterranean style with untreated framing timber. The choices of Mediterranean-style design (including monolithic cladding) and untreated framing timber were made at separate times as independent decisions for different reasons. Rather, than a single clear-cut change, building design and practice drifted through a series of gradual changes that progressively eliminated the lines of defence that made conventional houses resistant to moisture damage and delivered the standard of weathertightness that stakeholders had come to expect. These decisions seem to have been based on expectations rather than evidence of their potential effects on weathertightness.

The leaky homes crisis developed from a result of a series of decisions that were affected by the factors listed above. A high-level timeline of the evolution of the crisis is shown in the following table.

Table 12.1: Leaky Buildings[59]

Table 12.1: Leaky Buildings

[*] Economist, New Zealand Institute of Economic Research.

[§] Senior Economist, New Zealand Institute of Economic Research.

[#] Senior Fellow, New Zealand Institute of Economic Research.

[1] Brent Layton "Regulating the Building Industry – A Case of Regulatory Failure" in Susy Frankel (ed) Learning from the Past, Adapting for the Future: Regulatory Reform in New Zealand (LexisNexis, Wellington, 2011) 311.

[2] Peter May Performance-Based Regulation and Regulatory Regimes (Paper prepared for the Global Policy Summit on the Role of Performance-Based Building Regulations in Addressing Societal Expectations, International Policy, and Local Needs, Washington (DC), November 2003) at 1, available at <>.

[3] See Peter May "Performance-Based Regulation and Regulatory Regimes: The Saga of Leaky Buildings" (2003) 25(4) Law & Policy 381 at 383: May quotes as examples a survey by the National Association of Homebuilders in 1998 that 10 per cent of the cost of building a typical new home (in the United States) were attributable to unnecessary regulation, regulatory delays and fees, and other studies that argue that the inflexibility of prescriptive regulations limit innovation or constrain international competitiveness.

[4] Peter May "Performance-Based Regulation and Regulatory Regimes: The Saga of Leaky Buildings" (2003) 25(4) Law & Policy 381 at 382.

[5] Peter Mumford Enhancing Performance-Based Regulation: Lessons from New Zealand's building control system (Institute of Policy Studies, Wellington, 2011) at 11, available at <>.

[6] See Government Administration Committee Weathertightness of Buildings in New Zealand (Forty-seventh Parliament, March 2003) at 28, available at <>: "BRANZ states that building forms and materials have evolved rapidly since the mid-1990s, as part of an almost 'exponential growth in variety' over the last 10 years. Design forms for urban densification and upmarket housing have become increasingly complex without there being much experience of the design and building of these types of buildings in New Zealand."

[7] See HM Treasury "Supplementary Green Book Guidance: Optimism Bias" (United Kingdom, 2002), available at <> for an estimate of the adjustments suggested to compensate for optimism bias in large-scale procurement projects; see also Tali Sharot "The Optimism Bias" (2011) 21(23) Current Biology R941 for a discussion of the biological explanation for optimism bias.

[8] This framework is adapted from Peter May "Performance-Based Regulation and Regulatory Regimes: The Saga of Leaky Buildings" (2003) 25(4) Law & Policy 381. In particular, it has been augmented by the final row, to address the issues of concern here.

[9] Monitoring regulation is an important cross-cutting theme in this project and will be the topic of a paper that will be published in 2013. See Derek Gill and Susy Frankel "Learning the Way Forward: The Role of Monitoring, Evaluation and Review" (forthcoming).

[10] For an intital discussion see Charles Kolstad, Thomas Ulen and Gary Johnson "Ex Post Liability for Harm vs. Ex Ante Safety Regulation: Substitutes or Complements?" (1990) 80(4) The American Economic Review 888 at 888.

[11] Richard Robinson and Gaye Francis Engineering Due Diligence in James Robinson (ed) Risk and Reliability – Engineering Due Diligence (8th ed, Risk & Reliability Associates Pty Ltd, Melbourne, 2010).

[12] Peter Mumford Enhancing Performance-Based Regulation: Lessons from New Zealand's building control system (Institute of Policy Studies, Wellington, 2011) at 133, available at <>.

[13] See Elizabeth Eppel, David Turner and Amanda Wolf "Complex Policy Implementation: The Role of Experimentation and Learning" in Bill Ryan and Derek Gill (eds) Future State: Directions for Public Management in New Zealand (Victoria University Press, Wellington, 2011) 182 at 195. The authors make the following comment: "The understanding of the problem formed through the traditional means of policy analysis is treated only as a tentative hypothesis that requires testing in the real world outside the government agency. Testing takes the form of actions with collaborators such as individuals and community groups."

[14] Peter May "Performance-Based Regulation and Regulatory Regimes: The Saga of Leaky Buildings" (2003) 25(4) Law & Policy 381 at 382.

[15] See Government Administration Committee Weathertightness of Buildings in New Zealand (Forty-seventh Parliament, March 2003) at 22, available at <>.

[16] Bruce Anderson and Lynne Dovey Whither Accountability? (Working Paper No 18, State Services Commission, September 2003) at 14 (footnotes omitted), available at <>.

[17] Peter May "Performance-based Regulation" in David Levi-Faur Handbook on the Politics of Regulation (Edward Elgar, Cheltenham, 2011) 373 at 380.

[18] Peter May "Performance-based Regulation" in David Levi-Faur Handbook on the Politics of Regulation (Edward Elgar, Cheltenham, 2011) 373 at 380.

[19] Peter May "Performance-based Regulation" in David Levi-Faur Handbook on the Politics of Regulation (Edward Elgar, Cheltenham, 2011) 373 at 381.

[20] Peter May "Performance-based Regulation" in David Levi-Faur Handbook on the Politics of Regulation (Edward Elgar, Cheltenham, 2011) 373 at 381.

[21] Elizabeth Eppel, David Turner and Amanda Wolf "Complex Policy Implementation: The Role of Experimentation and Learning" in Bill Ryan and Derek Gill (eds) Future State: Directions for Public Management in New Zealand (Victoria University Press, Wellington, 2011) 182 at 195.

[22] See Asher Wolinsky "Competition in a Market for Informed Experts' Services" (1993) 24(3) RAND Journal of Economics 380. Wolinsky is dealing with the problem of using experts who can only be assessed by other experts.

[23] See Government Administration Committee Weathertightness of Buildings in New Zealand (Forty-seventh Parliament, March 2003) at 15, available at <>.

[24] See [12.2.2] above.

[25] The particular confluence of features is generally considered to be the monolithic cladding, untreated timber, and lack of a drainage cavity behind the cladding.

[26] Department of Building and Housing Weathertightness – Estimating the Cost (Report prepared by PricewaterhouseCoopers, 29 July 2009) at 3, available at <>.

[27] See Peter Mumford Enhancing Performance-Based Regulation: Lessons from New Zealand's building control system (Institute of Policy Studies, Wellington, 2011) at 77, available at <>.

[28] These standards related to requirements, for example, for resistance to moisture penetration and structural integrity. The main clauses in the building code in relation to cladding are B2 "Durability" and E2 "External Moisture", see Government Administration Committee Weathertightness of Buildings in New Zealand (Forty-seventh Parliament, March 2003) at 117–118, available at <>.

[29] See Peter May "Performance-Based Regulation and Regulatory Regimes: The Saga of Leaky Buildings" (2003) 25(4) Law & Policy 381; Peter Mumford "Best Practice Regulation: Setting Targets and Detecting Vulnerabilities" (2011) 7(3) Policy Quarterly 36.

[30] Experimentation in regulation will be discussed further in this project in a paper to be published in 2013. See Joel Colón-Riós "Experimentation and Regulation" (forthcoming).

[31] Government Administration Committee Weathertightness of Buildings in New Zealand (Forty-seventh Parliament, March 2003) at 15, available at <>.

[32] Examples of this approach include not considering how kiln-dried framing timber would be affected if cladding leaked, accepting cladding designs that reduced the opportunity for the wall cavity to dry or drain without considering the consequences of trapping moisture in the wall cavity and the lack of appreciation of the rot prevention properties of boron treatment of timber.

[33] "My final conclusion is that the regulatory regime did not fail because of weaknesses in the building sciences and risk assessment. It failed because there was [no] early detection that there might be a problem, and a timely and appropriate response to relevant information that did become available, which led to a large number of buildings failing with significant consequences": Peter Mumford Enhancing Performance-Based Regulation: Lessons from New Zealand's building control system (Institute of Policy Studies, Wellington, 2011) at 82, available at <>. See also the comments made on the precautionary approach taken in British Columbia and North Carolina.

[34] See New South Wales Government "Tradespeople: Home Warranty Insurance" (June 2012) <> for a description of the cover offered under the New South Wales Home Warranty Insurance scheme. Since 2010 this scheme has been operated by an entity owned by the New South Wales State Government. The scheme cover for loss arising from defective work is provided for a period of six years from the date of completion of the work or the end of the contract for the work (whichever is later) for loss arising from a structural defect, and two years for loss arising otherwise than from a structural defect.

[35] See Peter Mumford Enhancing Performance-Based Regulation: Lessons from New Zealand's building control system (Institute of Policy Studies, Wellington, 2011), available at <>.

[36] See Brent Layton "Regulating the Building Industry – A Case of Regulatory Failure" in Susy Frankel (ed) Learning from the Past, Adapting for the Future: Regulatory Reform in New Zealand (LexisNexis, Wellington, 2011) 311.

[37] Mathematically, the risk impact is Sp.L, where p is the expected probability, and L the consequence, in the particular state, and the summation is over all possible states. See NZ Standard AS/NZS 4360:1999.

[38] Brent Layton "Regulating the Building Industry – A Case of Regulatory Failure" in Susy Frankel (ed) Learning from the Past, Adapting for the Future: Regulatory Reform in New Zealand (LexisNexis, Wellington, 2011) 311 at 336.

[39] See Peter Mumford Enhancing Performance-Based Regulation: Lessons from New Zealand's building control system (Institute of Policy Studies, Wellington, 2011) at 99, available at <>.

[40] Peter May "Performance-based Regulation" Jerusalem Papers in Regulation & Governance, Working Paper No 2, April 2010, at 16.

[41] See [12.2.2] above.

[42] Monitoring is part of the "experimental" side of all regulations in Mumford's model, see [12.6.4] below.

[43] See also the discussion of regulating consumer choices in Graeme Austin "The Regulation of Consumer Credit Products: The Effects of Baseline Assumptions" in Susy Frankel (ed) Learning from the Past, Adapting for the Future: Regulatory Reform in New Zealand (LexisNexis, Wellington, 2011) 295.

[44] For an example of light-handed regulation see Paul Scott "Competition Law and Policy: Can a Generalist Law be an Effective Regulator?" (ch 5) in this volume.

[45] George Akerlof "The Market for 'Lemons': Quality Uncertainty and the Market Mechanism" (1970) 84(3) The Quarterly Journal of Economics 488.

[46] For discussion on the "precautionary approach" see Graeme Austin "The Regulation of Consumer Credit Products: Interrogating Assumptions about the Objects of Regulation" (ch 8) in this volume.

[47] Peter Mumford Enhancing Performance-Based Regulation: Lessons from New Zealand's building control system (Institute of Policy Studies, Wellington, 2011) at 153, available at <>.

[48] Experimentation in regulation will be discussed further in this project in a cross-cutting theme paper to be published in 2013. See Joel Colón-Riós "Experimentation and Regulation" (forthcoming).

[49] Office of the Minister for Building and Construction Building Act Review: Review of Joint and Several Liability (Prepared for Cabinet Economic Growth and Infrastructure Committee, 2011) at 7.

[50] Office of the Minister for Building and Construction Building Act Review: Review of Joint and Several Liability (Prepared for Cabinet Economic Growth and Infrastructure Committee, 2011) at 7.

[51] Note the fascinating counterexample of the post-earthquake Christchurch and the EQC. This is not a typical socialisation of loses: the coverage has always been confined to the insured population (as its "funding" was a levy on insurance premiums). So far, there has been no breach of the line. Compare this with AMI below.

[52] See Marta Steeman "Quake-hit AMI Insurance Bailout Could Cost $1 Billion" (New Zealand, 7 April 2011) <>.

[53] The scheme covers homeowners against the risk that the builder will not complete the contracted work due to illness, death, disappearance or insolvency of the builder and also covers the homeowner against defective workmanship.

[54] The initial shock to the reinsurance market was the collapse of HIH in 2002 – a local problem that made reinsurance much harder to obtain. The more recent withdrawal of private insurers is probably related to the global financial crisis encouraging insurers and reinsurers to withdraw from less profitable markets.

[55] See Peter May "Performance-based Regulation" in David Levi-Faur Handbook on the Politics of Regulation (Edward Elgar, Cheltenham, 2011) 373 at 380.

[56] Department of Building and Housing Weathertightness – Estimating the Cost (Report prepared by PriceWaterhouseCoopers, 29 July 2009), available at <>.

[57] Brent Layton "Regulating the Building Industry – A Case of Regulatory Failure" in Susy Frankel (ed) Learning from the Past, Adapting for the Future: Regulatory Reform in New Zealand (LexisNexis, Wellington, 2011) 311.

[58] Peter Mumford Enhancing Performance-Based Regulation: Lessons from New Zealand's building control system (Institute of Policy Studies, Wellington, 2011) at 11–12, available at <>.

[59] Parliamentary Library Leaky Buildings (Background Note, 2002/10, 6 November 2002) at 18; see also Andrew Laxon "Where the Rot Really Set In" The New Zealand Herald (New Zealand, 10 October 2002).