Why no climate warranties?

The market for climate risk forecasts suffers from a problem economists call “asymmetric information”: the sellers of forecasts probably know more about their accuracy than the buyers. A short or medium range weather forecast can be verified in a couple of weeks, and reasonable statistics about their accuracy can be accumulated in a matter of months but, because a climate forecast might not verify for years or decades, buyers cannot usually use a forecaster’s track record to assess their accuracy. A market in which the quality of a product won’t be apparent for years is susceptible to “adverse selection”, which is economists’ way of saying it will be a magnet for charlatans. This creates a challenge for providers of climate forecasts who are competent and have invested in research and development to improve the accuracy: how can they persuade buyers that they are not charlatans?

“Talk is cheap.”

Climate service providers can emphasise their credentials and highlight the scientific effort that goes into their products; they might allow users to see how their models work — although many users don’t have the expertise to assess the models, and if the seller uses proprietary models this might not be option anyway. The problem with these solutions is that “talk is cheap”, and it is difficult for a buyer to distinguish authentic expertise from slick marketing.

The economist George Akerlof, who won a Nobel Prize for his work on information asymmetries, explained the problem in a classic paper published in 1970 which described the market for used cars. As with climate forecasts, it is difficult for the buyer of a used car to know its quality and, also like climate forecasts, a car is an infrequent purchase so there is little opportunity to accumulate much first-hand knowledge about the reliability of a seller. These attributes made the market for used cars attractive to less-than-reputable characters, creating the stereotype of the untrustworthy used-car dealer.

“Dealers offered warranties to differentiate themselves from competitors.”

The reputation of used-car dealers has improved since the 1970s, partly because the sector adopted one of the strategies for dealing with asymmetric information described by Akerlof: offering cash back warranties. In the U.K. used-car warranties became mandatory in 2015, but even before then, dealers offered warranties to differentiate themselves from competitors and enable them to charge higher prices.

A promise of a refund if a product fails can serve two distinct purposes: indemnifying the buyer for losses caused by the failure and providing a credible signal that the seller believes failure is unlikely. An inaccurate climate forecast may cause the user a loss far above the price they paid for the forecast: for example, they may build sea defences that are too low, or too high. While a forecast provider might not be able to indemnify the user against all costs, they can still offer a refund to signal their confidence in the accuracy of the forecast. The credibility of the signal is determined, not by whether the refund will compensate the user, but whether it will be painful for the provider.

Climate forecasts, generally, are not categorically right or wrong: they are probabilistic forecasts — or they should be — in which probabilities are assigned to specified outcomes. How could the terms of a refund be structured for this type of product? There is a literature on this subject going back 70 years, when Glenn Brier introduced a quadratic scoring rule for assessing probability forecasts and I.J. Good suggested the logarithmic scoring rule as a way to reward forecasters. Since then more work has been done on the concept of “proper scoring rules” for incentivizing forecasters. More recently, proper scoring rules have been suggested as the basis for incentives that screen the views of informed and uninformed experts when only a single forecast is being made. There are subtleties to be appreciated, and pitfalls to avoid, but in principle a proper scoring rule could be used to evaluate the quality of a probabilistic forecast and determine the refund to which the buyer is entitled. This can only be done when the actual outcome becomes known, which in some cases may be more than twenty years after the forecast is made. Some people might think that providing a warranty on a timescale of decades is unrealistic but other industries do it: e.g., 10-year structural warranties for new houses or the lifetime warranties offered by high-end manufacturers that can be claimed decades after a product is purchased. It is also analogous to deferring bonuses in the finance industry. Climate service providers unwilling to expose themselves to some risk on climatic timescales are arguably in the wrong business.

“A well-structured warranty can be used to communicate how much confidence a forecast provider has.”

The information asymmetry in the market for climate forecasts is at least as bad as the one for used cars, so why aren’t warranties commonplace? There is no need for a regulator to force forecasters to provide warranties. As with used-cars, any provider is free to unilaterally offer warranties to differentiate themselves. One possibility is that few climate service providers have enough confidence in their forecasts to give warranties. However, a probabilistic forecast combined with a well-structured warranty can be used to communicate how much (or little) confidence a forecast provider has, and this is information that the forecast user should want. An alternative explanation is that there is little demand from buyers for sellers to provide a credible signal of their confidence in forecast accuracy. This interpretation is consistent with the view that buyers of climate risk information use it to satisfy regulators, auditors, or other stakeholders, but aren’t really worried about its accuracy; they just need it to give them “plausible deniability”. Worse than indifference, is the possibility that some organisations want inaccurate forecasts. Back in the early 2000s, the issuers of mortgage-backed securities didn’t want accurate credit ratings for them, they wanted top-notch ratings, even if these did not reflect the true risk of default. These issuers would play rating agencies off against each other to get their desired ratings, and the ensuing proliferation of mis-rated securities contributed to the subprime crisis of 2007/08.

End-users of climate risk forecasts who care about accuracy should signal their commitment by being prepared to pay more for forecasts that come with a warranty. This will motivate reputable providers to offer such guarantees, thus differentiating themselves from less able providers. Without a strategy to combat asymmetric information about climate forecasts it is possible that climate service providers will gradually acquire the reputation of 1970s used-car dealers.

This article originally appeared on Medium as Why no climate warranties?