In California’s recent record-breaking heat, a major provider of demand response to the state’s wholesale market learned why the system operator is about to change the compensation rules.

The tricky thing about demand response (DR) compensation is what policymakers call “the counterfactual.” To know how much the DR reduced load, the system operator estimates what the load would have been without the DR. If that estimate is not accurate, the DR value proposition, and therefore its competitiveness, is compromised.

North American (primarily U.S.) DR adoption is projected to be 49.3 GW by 2025, according to Navigant Research. A recent GTM Research survey of operational DR found 9.1 GW of the surveyed 17 GW in service were monetized in wholesale markets. With almost 2,500 MW, the California Independent System Operator (CAISO) had the third biggest share.

'Deal killer'

The settlement mechanism in California’s market “is a deal killer,” said Susan Kennedy, CEO of leading CAISO DR provider Advanced Microgrid Solutions (AMS). “If you don’t have a metered battery that actually measures output, the only thing to measure against is the counterfactual, what the load was in the last 10 days. It’s called the 10 of 10 baseline.”

Remuneration to AMS’s DR during California’s recent record-breaking heat wave revealed the CAISO settlement mechanism’s failure, Kennedy said. “AMS delivered 100% of what it was asked to deliver and they’re only going to recognize 53% of what we delivered because the baseline was so much lower.”

Jill Powers, CAISO infrastructure and regulatory policy manager, said the settlement mechanism was put in place in 2010, before there was any DR on the system. It was designed to allow non-generating resources like DR to participate in energy and ancillary services day-ahead and real-time markets, like other generators.

For generation, “we normally have a meter and can settle based on the meter reading,” Powers said. For the DR in the ISO’s proxy demand resource and reliability demand response resource programs, “we have established how we will estimate what that performance was.”

During the heat wave, the ISO’s utility DR programs “were evaluated economically” to meet market demand, Powers said.

But, as the Navigant study pointed out, “since 2010, DR has matured in the electricity market.” It now includes not just load reduction-based services but battery-based distributed energy resources systems like those provided by AMS.

Even before the heat wave, CAISO stakeholders realized that DR's maturity and new technologies require a new settlement mechanism. An initiative led by two of the state’s major investor-owned utilities (IOUs) offers major changes. But stakeholders like Kennedy say the changes don’t go far enough.

The CAISO fix

CAISO’s Baseline Analysis Working Group (BAWG) was led by Southern California Edison (SCE) and San Diego Gas and Electric (SDG&E). It included Pacific Gas and Electric (PG&E), Tesla, the California Large Energy Consumers Association and the California Energy Storage Alliance, among others.

Nexant wrote the final proposal, which now goes to the Federal Energy Regulatory Commission (FERC) for approval.

The BAWG identified three approaches to new settlement mechanisms. One was “alternative but traditional baseline methods to estimate the load impacts” of DR. Another was “using control groups rather than traditional baselines to estimate the load impacts.” The third was a way to “accurately measure load impacts.”

In the 2010 mechanism, “we look backwards from the day of the DR dispatch to establish a baseline,” Powers said. Iterating backwards, the CAISO looks for 10 similar non-event days and then averages them, she said. 

Any day on which something occurs that atypically affects load, such as a DR event or an outage, is excluded, she went on. “We are trying to look for an average baseline of what the load looked like.”

There is also an adjustment, called a load point adjustment, used if load was atypical on the day of the event. Currently, the adjustment can be 20% up or down from the averaged baseline. It prevents the event day’s atypical load from having too large an impact on settlement.

The market transaction is with the wholesale supplier, Powers said. That could be an IOU with DR resources, an IOU acting as a scheduling coordinator, or a third party DR provider.

The BAWG’s final report was part of Phase 2 of the CAISO’s Energy Storage and Distributed Energy Resources (ES-DER) initiative, Powers said. Because the group was utility-led, it had access to actual utility customer data and was able to make determinations about settlement methodologies with a new level of precision.

Control groups

In control group methodologies, electricity use of a group of customers not participating in the DR event is compared to that of participating customers. Based on control groups used by other system operators, the proposal suggested using either a randomized controlled trial or a matched control group “of non-participants with similar characteristics to participants.”

A second settlement mechanism would use “weather-matching,” Powers said.

This baseline calculation methodology estimates what load would have been without DR dispatch, using only “electricity use data for customers who were dispatched,” the BAWG proposal reports. It compares their usage for “a subset of non-event days with the most similar weather conditions” with their use on the event day.

The third proposed settlement mechanism is through new “day-matching baselines,” Power said. Like the current methodology, it applies the 10 of 10 counterfactual estimate to commercial-industrial customers. But it offers a 5 of 10 estimate for residential customers on weekday event days and a 3 of 10 estimate for weekend event days. Those take the highest use 5 or 3 days of the 10 days selected in the backwards iteration.

CAISO will also offer its current settlement mechanism to DR providers who prefer it, Powers said.

The BAWG also proposed increasing the maximum load point adjustment from 20% to 40% to better account for atypical event days like those that happen during a heat wave, Powers said.

The CAISO Board of Governors has approved the new methodologies, she said. If FERC approves, CAISO hopes to implement them in Spring 2018.

“The DR provider will select one of the four options when the resource is registered in the CAISO system,” Powers said. “All wholesale suppliers, market participants, and scheduling coordinators will know what the settlement will be for each option, based on the performance evaluation in place, and will have to decide which best fits their resource.”

Utilities endorse, raise questions

SCE’s BAWG filing endorsed the final proposal.

SCE spokesperson Robert Laffoon Villegas said research has shown the current baseline “to be accurate for many medium and large commercial-industrial customers, but not accurate for all customer types, particularly those with weather-sensitive demand.”

Changing the calculation of the baseline to the Board-approved BAWG proposals addresses this issue, he said. But if this inaccuracy in the settlement is not rectified, “SCE will need to evaluate the financial impacts of continuing to offer weather-sensitive demand response into the CAISO market.”

PG&E Principal Strategic Analyst Gil Wong agreed that DR portfolios with weather-sensitive customers reveal the need for new baseline calculation methodologies. “The extent of undervaluation of a DR portfolio depends on how much of it is made up of weather-sensitive customers.”

The utility is “evaluating the implications of the new wholesale baselines on retail settlement and considering how to revise the retail baselines,” he added.

PG&E’s filing addressed other “key questions” for the ES-DER initiative's implementation phase.

Providers should not be able to update their baselines “more frequently than monthly (in alignment with the ability for a control group to be updated monthly),” the filing recommended.

More frequent updating could result in “unintended implementation consequences,” PG&E argued. The process could become complicated if a DR provider finds a selected settlement mechanism does not accurately capture performance or impacts development costs.

Market participants should also be required to select a baseline in advance, PG&E argued. “If participants are not required to do so, they could change their baseline after the fact, leading to gaming concerns if the later baselines that are selected are more advantageous for the DR participant.”

Finally, PG&E recommended a “near term” implementation phase because of anticipated regulatory hurdles.

Stakeholders endorse and look ahead

Matthew Tisdale, formerly California Public Utilities Commission (CPUC) Distributed Energy Resources Policy Head and now Executive Director of California think tank More Than Smart, endorsed the BAWG proposal.

Market integration of DR is a priority for the state, he said. “It is critical to get the details of integration right and one of those details is baseline methodologies that measure what would have happened otherwise.”

The proposal’s methodology options are valuable because “different approaches may be necessary to accommodate different resources,” he said.

A baseline for customers who respond to a DR event by lowering their thermostats can be set by comparing them with a control group, he said. But for a customer with a unique application like a big battery connected to a big industrial process, there is no control group.

“The ISO working group’s different methodologies for different kinds of resources with different characteristics is a better approach,” Tisdale said.

AMS’s Kennedy said DR compensation is “the seminal question.” But, she argued, AMS can determine it more accurately than the new settlement mechanisms allow for.

“They are still too focused on what the behavior was beforehand,” she said. “The battery meter measures how much load the battery is carrying and it can’t carry a load that’s not there. Every kWh the battery is carrying is load dropped from the grid.”

AMS’s Demand Response Auction Mechanism (DRAM) contracts with SCE, like the DRAM contracts the IOUs have with Stem and Green Charge Networks, require DR to be "transactable in the wholesale market," she said.

Overpayment, underpayment

With counterfactual-based mechanisms “either SCE will overpay AMS for a resource it can’t get paid for in the wholesale market or AMS will be underpaid for delivering it,” she has concluded. “AMS can’t build unless it gets paid for what it delivers because the margins aren’t there.”

Either way, it “a golden reason for utilities not to buy distributed storage, or distributed solar plus storage, or any kind of distributed technology,” she said.

This is slowly being rectified in regulatory proceedings and CAISO initiatives because “this is an irreversible march,” Kennedy said. "But utility planners, developers, customers, regulators, and CAISO need to work toward the solution.”

Many stakeholders backed Kennedy’s arguments for more precise performance measurements.

A joint filing from AMS, SolarCity, and Stem in the ES-DER Phase 2 initiative urged “metering and settlement for less than 24-hour periods” for non-generating resources.

A California Large Energy Consumers Association filing endorsed an SCE proposal with the CPUC for reprograming meters to "increase the accuracy of the settlement calculation." Non-residential customers in DR programs could move “to 5-minute data” and residential customers could move “to 15-minute data,” the association said.

A filing from eMotorWerks, which supports electric vehicle (EV) smart charging for grid service, said a metering option offers more DR options to more types of DR providers. It also opens the opportunity to aggregators of EV batteries.

An early Tesla filing refuted CAISO doubts about metering but recognized estimation methodologies might be a necessary interim step. Phase 3 of the ES-DER initiative could “build on this previous step by fully recognizing and developing a mechanism for compensating the ability for behind the meter storage to export energy during DR events.”

The BAWG proposed using the current 10 of 10 baseline methodology to estimate 5-minute intervals for settlement but went no further on metering.

More Than Smart's Tisdale recognized that this initiative might not be the complete and perfect fix. “This is the cutting edge of DER participation in markets and there are likely to be growing pains,” he said. “But if parts of the methodologies do not work in extreme conditions like the recent heat wave, they should be addressed because extreme weather conditions like heatwaves are not going away.”