Decentralizing the Grid

Technological and institutional change required

The impetus for change in monopoly electric systems has ebbed and flowed for over three decades. Over the past 15 years the interest in and ability to update electricity system technologies has grown, due to the combination of changing policy objectives and widespread digital innovation. Whether you call it smart grid or grid modernization, digital technologies by now have long had the potential to transform electric systems, improving their operations, reducing waste and idleness, and (but I repeat myself) having more market-based systems. Separate but complementary improvements in the performance and production costs of distributed energy resources like solar PV, electric vehicles, and battery storage, along with the policy focus on decarbonization, have amplified interest in and work on digitalization. Digitalization has reduced transaction costs and created unforeseeable types and amounts of value in the rest of the economy. Shouldn't it do so too in electricity?

Source: U.S. Department of Energy, Smart Grid System Report (2022)

But it’s a slow process in such a large, capital-intensive, regulated industry, in contrast to the more nimble and entrepreneurial technology innovation that has created our digital economy in a surprisingly short amount of time. As a researcher writing about technological change and dynamic pricing in electricity since the early 2000s and a participant in initiatives (such as the GridWise Architecture Council) to reduce barriers to innovation in electricity systems, this thinking frames my reading of a very useful analysis from Ryan McEntush at the venture capital firm Andreesen Horowitz.

McEntush argues that to meet the expectations of greater demand alongside decarbonization initiatives, grids have to become more decentralized; to meet demand effectively, including cost-effectively, that decentralized grid must also be digitalized.

To seize a future of energy abundance, we must simplify the generation, transmission, and consumption of electricity; this entails decentralizing the grid. Big power plants and long power lines are burdensome to build, but technologies like solar, batteries, and advanced nuclear reactors present new possibilities. It will be these, and other more “local” technologies, that can circumvent costly long-haul wiring and be placed directly on-site that will help support significant load growth over the coming decades.

He argues that the grid is and should be decentralizing to take advantage of the benefits of new technologies and avoid costs of older ones. In other words, the grid architecture must change as its components change.

This observation hinges on an important point: the newer, smaller scale technologies that are more amenable to a decentralized architecture have qualitatively different features from the traditional electricity technologies. We are interested in adopting more decentralized components like DER (and are being pushed in that direction by government policies) to the extent that they provide consumers with more value, provide system operators with better tools, and/or provide technological alternatives that are more compatible with policy objectives.

The set of energy generation technologies, as a portfolio, contains technologies with different combinations of attributes — some are large scale and have lower financial (capital and operating) costs but higher pollution and greenhouse gas emissions, some are smaller scale and have lower emissions but are not always available, some are inexpensive to schedule/dispatch and some are not, some are expensive to start up or ramp up and some are not. Every single energy technology has pros and cons. Every. Single. One. There are no solutions, only tradeoffs, especially in an energy transition that is going to take decades.

The material question here is whether these newer technologies that operate in a more decentralized system have combinations of attributes to provide that set of [consumer value, operator tools, government policy] more effectively than continuing to rely solely on the older technologies.

Answering that question is hard. Really hard. Hard not just because electricity systems are such complex systems with so many components, but also because they are infrastructure systems that we all participate in together, so decisions are made collectively. That means collective action problems including free riding, concentrated benefits and diffuse costs, rent seeking, and regulatory capture. The fact that grids are socio-technical systems means that these complexities come in interacting layers, human uses and human organizations on top of physical networks.

McEntush then goes on to survey the challenges, both physical and financial, of grid expansion; building new transmission is challenging and takes a long time. Without investment in that transportation system, and because most interconnection rules are not conducive to innovation (except for Texas), new generator/resource interconnection queues have stretched towards a decade in some regions. His discussion of the interconnection issues is great and full of informative links, as is his discussion of supply chain problems for essential equipment like transformers, and of changes at the distribution level that I’ll probably expand on another time. Even if I didn’t agree with his overall analysis I’d recommend reading this piece, just for the extensive links to other valuable resources and analyses.

The punch line of his analysis is smart grid: “As electricity demand grows, the grid struggles to manage growing complexity from both decentralization and intermittent renewables. We cannot brute force this shift; if we’re going to do it, we really need to get smart.” There’s also an implicit point in there that we cannot and should not go back to centralization and old generation technologies. I probably would make a stronger case than he would for the persistence of natural gas in the generation portfolio, but even that is smaller scale than other traditional generation technologies; natural gas is a topic for another day.

Again he makes the architecture point forcefully; if existing systems are insufficient to meet changes in the magnitude and nature of demand, energy-intensive industries will invest in low-carbon self-generation (and have already been doing so for most of the past decade). Those investments will decentralize grid architecture.

To be clear, the centralized grid we have today will certainly not disappear – in fact, it also needs to grow in size – but it will be consumed by networks of decentralized assets growing around it. Ratepayers will increasingly adopt self-generation and storage, challenging traditional electricity monopolies and prompting regulatory and market reform. This self-generation trend will reach its extreme in energy-intensive industries that especially prioritize reliability – Amazon and Microsoft are already pursuing nuclear-powered data centers, and we should do everything we can to accelerate the development and deployment of new reactors.

I agree with this analysis; indeed, I’ve spent much of the past two decades analyzing the aspects of utility regulation that are barriers to such innovation and suggesting ways to reduce those barriers.

Not surprisingly, a tech-focused VC firm is going to focus on the types of projects and investments that need to be accomplished to transition to a more decentralized grid:

Critically, to build this decentralized grid demands our most talented entrepreneurs and engineers: We need a “smart grid” with serious innovation across ahead-of-the-meter, behind-the-meter, and grid software technology. Policy and economic tailwinds will accelerate this electricity evolution, but it will fall to the private sector to ensure this decentralized grid works better than the old one.

McEntush’s analysis of the current state of technology, the evolution toward decentralized electric systems (plural), and the essential role of digitalization and smart grid is excellent, but incomplete. His analysis does not reflect the crucial role that institutions, and institutional change, play in the types of system-level transitions he is analyzing. It's not just making sure the decentralized grid works better than the old one, it's whether or how the decentralized grid and its attendant benefits get to emerge at all. He mentions the challenge when he observes that “... [r]atepayers will increasingly adopt self-generation and storage, challenging traditional electricity monopolies and prompting regulatory and market reform.”

That process will not be easy. Or short. Or certain.

If this were a greenfield, from scratch, brand-new industry, the focus on entrepreneurship and engineering would suffice. But this is a 130-year-old “iron in the ground” infrastructure industry, and the kind of technological and architectural changes that digitalization and decentralization entail are a complicated process of change, working on updating the plane's engines while it’s in flight. It’s also a socio-physical system of machines and humans, firms and regulators and consumers and communities, all of whom have some entrenched status quo interests and expectations for their relationship to the electric system. Protecting those interests slows innovation, and for some utility executives and regulators (and others) in the industry, slowing innovation is a feature, not a bug. To deliver on the potential economic and environmental value of these new technologies and new system architectures requires organizational and social change as well, and the political will to reduce barriers to innovation.

It’s more than a process of technological change; it’s also necessarily a process of institutional change. Technologies and institutions are co-determined.

In a recent interview with Cipher's Cat Clifford, Jigar Shah highlighted that point. Shah, a former solar power entrepreneur and currently the head of the U.S. Department of Energy's Loan Programs Office, recognizes that such large, systemic change is both technological and institutional:

I asked Shah why old transmission lines aren’t being replaced by advanced, more efficient wires (which I recently wrote about here).

“Culture and norms,” Shah told me. Faced with a new technology or innovation, a utility or regulator might respond that they don’t think a new technology is proven or dependable enough, Shah said. But sometimes pushback may be due to a lack of a familiarity with what’s new, he said, eliciting a vague and conservative response from a utility or regulator, like: “‘It’s just not something that our grid operators like to do." ...

Decision makers are often rewarded for operating defensively, accustomed to thinking if they deploy a new technology and it doesn’t work “perfectly” they could face consequences, Shah said. Ergo, the status quo marches on.

“We’ve got to figure out how to give people protection so that they can do the right thing,” Shah told me. One way the Energy Department can do that is by providing validation and case studies giving utilities and regulators confidence a new technology works, he said.

Institutions fall into two categories: formal rules and culture. In this industry both formal rules and culture dissuade entrepreneurship and investing in new technologies, encouraging people to resist change. The dominant formal rules shaping incentives are the existing cost-based rate-of-return (ROR) regulations that lead utilities to propose more capital intensive investments and to lobby to deter others from any business activity that could substitute for the service over which they have the legal monopoly. In his interview Shah did not comment on the perverse incentives that ROR regulation presents to utilities (and to regulators, for that matter), nor would I expect him to touch that third rail.

Regulation is the dominant institution shaping incentives of regulators and regulated, but do not dismiss culture. As a whole this industry — utilities, regulators, transmission and market operators, consumers, communities — shares a culture of caution and risk aversion, both physical and financial. This risk aversion is understandable, as everyone also shares an interest in keeping the lights on cost-effectively. It’s less risky for utilities to propose and regulators to approve “iron in the ground” investments that are well-known, even when the world is digitizing around them and they face incredible pressures to change.

It’s also a challenge to make the argument of going through the difficult technological and institutional change processes in an infrastructure industry and set of systems, which means collective decision-making in a pluralist setting where people have different perspectives and experiences and reach different conclusions about what the best change paths to try are, let alone whether or not they are even worth doing. Returning to McEntush’s brief, delivering on the technological and institutional changes that make up the energy transition will be orders of magnitude more complicated than even a particularly difficult tech startup.

Here's where McEntush’s perspective is crucial: the more creativity, innovation, and entrepreneurship that is attracted to developing and commercializing technologies in this future energy ecosystem, the more attractive that future will be, the less risky it will seem, and the easier it will be to achieve consensus and change in these complex collective-action settings.