Part of the problem in modeling CA's loads comes from the fact that their "deregulation" was this mish-mash of nonsense. Large industrial loads were given "deregulated" power pricing, where they can see spot prices for power fluctuate rather significantly in the afternoons when power becomes scarce. So some of the "deregulated" customers will drop their loads on some days, and not on other days - they're making a local decision about how much power they use without consulting with the utility. It it up to the utility to figure out on which days the cost of power exceeded some threshold for their various industrials customers.

Then we have the residential customers, who were prevented from seeing the pass-through of "deregulated" pricing - who are gobbling up ever more power for A/C units all the time.

Then added into this mix was the changes in how transmission was priced and how various transmission paths would become "congested" with too much power offered (and subsequently re-routed or offer withdrawn) while the demand was still in place.

The whole net:net effect exceeded the ability of CA-ISO to really predict their loads there for awhile. They've effectively backed out of "deregulation" and just started jacking the prices up for everyone, but they still have problems modeling their loads with all the new pricing structures they've put in place.

25 years ago, I got into computer networking because I thought that the mathematical problems involved in finding the optimal path to get a packet of data from the source to the destination was a pretty interesting problem space. Truth is this: It is a pretty interesting problem space. All through the 90's, we were hammering down the last nails into the coffin of this problem and now there's only embellishments being made.

Now, looking back over the issue, it is the kindergarten of the problem space. In the next 10 years, the varsity guys are going to take on the same mathematical problem, but with For Real Consequences. Mucking up a routing decision on a computer network isn't a mistake with huge consequences most of the time - some packets get dropped. Big Deal; the higher-level protocols re-transmit and the user experiences a delay - at worst, a time-out.

Mucking up a real-time computed routing decision on a power network - that will have expensive and possibly disasterous consequences most of the time.

As I remember some of the mistakes we made in computer network routing theory and implementation, I shudder to think of what sorts of mistakes are going to happen from all this "Smart Grid" happy talk.