Forecasting UK electricity prices

I wrote a blog post a couple of weeks ago discussing the challenges of getting the data to forecast hourly day-ahead electricity prices for Great Britain. I concluded that it was always going to be a challenge, especially to verify how well any approach was likely to work. That wasn’t meant to suggest that the challenge was impossible, and this blog post describes a method that produces forecasts, although I can’t yet report how good they are.

I have implemented this model in a python on a server. You can see the code in github and the results at https://energy.guylipman.com/forecasts.

Forecasting demand

As a result, I have assumed that the demand for an upcoming weekday/time will be the same as it was on that weekday/time in the most recent week. For each day in the past 7, therefore, I extract the reported demand from BMRS report FORDAYDEM, type DANF. I extract the solar generation from these days from BMRS report B1630, and add it back to the demand to get the gross demand for each weekday/time. I haven’t implemented handling of public holidays, but presumably these will follow some combination of a Sunday and a weekday profile.

This method will inherently have a bit of a lag — for example, as we approach winter, we would expect gross demand to gradually rise, however I feel that being a week behind won’t be too much of a problem and allows us to pick up fundamental shifts in demand like we have seen from Covid lockdowns. A bigger concern is that this approach won’t take into account demand changes that would have been predictable to a fundamental analyst. However, applying this kind of insight requires ongoing judgement, so is beyond the scope of my project.

Forecasting solar generation

I chose to use 14 days of history to increase my chance of having a clear day for my true solar, but also to minimise the lag as the days get longer or shorter. Ideally I would also have access to UV history, which would be useful in understanding how much to adjust future days. I also made a conscious decision not to use solar generation data from previous years, given the rapid roll-out of solar capacity.

Forecasting wind generation

I don’t have much sense of how good the wind generation forecasts from BMRS are, especially as BMRS do not offer historical forecasts. I have started storing the wind forecasts, so hopefully over time I can build up a better picture of this.

The wind generation forecasts are presumably based on the expected level of wind. However, weather forecasting agencies typically have additional data about how much higher or lower wind is likely to be, which can be used to give a sense of how much the wind forecasts are likely to change. Ideally I would take this data into account, however I haven’t yet found any open sources of this data.

Net Demand

Estimating electricity prices

My approach has been to assume that the dispatchable generation supply curve (how much can be produced at what cost) over the next week will be similar to that over the past 14 days. I therefore estimate the curve of best fit, and apply it to net demand forecasts for the next 7 days. This assumption isn’t perfect: it may be that the likely available generation supply in the next week is different to the past two weeks. An analyst who could follow generator availability could adjust for this and improve their forecasts, however I don’t have the data for this.

Ideally we would fit day ahead electricity prices to day ahead estimates of net demand. Unfortunately BMRS do not provide historical data for day ahead demand or wind forecasts or wind forecasts. As a result, I have used the latest demand (FORDAYDEM, DANF) and wind generation (WINDFORFUELHH). I have compared the hourly net demand from these sources to day ahead hourly prices from Nordpool (https://www.nordpoolgroup.com/historical-market-data/).

If I look at the net demand and day ahead prices from 27 June to 10 July 2020, I get the scatter in the plot below. It is represented quite well (r=0.89) with a logarithmic function Price = 18.54 x ln(net demand)-150.6.

Every day I perform a regression and record the slope, intercept and r value. I then use the latest slope and intercept to convert my forecast net demand to prices:

Each day I store my forecasts, and the latest ones can always be seen at https://energy.guylipman.com/forecasts. You can see earlier forecasts by adding a before flag, eg https://energy.guylipman.com/forecasts?before=2020-07-11T23:00 (though I don’t have any before the evening of 11 July).

The motivation for this exercise was to generate forecasts of Octopus Agile tariffs, which are based on wholesale prices. The formula to convert from wholesale prices (in £/MWh) to retail tariffs (in p/kwh) is slightly different in each region. To save doing the conversions, the webpage can convert to any region, for example, for London, go to https://energy.guylipman.com/forecasts?region=C. (The region codes for each region are shown on https://en.wikipedia.org/wiki/Meter_Point_Administration_Number#Distributor_ID).

Further Work

Disclaimer

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Guy Lipman

Fascinated by what makes societies and markets work, especially in sustainable energy. http://guylipman.com. Views not necessarily reflect those of my employer.