Letting people access their electricity data

It has historically been incredibly difficult for UK electricity customers to find out how much electricity they’ve used and when. However, that may finally be changing. In this blog post I discuss some of the past challenges, and review a new service that may finally allow many customers to access their data.

For the past few years, one of my biggest complaints about the UK electricity industry has been how hard it is for people to find out how much electricity they’ve used and when. Sure, some people just don’t care. But I’m convinced that if we could make it less of a struggle, a lot more people would find it worth their while.

Without a smart meter, which records how much electricity is used every half hour, there will be limited data available. At the end of March 2020 10.9m of the 26.6m domestic meters were smart [BEIS data]. The customers who don’t have smart meters fall into three categories: those who are already keen to get one, those that would get one if they thought it was more useful/less hassle, and those that wouldn’t want one no matter how useful it was. The first two categories of users will ultimately be helped by making smart meter data more accessible, as will customers that already have a smart meter. As for the final category — I suspect that there aren’t actually that many people in it (though they do get lots of media coverage).

As a result, for the rest of this post I will focus on users that have smart meters.

How it has been

At the moment there are three ways for such users to access electricity data: through an in-house display, through advanced monitoring equipment, or through their electricity supplier (if they’re lucky).

In-house display

When customers get a smart meter, they are also given a small display which is meant to give them realtime access to their meter data. Most customers put these into a cupboard and promptly forget about them. If you do plug it in, it shows you your current consumption, and how much you’ve used so far that day. But they don’t show you any useful historical data. And they don’t offer any ability to export data for analysis.

Advanced monitoring equipment

You can generate your own electricity consumption data using separate monitoring equipment, alongside your smart meter data. A really good example of this is the products/service offered by Hildebrand. They sell metering equipment that will capture your consumption as often as every second, and can monitor individual appliances in order to track how much different things use. They then allow you to import that data into a Home Energy Management System (HEMS) or perform sophisticated analysis. Most people probably can’t justify the cost of this set-up, but it could make sense if you were a big electricity consumers, or if you had solar panels or a battery that you wanted to optimise.

Access via your utility

The final way to access your consumption data is via your electricity supplier, which is what I do. The biggest weakness of this approach is that it relies on your supplier being able to provide you with the data in a useful format — I don’t actually know any suppliers that do, other than Octopus Energy. Octopus provide an API, on which a number of developers have built useful apps and websites to provide data in useful formats.

There are a few other weaknesses of this approach. The data typically takes around a day to get from the meter to the supplier, so this cannot be used to get real-time consumption data. There can be technical issues which cause it to take even longer. Secondly, if you’re trying to check that you are being billed correctly, you may want to get your consumption data independently from your supplier. Finally, some customers don’t want to provide their supplier with their half-hourly consumption data, perhaps worried that they will end up being charged more.

A new approach

I very recently discovered a new approach to accessing your smart meter data, which is offered by UK company n3rgy. This company’s primary business is providing a conduit through which customers can permission other companies and organisations to access their consumption data, irrespective of what model smart meter they have or who their supplier is. For example, energy services cooperative Carbon Co-op are allowed by their members to access and report their consumption data in this way. (Note that Hildebrand do offer a similar platform for their customers, however it currently requires you to have their equipment or a SMETS2 meter.)

While I’m genuinely keen to see the business models that develop using customer data, in the meantime I’m particularly excited by the fact that the same platform allows just about any smart meter customer to access their own data, right now, at no cost. You can register at https://data.n3rgy.com/home (click on the “I’m a consumer” button) by providing some details (most importantly the MAC code from your in-house display — hopefully you remember where you stashed it), and then the service will start collecting all the data stored in your smart meter (13 months, or since you got it if it hasn’t been that long). The service then checks in with the smart meter each night and attempts to collect the latest data — mostly successfully (and if it fails one night, it will collect the missing data the next night).

The n3rgy website offers the ability to download a csv with up to 90 days of historical half-hourly consumption. However, more usefully it provides API access to your data. In 20 minutes I was able to use python to retrieve data and start playing around with it. For example, the following code creates a pandas data series with all the half-hourly consumption from May 2020:

import requests
import pandas as pd
headers = {'Authorization': AUTH} #AUTH is my MAC code
url = "https://consumer-api.data.n3rgy.com/"
path = 'electricity/consumption/1/'
query = '?start=202005010000&end=202006010000'
api_url = url + path + query
r = requests.get(url=api_url, headers=headers)
data = pd.Series({x['timestamp']: x['value'] 
                   for x in r.json()['values']})
print(data)
Out[118]: 
2020-05-01 00:00    0.030
2020-05-01 00:30    0.018
2020-05-01 01:00    0.025
2020-05-01 01:30    1.108
2020-05-01 02:00    0.021
                     ...
2020-05-31 22:00    0.041
2020-05-31 22:30    0.037
2020-05-31 23:00    0.027
2020-05-31 23:30    0.016
2020-06-01 00:00    0.030
Length: 1489, dtype: float64

Although I still prefer python for coding and analysis, over the past few months I’ve been writing a number of javascript webpages that retrieve Octopus API data and perform calculations. This allows anyone to run the code, on any computer, without sending authentication details to my server. This works well with the n3rgy API. For example, I built a webpage that checks how much electricity and gas consumption data a customer has stored with n3rgy, and also checks that there are no missing half-hours within the last 30 days. Any registered n3rgy user can use this, just by including their MAC code in the url (https://www.guylipman.com/octopus/n3rgy.html?MAC=XXX).

I am now thinking about useful webpages I could create using the data, for example:

• Calculating how much a customer used between two dates, as a way of checking their electricity bill
• Calculating how much a customer spent on electricity between two dates, as a way of checking their electricity bill (either for a flat tariff or a time of use tariff)
• Calculating the emissions generated as a result of a customer’s electricity consumption
• Calculating how much their electricity would have cost on various flat or time-of-use tariffs
• Analysing how successfully a customer was flexing to cheaper or greener time periods (see previous blog post for discussion of this)
• Producing charts showing how consumption changed over time

But I’m sure there are other useful reports that could easily be delivered using the n3rgy API, and ways to use mobile apps to make accessing this data more user-friendly.

This underlying data is still delayed by a day, and doesn’t go more granular than half-hourly, or to the appliance level: as a result, I expect there will still be some customers that want more data, and will be willing to pay for it. Still, I think what n3rgy are offering is a brilliant service that offers an easy way for millions of smart meter customers to access their data and gain valuable insights.