A website for viewing your smart meter data

This post explains the ideas behind a website I have built to allow UK owners of smart meters access to their data, in a way that makes that data useful to them. I hope that sharing my thinking with others in the field will generate some useful suggestions and debate.

Key principles

My goal is to build something that makes it free, and as easy as possible for as many people as possible to access their data.

Next, I know that a lot of customers aren’t very trusting when their data is concerned. I want to make it as easy as possible for people to trust the website, and to learn to appreciate the value in having access to their data. To do this, I will take data security concerns extremely seriously. And I will always make decisions in the customers’ interests (rather than, say, in the interests of companies that may want access to customer data).

Design decisions

I chose a website for a number of reasons (over, say, a mobile app). Most customers have access to a web browser. Web access allows use on a computer screen, which increases the real estate for me to provide information. I already know python (which can be used to build websites), reducing the learning curve. And websites offer a much quicker turnaround on changes and releasing new features, without any worry that customers will be stuck on old versions.

The next difficult decision was what customer data to store. I have previously built websites that don’t store any customer data, making them very slow, especially if you want to look back a year. The other extreme would be one in which customers logged in, and provided their login details for their utility. Some customers are happy to do this, but I worried that many would not, reducing the accessibility of the service. I also worried about the risks if any of this data got stolen.

I decided to store consumption data, but nothing else, and only for as long as needed. The user provides their login key to load the data, however the login key is not stored. The user can then perform any number of calculations on that data, before deleting it (or it is deleted automatically within a few days).

I have initially hosted my website on a Digital Ocean virtual server, with NGINX/Django/PostgreSQL. Even the cheapest tiers are good enough to get started, and handle hundreds of customers a day, and there is plenty of scope to scale up if or as I need. I will be adding additional features over the coming weeks, so it may occasionally be down, but it should be available most of the time. Obviously nothing would stop the logic being ported across to another technology stack should it be necessary.

A final design decision is that I mostly stick to stateless webpages, that is, what is shown on the webpage is largely dependent on the url. It may not look as tidy having lots of parameters in the url, and may not run as quickly. However, from the perspectives of consistency and transparency, it makes development much simpler, and also allows users to bookmark particular reports/configurations.

The Website

The core user flow is:

  1. Go onto the website and provide a key to access your consumption data. At this point either Octopus or n3rgy are available, but I am eager to add more.

If you don’t want to load data, you can also try out the website functionality using a made-up data set.

Data Structure

Feel free to ignore this section, but I figured it would be useful/interesting for some readers.

I have three main tables in which I store data:

  • sm_quantities which stores the quantity consumed for each half hour for each user, for each of electricity consumption, gas consumption and electricity export.

I have a number of daily scripts that keep the variable_vals tables populated with the latest prices and carbon intensities.

My website is able to perform all sorts of calculations efficiently using sql, using the data in these tables.

Call for Suggestions

I’d love to get suggestions, especially from others that have thought about this topic before. A few particular points I’d love to hear on.

  • Are there any UK utilities other than Octopus that offer customers with smart meters API access to their data? I’d be very happy to write the interface layer to load that data into the database.

Feel free to comment below, or reach out to me directly via twitter or email.

If you’ve made it this far, you will probably want to visit the website itself on https://energy.guylipman.com/sm/home, otherwise I’ve included some screenshots below.

Fascinated by what makes societies and markets work, especially in sustainable energy. http://guylipman.com.