NGDAY

For some it might seem like overkill, but depending on how much money you have at stake you may not think that way. Machine learning has many benefits, our favorite is that the model itself determines what is important to make a prediction and basically discards what is not. All we have to do is provide the data, meaning that one type of model can be eaily adapted for many uses and by many different users.
The current machine learning technolody also means that adapting to a new model could be as easy as changing one line of code. Unlike a linear model, you dont really have to understand what goes behind the scenes so long you like the results.

Our favorite and most proven model to forecast natural gas load is XGboost simply due to its accuracy over other models. We determined this by conducting hundreds of hours of real backtesting to determine which models would result in the smallest average error for our dataset. Keep in mind that though we use a generally heat sensitive dataset some models may be more effective than others given unique datasets. This is the reason we are currently implementing new predictive models and testing them as they evolve.

If you aren’t conviced, XGboost consistently shines in Machine Learning competition website Kaggle as one model that is always top rank for results.

Kaggle - XGboost

We use the actual weather for a certain day and run through the model using only data previous to that day. This ensures the model is realistic in that it couldn’t possibly know the resulting actual while it was trying to predict.

For Example.

Create models using data from 1/1/2020-11/15/2020, and use actual weather from 11/16/2020.

In the following example we see that catboost was the best predictor using the historical data. We repeat this exercise for at least 10 years of data to determine an average absolute error and choose the best model.

Other Models.

As listed above we also attempted to use other models and will look to add them as we acquire more data. We offer model backtesting as a value added service to help with model selection but we feel that our testing has time after time determined that XGboost provides the best results, while not being the most resource intensive.

The future.

We will continue to monitor machine learning channels for what is the newest and offer that to our customer base. As we grow our data we hope to better able to hone in a recommendations for certain models under certain circumstance or scenarios. More data will allow as us to also better tune the models to even further reduce any expected error. For now we must generally recommend XGboost to everyone because it is that good.

We have a list of weather variables that we extract from our weather sources to determine what impact if any they may have on load changes each day. Keep in that not every load is affected by each variable, and the model determines to what extent if any a certain variable affects load. For example, a model for a school should know on Saturday and Sundays load will be greatly reduced and so it will an important factor; while day of the week for a poultry operation not be a load determining factor and hence will be generally ignored by the model. We are always open to suggestion on new variables that we can make available to everyone, and we also offer the option to use custom variables that you determine uniquely affect your load.

These are the variables that we use in our models, most are self-explanatory. Please reach out with questions. Keep in mind that the importance of these factors are 100% determine by whether the load is affected by it or not, and it also requires a certain number of data points. For example, you may be affected by snow but if it rarely snows the model has trouble making that assertion.

Ø  Date

Ø  Is holiday – Is is one of the US Federal Holiday?

Ø  Day of the week – What day of the week is it?

Ø  Month – What month?

Ø  Year – What Year?

Ø  Moon-Phase

Ø  Precipitation Intensity

Ø  Precipitation Type – Rain or Snow?

Ø  Temperature High – Day High temperature

Ø  Temperature Low – Overnight low

Ø  Dew Point

Ø  Humidity

Ø  Pressure

Ø  Wind Speed

Ø  Cloud Cover

Ø  Average Temperature

Ø  Weighted Average Temperature

Ø  Day Length

Ø  Weighted Average HDD

Ø  Weighted Average CDD

Ø  Previous Day Weighted Average Temperature

Ø  Previous Day Weighted Average HDD

Ø  Previous Day Weighted Average CDD

A custom variable for a Football Stadium could be the number of people expected to attend on a certain date. We would need historical information on the attendance numbers that together with weather can forecast the expected load. In this case a day with expected attendance of 100 persons will yield a lower load, while a day with attendance of 10,000 person would yield a much higher load. The model itself given the information makes a decision as to what degree of importance a certain variable has.

A pool is just an aggregation of meter reads on which given historicals we will make a load forecast. This aggregation could be a single meter or 100k meters, really its up to you and your needs. The larger the aggregation generally the less error that we see.

Dark Sky has been our sole provider of weather data for while now due to the simplicity of its API coupled together with its accuracy. We are actively researching a replacement since they have been bought by Apple and will no longer offer their API. We hope our next provider will have historical forecasts that will allow us our forecast would have been based on a weather forecast and not a weather actual.