Monitoring Forecasting Models at Scale

Photo by Volodymyr Hryshchenko on Unsplash

Monitoring Metrics

First, let’s define what we mean by “well” and by “lately”.

Forecast with small MAPE. Model forecast is very close to actual price movement.
Forecast with high R² and high MAPE. Model predicted the trend but not the price fluctuations.
Example of 2 time series matching using the DWT algorithm (Source: Wikipedia)

In action

We update our metrics daily for more than 500 Deep Learning models.

An overview of aggregated performance statistics of a subset of models
Zooming in on VRSK model recent performance
Identifying individual forecasts based on a specific metric

What’s next

There is a lot more to cover but we will stop here for now. We plan to write a dedicated article for our Simulated Trading Average Return (STAR) metric and reveal more details in the near future.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Predicto

Predicto

Stock & Cryptocurrency Forecasting AI. Based on News and Options Data. Powered by Intelligible Deep Learning models. https://predic.to