<!-- Canonical: https://myoctopus.ai/technical/measuring-forecast-accuracy-improvement -->

# How to measure forecast accuracy improvement from AI

Accuracy claims are only meaningful against a locked baseline. Without one, an improvement is indistinguishable from a change in the forecasting process.

The measurable question is narrow: for the same periods and the same accounts, did absolute error fall after the workflow changed?

This page describes the method. Deployment results are reported separately and only with sourced figures.

## Questions and answers

### What baseline should be used?

The prior forecasts as submitted, locked at submission time, for the same accounts and periods. Comparing against a restated or re-derived baseline invalidates the result.

### Which error metrics are appropriate?

Absolute and percentage error at the line-item level, aggregated by cost centre and entity. Report distributions rather than a single average, because a few large accounts otherwise dominate the number.

### How do you separate process improvement from model improvement?

Hold one variable at a time. A governed workflow that collects commentary from budget owners changes the inputs, which is a process effect, and should be measured separately from any change in model configuration.

### What else should be measured beside accuracy?

Cycle time from close to a defensible variance narrative, the share of variances with a named explanation, and the proportion of data-quality errors caught before they reach the forecast.

### Are accuracy figures published?

Only sourced ones. The site-wide proof figures are the reference client results shown below; other numbers are reported per deployment with the measurement conditions stated.

## Proof

*   Reference client: $523K total contract value over 3 years.
*   Approximately 430 analyst hours saved.
*   Forecast cycle cut from about 60 hours to under 6 hours (90% faster).
*   Error rate below 0.1%.
*   2,500+ autonomous tasks completed in month one.

## Related pages

*   [Forecasting system card](/trust/system-cards/forecasting)
*   [Best AI tools for finance teams in 2026](/best-ai-tools-finance-teams-2026)
*   [All technical briefs](/technical)
