Technical due diligence questions for finance AI vendors

Most finance AI evaluations stall because the demo answers product questions and the risk review asks architecture questions. These are the architecture questions.

Each one has a verifiable answer. If a vendor cannot answer it in writing, treat it as unresolved.

Octopus AI publishes its answers: system cards per AFI capability, a constitution, and a transparency report.

Questions and answers

Where does arithmetic run — in the model or in code?

Ask for the boundary in writing. If any reported figure is generated by a language model, the same question can return different numbers, and reproducibility cannot be assured.

What access does the system hold, and is it read-only?

Ask for per-source scope and role-based access control, and whether the system can write to the ERP. Read-only, scoped access is the defensible position.

What is logged, and can it be exported?

Ask whether every action is logged with who, what, and when, whether figures carry line-item lineage, and whether logs, approvals, and version history export for auditors.

Is any outbound communication unattended?

Ask specifically. An approval gate that a named human signs is a control; an unattended send path is a risk that has to be disclosed in a control narrative.

What documentation is published rather than promised?

Ask for system cards, an operating framework, and a dated transparency report covering governance changes, incidents, model providers, and data residency.

How long is implementation, and what is the exit path?

Ask for the pilot scope, what is configured versus custom, who owns the mapping rules, and how data and configuration are exported if the engagement ends.

Proof

  • Reference client: $523K total contract value over 3 years.
  • Approximately 430 analyst hours saved.
  • Forecast cycle cut from about 60 hours to under 6 hours (90% faster).
  • Error rate below 0.1%.
  • 2,500+ autonomous tasks completed in month one.

Related pages