Buying an AI tool or service is easy. Living with it is the hard part.
In 2026, most AI demos look good. The real cost shows up later, when your team has to wire data sources, adjust workflows, retrain users, set rules, and prove to auditors that the system behaves. Switching costs can be sneaky too. Once your prompts, connectors, and evaluations live inside one platform, “just move later” turns into a quarter-long project.
That’s why AI vendor due diligence has changed. The risk isn’t “does it work?” It’s “does it fit our business, and can we operate it without drama?”
Use the checklist below before you sign. It’s built for long-term fit, risk, and operating cost, not shiny features.
Data and Architecture Fit (Will it work with how we run today?)
Most AI projects don’t fail because the model is weak. They fail because the data path is fragile, slow, or expensive. Early architecture choices also become long-term lock-in, so it’s worth pressing for clear answers now.
Where does our data live, and how does the vendor access it?
Ask where processing happens: your VPC, on-prem, a managed cloud, or the vendor’s hosted stack. Then ask how data moves.
Key questions to ask:
- Does data stay in place, or is it copied into the vendor system (and cached)?
- What connectors exist for your core systems, and who maintains them?
- What are the data egress costs, and who pays them?
- What latency should you expect when pulling from systems like your CRM, ticketing tool, or data warehouse?
- What happens when a source system is slow or down, does the AI fail safely?
Does the vendor adapt to our architecture, or do we have to reshape ours?
“Implementation included” can mean anything from a kickoff call to months of engineering work on your side.
Ask what you must change to go live:
- Identity setup (SSO, SCIM, MFA), and whether it supports your current identity provider
- Network controls (private endpoints, IP allowlists, outbound rules)
- API patterns (webhooks, event streams, batch jobs) and rate limits
- Your data lake or warehouse conventions, including how access is granted
Also ask who builds what. If your team has to write and own the glue code, the vendor isn’t really providing a platform, they’re providing a tool.
How is data modeled, stored, and versioned (and what happens when schemas change)?
AI systems break in boring ways, like when a field name changes or a table is split.
Ask how the vendor handles:
- Schema evolution and lineage (can you trace outputs back to sources?)
- Feature definitions, embeddings, and knowledge bases (who owns updates?)
- Backfills and re-indexing when source data changes
- Versioning for prompts, workflows, and retrieval settings
You want to hear a clear plan for change, plus a rough cost and time estimate for re-indexing at your scale.
Security, Privacy, and Governance (Can we scale safely?)
Governance isn’t just about passing a review. It’s how you scale without turning every release into a fire drill. Strong controls let teams ship faster, because rules are clear.
How is customer data isolated across tenants, and how are permissions enforced?
Multi-tenant systems can be safe, but only if isolation is real.
Ask about:
- Tenant isolation design, not just “we’re multi-tenant”
- Encryption at rest and in transit, and key management options (including customer-managed keys if offered)
- Role-based access control (RBAC), and whether it maps to your org chart and your systems (not just roles inside their UI)
If permissions can’t align with your identity groups and data entitlements, expect ongoing manual work.
What audit logs exist, and how are prompts, outputs, and access logged?
Audit logs are your black box recorder.
Ask what gets logged:
- Prompts, retrieved sources, tool calls, and outputs
- Admin actions, configuration changes, and access events
- Log retention periods, export options, and formats that work for investigations
Also ask if sensitive inputs can be redacted or masked, and how policy violations are detected and surfaced.
What guardrails exist for data leakage and unsafe outputs?
Guardrails need to work at the system level, not as a “please be safe” prompt.
Ask about:
- DLP controls and blocked categories (PII, secrets, client data)
- Allowlists for tools and connectors, plus approval workflows for exceptions
- Sandboxing, policy rules, and how rules are tested before rollout
If exceptions are handled through informal admin tweaks, you’ll get inconsistent behavior across teams.
Cost, Lock In, Reliability, Evaluation, and Operating Model (Can we afford it, run it, and exit it?)
This is where AI vendor due diligence pays off. The hidden risks here are budget surprises, platform dependence, outages, quality drift, and unclear ownership.
Cost transparency and commercial risk: How will pricing grow over time?
Ask how pricing is calculated, then ask what drives it up.
Cover the basics:
- Is it priced per user, per call, per token, per workflow run, or a mix?
- Do longer contexts, retrieval, tool use, or concurrency increase cost?
- Are evaluations, fine-tuning, or logging billed separately?
Then push for visibility. Can you see cost by use case, team, or workflow? During spikes, what happens: throttling, rate limits, or surge pricing? Finally, confirm minimum commits, overage terms, and renewal mechanics, because that’s where “good first-year pricing” can turn into year-two pain.
Model and vendor lock in: Can we swap models, and can we leave without starting over?
Lock-in isn’t always a contract problem. It’s often a technical one.
Ask what is proprietary:
- Agent logic, orchestration, workflow builders
- Vector stores, prompt management, evaluation harnesses
- Any “secret sauce” layers you can’t export
Then ask about portability. Can you export prompts, embeddings (or regenerate them easily), evaluation sets, and audit logs? Ask for exit terms in writing: export format, timelines, deletion proof, and what support is included. Also ask if you can run multiple models side-by-side for cost, safety, or performance without rebuilding everything.
Reliability and production readiness: What happens on a bad day?
AI in production needs “boring” engineering.
Ask for SLAs on uptime and latency, plus planned maintenance practices. Confirm incident response times, status reporting, and post-incident reviews.
Then ask how failure is handled:
- Retries, circuit breakers, and safe mode behaviors
- Fallback models or degraded responses when a dependency fails
- Monitoring and alerting hooks that fit your stack
Also ask how updates roll out, including change logs, canary releases, and rollback paths.
Evaluation, monitoring, and feedback loops: How do we measure quality and catch drift?
If you can’t measure quality, you’ll argue about it.
Ask how they track accuracy, relevance, and hallucinations for your use cases. Do they support offline evaluations, regression checks, and side-by-side comparisons between versions? Confirm human review flows, feedback capture, and how changes are tested before release. Also clarify who owns improvement work, your team, the vendor, or both.
Operating model and ownership: Who runs this after the launch team leaves?
Many AI rollouts look fine until the initial team moves on.
Ask who will own the system internally, who approves changes, and what ongoing effort is expected from data, security, and app teams. Clarify needed skills (prompt work, evals, policy, MLOps) and what the vendor provides: onboarding, training, escalation paths, and what is included versus paid services.
Red Flags to Watch For (Fast ways to spot hidden risk)
Some signals should slow the deal down, even if the demo was strong.
Red flags that should pause the deal
- “Fully autonomous” claims with no clear guardrails, review flows, or safe mode
- No clear cost model, or pricing that can’t be tied to usage drivers
- Vague answers on tenant isolation, keys, and permission mapping
- No export plan, no exit terms, or unclear data deletion proof
- Limited audit logs, or logs you can’t export for investigations
- No customer references at a similar scale or in a similar industry
- Unwillingness to run a proof under real constraints (security, latency, budget)
A Simple Scoring Framework (Optional, but useful for procurement and execs)
Decision-making gets easier when tradeoffs are visible and written down. Score each vendor from 1 to 5 in five areas, then weight based on what matters most to your business.
Score each vendor on what matters most, then choose with eyes open
Use these categories:
- Architecture fit
- Governance maturity
- Cost transparency
- Operational readiness
- Exit risk
A vendor doesn’t need perfect scores across the board. The goal is to avoid surprises and document what you’re accepting. A hands-on pilot helps too. For example, a guided data pilot that tests fit using your real data and workflows can reduce risk without turning the process into a months-long RFP.
Conclusion
The best AI vendor isn’t the one with the longest feature list. It’s the one you can operate, scale, and exit if the business changes. Solid AI vendor due diligence protects you from quiet risks, like data sprawl, unclear ownership, and pricing that balloons once adoption starts.
Pick one high-value use case, run a time-boxed pilot under real controls (security, cost limits, logging), then decide with facts. If you want a partner that tends to be practical and people-first about pilots and change management, Data Pilot is worth a look (www.data-pilot.com).