Rated 5 star on
Due Diligence on a Custom AI Partner
Quick answer
Check every claim before you commit. Ask to see the system work on your own data, and how accuracy was measured and what errors remain. Call references yourself and ask about problems. Read the security and data-handling documents before sharing data, run a small paid pilot, and get ownership of code, data and accounts in writing.
Almost every AI vendor has a polished demo and confident claims. Due diligence means checking those claims before you commit: seeing the system work on your own data, reading the security documents, calling references yourself and testing with a small paid pilot. A partner worth hiring welcomes the scrutiny.
Key Takeaways
- Polished demos prove little; ask to see your own data processed.
- Ask how accuracy was measured, on what data, and what errors remain.
- Call references yourself and ask about problems, not just praise.
- Read security and data-handling documents before sharing data.
- Make sure you own the code, data and accounts, in writing.
- Use a small paid pilot as the final test before a larger contract.
Published: November 11, 2026 | Reading Time: ~8 minutes | Category: Industry-Specific AI
This guide covers demos on your own data, accuracy evidence, references you call, data handling, ownership, a paid pilot, scoring partners and red flags. Put plainly: verify with evidence, not promises, and test small before you sign big.
In This Playbook
- Demos on your own data
- What accuracy evidence should you ask for?
- References you call
- How they discover
- Data handling and security
- Ownership and contracts
- A paid pilot
- Scoring partners side by side
- Red flags
- What to measure
- The first three months of choosing
Demos on your own data
Your documents, calls or records. Not a prepared example.
Real messiness included. Odd cases and incomplete data.
Watch it fail too. How it handles what it can't do.
What accuracy evidence should you ask for?
How it was measured. On what data and against what baseline.
Errors named. Which mistakes remain and how often.
Your own test set. Samples you choose, scored by your team.
References you call
Similar businesses. Size, industry and problem.
Ask about problems. What went wrong and how it was handled.
Still running? Systems in use a year later, not just launched.
How they discover
Process mapped first. As covered in discovery.
Measure before building. As covered in building before measuring.
Willing to say no. Recommending ready-made tools when they fit, as in off-the-shelf first.
Data handling and security
Written policies. Where data is stored, who sees it and how long it's kept.
Training on your data. Whether your data trains anyone else's models.
Testing practices. As covered in testing agents.
How it will connect. As covered in connecting agents to your systems.
Ownership and contracts
Code, data and accounts in your name. As covered in owning your code.
Exit terms. Handover and data export if you leave.
Support terms. Response times and upkeep costs.
A paid pilot
Small scope, fixed price. One task, one team.
Success defined up front. Measured against today's baseline.
Payback worked out. As in what ML projects cost.
A real test of working together. Communication, speed and openness about problems.
Scoring partners side by side
Same criteria for each. Evidence, fit, security, ownership and cost.
Scores backed by notes. What you saw, not impressions.
Vertical or general tools too. As weighed in vertical AI vs horizontal AI.
Red flags
No demo on your data. Only prepared examples.
Vague accuracy claims. "Highly accurate" with no numbers.
Ownership kept by them. Code or accounts in their name.
No discussion of risks. As covered in AI ethics for business owners.
More questions for model work are in choosing an ML partner.
What to measure
Pilot results. Against the agreed baseline.
Accuracy on your test set. Scored by your team.
Reference feedback. Problems and how they were handled.
Contract terms secured. Ownership, exit and support.
Time to first value. From signing to useful results.
The first three months of choosing
Month one: shortlist and test
Shortlist partners, request demos on your own samples and ask how accuracy was measured.
Month two: check
Call references, read security and data documents and review ownership and exit terms.
Month three: pilot
Run a small paid pilot with the top choice, measure against the baseline and decide.
How Astra welcomes due diligence
Astra Results Marketing expects to be vetted. We demo on your data, explain how accuracy is measured, share security practices, keep ownership with you and start with a small pilot.
Engagements begin with a discovery session through our industry-specific AI team.
Related reading
Frequently asked questions
What is due diligence on an AI partner?
Checking a partner's claims with evidence before committing: demos on your data, accuracy proof, references, security and ownership.
Why ask for a demo on our own data?
Prepared demos show ideal cases; your data shows how the system handles real, messy work.
What should we ask references?
What went wrong, how problems were handled and whether the system is still in use.
Who should own the code and data?
Your business, in writing, with exit and handover terms.
Why run a paid pilot?
It tests results and the working relationship on a small scale before a larger contract.
What should we measure?
Pilot results, accuracy on your test set, reference feedback, contract terms and time to first value.
Choosing a custom AI partner? Astra Results Marketing demos on your data, shows how accuracy is measured, keeps ownership with you and starts with a small pilot. ▸ CALL (786) 321-2866 · ▸ REQUEST YOUR CONSULTATION
