The seven come down to three things
Business outcome
Does the product solve a real problem for the department, and can you measure it?
Fit
Does it work where your team already works, and connect to what you already own?
Built to last
Is the AI underneath more than a wrapper on one model?
Mark each answer Strong, Okay, or Weak. Two or more Weak answers means skip the demo and move on to the next booth.
The questions
What business problem does your AI solve for a legal department, and how would I measure it?
Strong answer
A specific operational outcome: faster turnaround on requests, lower outside counsel spend, more questions answered without a lawyer, better visibility for leadership. They can show the report.
Weak answer
"It makes lawyers more productive," with no number attached.
Where does my team use this, and which of the systems I already own does it connect to?
Strong answer
It works inside email, chat, and the office tools people already use, and they name specific systems (document stores, contract repositories, ticketing, finance) that connect through configuration rather than a services project.
Weak answer
A tour of their web app and a promise about integrations.
Which AI models run in your product, what does each one do, and who decides when to change them?
Strong answer
More than one model, each matched to a job (a fast one to sort and route, a strong one to reason over a document), and a named person who tests new models against real tasks before they go live.
Weak answer
One model name, or "we use whatever the provider recommends."
The last time you upgraded a model, what changed for your users?
Strong answer
Nothing to relearn. Answers got better, screens and workflows stayed the same, and they can name the month and what improved.
Weak answer
A release with new buttons, or they cannot remember.
Where do the answers come from, and what happens when the AI does not know?
Strong answer
Answers come from your approved content and your own records, each one cites its source, and when the answer is not there the AI says so and hands the question to a person with the context attached.
Weak answer
It uses the model's general knowledge, or it always answers.
Can it complete work, and can I see everything it did?
Strong answer
It can take actions inside the system, shows a preview, waits for a person to confirm, and every AI call is logged with what it read, what it answered, which model ran, and what it cost.
Weak answer
It drafts text and people do the rest by hand, or logging is a roadmap item.
Is my data used to train any model, and what security proof do you hold today?
Strong answer
No, it is in the contract and backed by their terms with each model provider. SOC 2 Type II or ISO 27001 in hand, a penetration test within the year, or a dated plan if they are early.
Weak answer
"We take security very seriously."
The foundation decides the ceiling. These seven questions are how you check a foundation in five minutes. Use them on every vendor. Use them on us.
If you want the thinking behind the questions, start with why legal teams stall on AI, and what to ask instead. For what a good answer to question 5 looks like in practice, see how the Legal Knowledge Base cites its sources and hands off what it does not know.
Take it with you. One page, score boxes for all seven, no form to fill out.
Download the PDFComing soon: the long-form version
The AI-Native Legal Tech Checklist covers 46 items across eight sections, each with what to look for and how to validate it in a demo, plus a certifications reference and a scorecard. It is the one to bring to the full demo. Email us and we will send you an early copy.
Request the full checklist
Ask us the seven questions
We built LegalOperator to pass this test. Bring the checklist to the demo and score us.
Book a Demo