The first customer: due diligence for AI in law firms

No demonstration on its own data. No reference site. No pilot.

This is not a criticism of the vendor. Every product needs a first customer, and someone has to go first. But going first should be a deliberate decision, made with your eyes open and the right protections in place, not something you discover afterwards.

Five questions for any AI tool

1. Where does our client data go, and who can see it? Including whether it is used to train anyone’s models.
2. What has it actually been tested on, and by whom?
3. What happens when it gets something wrong? Who is accountable, and how would you know?
4. Who else is using it, and can we speak to them?
5. What are we committing to, and how do we get out?

Any firm would ask these before instructing a new expert or taking on a new supplier. Technology decisions too often skip them, because the presentation was impressive and the deadline was tight.

An independent second opinion

We have sat in the law firm, in the product team and in the vendor’s operations, so we know which questions get asked, which do not, and what it costs when they are missed. If you are about to sign for something new, AI or otherwise, an independent view costs far less than finding out the hard way.