Lawyers vs Artificial Intelligence: No prizes for guessing who won

In a match between Artificial Intelligence and lawyers, the machine left the lawyers far behind in day-to-day tasks of a counsel in terms of accuracy, performance and time taken to do mundane jobs.

The study conducted by LawGeex pitted the LawGeex Artificial Intelligence (AI) algorithm against 20 experienced US-educated lawyers and gave each of them a task of reviewing five non-disclosure agreements (NDAs)—the bread and butter of any corporate lawyer.

The study found that the AI achieved an accuracy of 94 percent compared to the lawyers’ 85 percent. Performance wise, the AI achieved an efficiency of 94 percent compared to lowest performing lawyer’s efficiency of 67 percent. However, the highest performing lawyer matched up the performance of the algorithm.

The AI, expectedly, took much less time than the lawyers who reviewed the agreements manually. The AI completed the tasks in mere 26 seconds, whereas the longest time taken by a lawyer was 156 minutes and the least time taken was 51 minutes. On average, the lawyers took 92 minutes to complete the tasks.

The lawyers, as well as the AI, went through 30 legal issues, 110 A4 size pages, 153 paragraphs of texts and 3,213 legal clauses in order to complete the task.

Dr Roland Vogl, Lecturer in Law at Stanford Law School, and consultant on the LawGeex AI project says: “The Robo-lawyer really has two faces. One is mechanizing and automating legal services – commoditized legal work and document checking. Legal prediction is the other aspect. This test falls in line with the first type of AI automation.”

Participant attorneys in the study also thought that technology can make the ‘mundane’ task of reviewing NDAs fast and simple for them and free up their time.

“The test gave me a practical glimpse into how technology can automate a staple of the legal profession – reviewing NDAs. The type of issue spotting carried out is credible and quite similar to how we (manually) do this type of work and have for decades,” Shena Shenoi - one of the 20 lawyers who took part in the study - said.