It seems fair to say that most lawyers believe generative artificial intelligence is pretty good at taking the bar exam and not so good at writing legal briefs.

The data points underpinning these beliefs are these:

• In 2022, OpenAI’s leading generative AI technology ChatGPT 3.5 failed a uniform bar exam but still managed to perform better than 10% of the human beings who took the same test. In March 2023, ChatGPT 4.0 passed the bar, beating out 90% of human test-takers.
• ChatGPT occasionally suffers from “hallucinations,” meaning it can cite non-existent court opinions as support for legal positions asserted in query responses. One attorney was sanctioned for careless use of ChatGPT in Mata v. Avianca, 678 F. Supp. 3d 443 (S.D.N.Y. 2023), and another attorney was referred to local bar regulators for similar behavior in Park v. Kim, No. 22-2057 (2d Cir., Jan. 30, 2024). (“Careless use” means, in this context, incorporating into legal pleadings ChatGPT’s output without checking its accuracy.)

But technological progress is unrelenting and, according to a pair of legal technology experts, these generally accepted beliefs about what generative AI can (and can’t) do will not survive the technological advances that are coming in the very near future.

Generative AI will make better lawyers, make civil justice more accessible to more people, and relieve lawyers of some of the traditional drudgery of legal work.

Damien Riehl, vice president and Solutions Champion at vLex, and Trisha M. Rich, a partner at Holland & Knight’s Chicago office and nationally known expert on legal ethics, are frequent speakers on technology adoption within the legal profession. Both spoke recently at the Illinois Supreme Court Commission on Professionalism’s The Future Is Now legal services conference. Their advice to lawyers, in a nutshell, went something like this:

Generative AI Will Get A Lot Better Soon

ChatGPT version 5 will be released sometime during the summer of 2024. Details about ChatGPT-5 are scarce but, according to public comments from OpenAI officials, ChatGPT-5 will be significantly more powerful than ChatGPT-4 in its ability to reason, and in its ability to comprehend and produce human-like text. OpenAI CEO Sam Altman said recently that GPT-5 will be able to understand email messages and calendar details, and that it will be more easily customized to particular uses than ChatGPT-4.

Riehl demonstrated how ChatGPT is already a capable source of legal arguments and counter-arguments. ChatGPT can analyze a complaint or a statute. It’s able to spell out the basic elements of a cause of action. It can propose questions to be asked to potential jurors, and it can suggest a list of juror responses that could be disqualifying.

“Is this what I would file with the court? No way,” Riehl said. “Is this way better than a first-year associate can do? And did it give it to me in seconds? Yes, indeed, it is.”

In a prior post, we noted how generative AI can be productively used for deposition preparation. Artificial intelligence is also at the heart of technologies that can quickly produce deposition summaries like our new Intelligent Summary+™ offering.

Clients Will Demand That Attorneys Use AI

Just as clients today would push back against manual review of documents and computer records produced during pretrial discovery, clients will soon be objecting to billing statements that contain charges for associates doing work they know AI can do.

Rich, looking at the issue of computer-assisted legal work from a professional ethics perspective, said that the use of computers to sift through documents has become so ingrained in modern law practice that lawyers during the same work manually would likely run afoul of ethical rules prohibiting unreasonable legal fees. A lawyer’s decision to eschew generative AI for client work may, in time, raise similar ethical concerns. “That’s what I expect to happen with AI,” Rich said.

“Yes, right now we are seeing some fake cases and some issues with AI, but it’s going to get good, and it’s gonna get good really, really, really fast,” Rich said. “There is going to become a point in the future where somebody is going to say, a client is going to say to us, ‘Why aren’t you using a technological solution? And why are you charging me the hours and the manpower to do some of these things that technology could solve?’”

Riehl predicted that the growing use of AI to perform traditional legal work will also erode the prevalence of hourly billing practices. He said that the current percentage of flat-fee billing could rise from 10-20% to 80-90% as AI-created efficiencies take root in law firm operations. ### Current Legal Ethics Rules Adequately Address AI

When it comes to generative AI, the ethical rule that comes immediately to mind is the so-called duty of technology competence. Comment 8 of the American Bar Association’s Model Rule of Professional Responsibility 1.1 (Competence) – explicitly adopted in 40 states, and arguably the rule everywhere else by implication – obliges lawyers to understand how to use technology to advance the client’s interest, and to be able to balance the risks and benefits of using technology. Rich said the widespread adoption of generative AI will lead the legal profession to consider how to use it competently and, if it isn’t used, whether clients are being overcharged for legal services.

Rich noted other ethical rules implicated by generative AI. It’s clear, Rich said, that providing client information in a ChatGPT prompt would breach Rule 1.6’s duty of confidentiality. However, she added, generative AI products that have the ability to protect client confidential information will be available in the future.

Law firm use of generative AI also implicates the lawyer’s duty, under Rule 5.4 (Professional Independence) and Rule 5.5 (Responsibilities of a Partner or Supervisory Lawyer) to supervise the work of assistants at the firm. Drawing on her experience as an adjunct professor on legal ethics at New York University School of Law, Rich said that her students use ChatGPT “all day every day for everything.”

Older lawyers in supervisory roles should be aware their younger associates are going to be using ChatGPT and similar generative AI technologies. “It’s going to be second nature and we’re responsible for their work,” she said.

Court Orders Restricting AI Will Be Scaled Back

According to Rich, the Mata v. Avianca misadventure with ChatGPT was “splashy” and “awful,” but the case did not raise legal ethics issues that can’t be addressed by current rules. The outcome would have been the same if the brief had been drafted by a first-year associate who similarly made up a bunch of cases and included them in a filed brief. “These are just not new problems to us, I don’t think,” Rich said.

The same thinking led Rich to predict that court orders that require lawyers to disclose the use of generative AI – or refrain from using it altogether – will not stand the test of time.

Lawyers have never been required to disclose whether an associate produced the first draft of a pleading, Rich noted. “When else has a court reached into work product and said, ‘Did you check the work of that first-year associate? Did you check the work of that paralegal?’ And of course, they wouldn’t do that.”

In the federal system, Rule 11 of the Rules of Civil Procedure already provides for sanctions against a lawyer whose pleadings contain unsupported legal and factual contentions. “Rule 11 says everything above my signature is true and accurate whether a first-year associate did it or a paralegal did it or whether [generative AI] did it, and that’s why I think some of these orders are just really reactionary,” Rich said.

In the end, Riehl said, generative AI in legal practice will be commonplace.

“We ingest words, we analyze words, and we output words, and it turns out that large language models can do all three of those things at the same time,” he said. Better yet, he added, generative AI will make better lawyers, make civil justice more accessible to more people, and relieve lawyers of some of the traditional drudgery of legal work.