The use of artificial intelligence in courtrooms is increasingly causing disruptions, particularly due to the generation of false case citations. In Victoria, Australia, a solicitor faced changes to their practicing conditions, and a law firm was ordered to pay costs after relying on AI to produce court documents that contained errors. Caroline Counsel, a family law specialist and chair of family law at the Victorian Law Institute, discussed the issue on ABC Radio's Mornings program. She noted that while AI is often seen as a solution, it can produce misleading information. "Unfortunately, people sort of think AI is the genie that's going to solve all the problems, but in actual fact it's like a really keen intern that wants to please," Counsel said. "If you ask it something and don't ask how it got to that conclusion, it could spit out what they call hallucinations — something that looks like a case, smells like a case and might even have a judge's name in it that's familiar to you — but the rest of it is just made up."

AI's impact has been highlighted in various sectors, including education, where universities are using AI to detect potential academic dishonesty among students. However, this has also led to incorrect accusations. The Victorian Legal Services Board and Commissioner has raised concerns about the misuse of generative AI tools in preparing court documents.

In one notable case, a solicitor, referred to as Mr. Dayal, submitted a list of legal authorities that did not exist and admitted to not verifying their accuracy. Following this incident, he acknowledged the need to improve his professional standards. As a result, the Victorian Legal Services Board modified his practicing certificate, prohibiting him from acting as a principal lawyer or handling trust money for two years, during which he must practice under supervision.

Another case involved a junior solicitor who used AI to prepare citations for a native title claim. When she attempted to replicate her search, the results varied, leading to discrepancies in the citations. The court found that the junior solicitor had made an error and ordered her law firm to pay costs. Justice Bernard Murphy noted that the false citations likely stemmed from the use of generative AI, which can fabricate information.

The Office of the Victorian Legal Services Board and Commissioner has identified the improper use of AI as a significant risk in its latest outlook. A spokesperson emphasized that AI lacks the judgment and ethical standards of a trained lawyer. In response, the Supreme Court of Victoria and the County Court of Victoria have established guidelines for the use of AI by legal professionals. The Victorian Legal Services Board has collaborated with counterparts in Western Australia and New South Wales to issue a statement regarding AI's role in legal practice.

Counsel remarked that the courts are attempting to manage the challenges posed by AI technology in legal documentation. "It's basically saying, 'Hey, look, you can't just use this randomly, and you can't generate footnotes or citations because we know that AI has this capacity to hallucinate, and you can't actually generate witness statements affidavits using AI,'" she stated. The board is increasingly focusing on cases where lawyers have misled the court with false citations generated by AI. If misuse is proven, disciplinary actions may include reprimands or further education, and serious cases could lead to prosecution before the Victorian Civil and Administrative Tribunal for professional misconduct.