Ilan Barzilay Quoted in Massachusetts Lawyers Weekly: Hallucination-laced TRO Raises Alarm Bells on AI use by Courts

Excerpted from Massachusetts Lawyers Weekly

A Mississippi federal court’s use of generative AI in drafting a temporary restraining order has raised new questions about how courts should use AI and what safeguards are needed. 

In Jackson Federation of Teachers v. Fitch, an initial order contained inaccurate parties, allegations, declarations, and statutory language that were not supported by the record. The order was subsequently removed and replaced, but the incident prompted scrutiny from the U.S. Court of Appeals for the Fifth Circuit and renewed attention to the risks of AI-assisted judicial drafting.

As Massachusetts and Rhode Island courts develop permanent guidance on generative AI, legal professionals emphasize the importance of human oversight, factual verification, and clear accountability. Massachusetts currently prohibits judges, law clerks, and other court personnel from using generative AI for legal research and writing under interim guidelines.

Ilan D. Barzilay, co-chair of Pierce Atwood’s Artificial Intelligence/Machine Learning Practice stated, “Courts should ‘strongly consider’ requiring notice to parties when AI has been used in preparing an order or opinion. There may be a requirement that, if [AI tools] are being used, there be a documentation and an explicit indicator of where it was used and how.”

The Fitch case highlights the broader challenges courts face in establishing appropriate boundaries for AI use while preserving the accuracy, transparency, and accountability of judicial decision-making.

Click here for the compete article by Pat Murphy