Tim Fisher Quoted in Massachusetts Lawyers Weekly: Brazil Court Sanctions Lawyers for Hidden AI Prompt in Pleading

Excerpted from Massachusetts Lawyers Weekly.

A Brazilian labor court judge has imposed significant sanctions on two attorneys who secretly embedded a prompt injection in a court filing in an apparent attempt to manipulate an AI system used by the court to process and analyze cases. The hidden instruction, written in white font on a white background, directed the AI to challenge the petition only superficially and not contest the plaintiff’s documents.

The case has drawn attention as a potentially unprecedented example of attorneys attempting to manipulate a court-facing AI system through hidden instructions in a legal filing. Massachusetts District Court Judge Brian D. Palmucci said the decision offers important lessons for judges, lawyers, and policymakers as AI becomes increasingly integrated into legal practice.

Pierce Atwood IP attorney Timothy V. Fisher explained that the attorneys apparently knew the Brazilian court was using an AI model to analyze filings and may have believed they could influence the system's output:

“Apparently, the attorneys were aware that this particular court was using an AI model to analyze filings. I guess they believed that if the judge or another member of the court were looking at a summary or proposed response to their claim that came out of the model, they could command the model to put out a more favorable summary or recommendation based on the claim [they] filed.”

Fisher said he is not aware of any state or federal courts in the United States that have formally adopted similar AI tools to analyze court filings, though he noted, “I know that some judges in their chambers use AI tools for other purposes.” On the challenge of preventing prompt-injection attacks, Fisher said, “I imagine virtually every developer of AI models and tools tries to put guardrails in place to prevent these types of prompt-injection attacks. But a lot of the models are still vulnerable in some way or another.”

Fisher characterized the attorneys' actions as fundamentally different from more familiar examples of AI misuse in litigation. “No matter how you look at it, it was an attempt to bypass the normal judicial process and normal judicial reasoning by tricking a model into agreeing with you based on something the attorneys went to lengths to hide in the document. This is quite a departure from the typical AI misuse cases we’ve seen in the courts.”

Click here for the complete article by Eric T. Berkman