Regulation, AI accountability, and the human layer.
Legal practice has been regulated for centuries by professional rules, court oversight, and personal responsibility. AI-assisted legal work sits outside almost all of that. The systems I build are designed around the principle that the duty has not moved - and the early case law, the EU AI Act, GDPR, and the bar opinions emerging across jurisdictions all point in the same direction.
- The question
- Who is accountable when an AI-assisted legal output goes wrong?
- The short answer
- The human practitioner who signed it. Every reported case and every regulatory framework reviewed below points the same way.
- The design consequence
- Systems must keep the reviewer able to discharge that duty, never displace them from it.
- Disclaimer
- Observation about how the work should be approached. Not legal advice.
Why legal work is regulated at all
Legal practice is one of the most regulated forms of professional work in any common-law jurisdiction. Rules on conduct, conflicts, confidentiality, client funds, advertising, supervision, and admission all exist because the consequences of bad legal work fall on people who are not in a position to evaluate the work themselves. A client cannot reliably tell whether a contract clause protects them, whether a statute was correctly applied, or whether a case citation actually exists. The apparatus exists because the asymmetry exists.
In Sri Lanka, the shape of that apparatus is familiar. The Supreme Court controls admission to the roll of Attorneys-at-Law. The Bar Association of Sri Lanka maintains the rules of professional conduct. The Judicial Service Commission and the courts supervise the conduct of practitioners. Personal responsibility runs through every layer: a practitioner is not free to outsource it, and there is no version of the practice in which a software tool absorbs the duty. The same structural logic - personal accountability of the admitted lawyer - runs through the English, Australian, Indian, and Singaporean regimes that Sri Lankan practice is in conversation with.
The regulatory gap
AI-assisted legal work sits outside that apparatus. There is no admission process for a model. There is no professional rule prohibiting a model from acting outside its competence. There is no court that supervises the model's day-to-day output. The model has no duty of confidentiality. It has no duty of candour. It owes no fiduciary obligation to the client. It has no duty at all.
This is not a problem of capability. Capability is improving steadily. It is a problem of accountability.
When a model produces a confident wrong answer, the question of who is responsible has only one defensible answer: the human who used it. That answer holds regardless of how impressive the output looked, and it holds well before any regulator has formalised the rule, because the duty was already on the practitioner under the existing apparatus.
What the cases already say
The early case law is small but consistent. Each reported decision treats AI use as a delegated activity that does not relieve the lawyer of the underlying professional duty.
Mata v. Avianca, Inc.
22-cv-1461 (PKC), S.D.N.Y. (22 June 2023)
Two New York lawyers filed an opposition brief in a personal-injury matter using ChatGPT-generated case citations. Six of the cited cases did not exist. Judge Castel sanctioned the lawyers and their firm, holding that they had abandoned their professional responsibilities by submitting non-existent authorities and then defending those submissions when challenged.
The model produced the error. The lawyers were sanctioned for it. The duty did not transfer.
Park v. Kim
91 F.4th 610, 2d Cir. (2024)
Counsel on an appeal brief relied on AI for citations without verifying any of them. The Second Circuit reached the same conclusion as Judge Castel: the lawyer's duties of competence and candour are not discharged by delegating to a tool that has neither.
Confirmed at appellate level: AI use is delegation, not substitution.
US District Court standing orders
Multiple districts, 2023 - 2025
Several US district courts have issued standing orders requiring lawyers to certify whether AI was used in any filing and to confirm that all citations were independently verified. UK courts have begun publishing similar warnings; Australian state bars have issued comparable guidance.
None of these orders treat the model as an actor with duties. They all reach for the human.
The bar opinions
Professional-conduct opinions have begun to formalise the same principle in writing. ABA Formal Opinion 512 (July 2024) addresses generative AI specifically and concludes that the existing duties of competence, confidentiality, communication, supervision, and reasonable fees all apply to AI-assisted work. The model's involvement does not reduce any of them. Where AI assistance is material to the work, lawyers may need to disclose its use to the client. Where the AI is non-lawyer assistance, the lawyer's supervisory duties extend to it.
Several US state opinions - Florida, California, New York, New Jersey - have reached compatible conclusions, often with more concrete operational guidance. The consistent thread is that AI is treated as a category of delegated work, the kind that has always been governed by the duty of supervision (Model Rule 5.3 in the US; analogous provisions in most Commonwealth codes). The result is structural: the lawyer remains the responsible person.
The legislative direction
The legislative picture is moving the same way, more slowly and more bluntly. Three frameworks are doing the early work.
EU AI Act
Regulation (EU) 2024/1689 · In force 1 Aug 2024
Classifies AI systems intended to assist a judicial authority in researching and interpreting facts and law as high-risk. High-risk classification triggers human-oversight requirements, transparency obligations, record-keeping duties, and conformity-assessment processes. Obligations land on the provider and the deployer - both human or corporate actors. The Act does not designate the model itself as the accountable party.
GDPR Article 22
Regulation (EU) 2016/679 · In force since 2018
Gives a data subject the right not to be subject to a decision based solely on automated processing that produces legal effects. Decisions affecting a person's legal position - contracts, regulatory outcomes, employment determinations - fall squarely inside its scope when made on automated reasoning alone. Requires meaningful human involvement, not a rubber-stamp approval.
Sri Lanka PDPA
Personal Data Protection Act, No. 9 of 2022
Sri Lanka's own framework, administered by the Data Protection Authority. More recent and less tested than GDPR, but it adopts the same posture on automated decision-making: a person has the right to object to a decision made solely on automated processing where it produces legal effects. The structural point - that a human must be meaningfully in the loop - is identical.
Confidentiality, privilege, and the choice of tool
There is a separate accountability surface around confidentiality. Sending privileged matter to a third-party AI service can risk waiving privilege, depending on the jurisdiction and the vendor's terms. Several state bar opinions in the US have flagged this directly: lawyers must understand where their inputs go, who can see them, whether prompts are retained for training, and whether the vendor's terms of service preserve the duty of confidentiality. Most public-facing AI products do not preserve it without specific enterprise terms.
The choice of tool is itself a discharge of the duty of competence under the rules of conduct, not an operational footnote.
This is one of the reasons the document-execution infrastructure I deploy is self-hosted. It is also why the legal-AI reference corpus excludes anything that is live-client privileged material until it has been properly anonymised and cleared for reuse.
Indemnity, contract, and the procurement layer
The most interesting next move for AI-assisted legal work is not technical. It is procurement. Standard AI-vendor terms of service routinely disclaim accuracy, fitness for purpose, and any liability for downstream loss. Most professional indemnity policies for lawyers do not exclude AI-related errors - but they do not absorb the vendor's disclaimer either. The exposure is real and it sits with the practising firm.
Mature legal-tech contracts will, in time, have to allocate that risk explicitly: warranties on training data provenance, accuracy testing protocols, evidence retention requirements for human review, indemnity carve-outs for hallucinated authorities, and discovery treatment of AI-assisted drafts. Where the regulator is slow, the contracts and the insurance policies will do the early work. That is also where a reviewer-gated, audit-recorded workflow becomes practically defensible: it is easier to insure, easier to disclose, and easier to defend than an unstructured prompt-and-paste process.
What this means for the systems
Concretely, in the systems described elsewhere on this site, the human layer is not a courtesy. It is a control. The drafting workflow ends at a reviewer's signature. The operating record exists to put the relevant facts in front of the person making the judgment, in the order they need to see them. The execution workflow refuses to treat a document as signed until the version is locked, the attachments are complete, the signer authority is verified, and the audit trail is being kept. The reference corpus captures reviewer edits rather than confident outputs, because the edits are what carry judgment.
A model with no duty is a useful tool. It is not a substitute for the person who carries the duty.
These are not separate design choices made for separate reasons. They are the same choice expressed in different places. The systems are built around that distinction because the distinction is structural - and because every framework moving through the major jurisdictions in 2024 and 2025 has reaffirmed it.
A note on what this is, and isn't. I am reading for Attorney-at-Law in Sri Lanka, expected admission 2027. Where regulation is settled, follow it in the relevant jurisdiction. Where it is unsettled, the conservative posture is to design as if the human-duty regime were already binding - because for every reported case so far, it has been.
Next note
The legal-AI reference corpus