ChatGPT for legal work and where a general chatbot falls short

A general chatbot is a language model with a chat window. It predicts plausible text based on patterns learned in training and on whatever you type into the conversation. It does not know your matter, your client, your jurisdiction's latest rule amendments or your firm's standard positions unless you tell it, and it has no built-in way to check its answers against a source.

That design explains both its strengths and its failures. It is excellent at producing fluent text quickly. It is not a research database, not a document management system and not a record of what was done on a file. Treating it as any of those is where the trouble starts.

Used on the right material, a general chatbot saves real time. The common thread is that the input is not confidential and the output is checked by a lawyer who already knows the answer, or close to it.

• Brainstorming. Listing the issues a lease assignment might raise, or the questions to cover in a first client meeting about a small business formation.
• Plain-language rewrites of non-confidential text. Turning a dense standard clause from a public form into a paragraph a client can follow.
• Tone and structure. Making a generic follow-up letter shorter or more courteous, or reorganizing an outline you wrote.
• Explaining unfamiliar concepts at a high level as a starting point, before you read the actual statute or treatise.
• Drafting internal non-client material, such as a training checklist for new staff or a blog post outline.
• Generating alternative phrasings for a sentence when you are stuck.

In each of these cases, the chatbot is a writing aid. The lawyer supplies the judgment and checks the substance.

The failures are predictable, and each one maps to a professional responsibility duty.

• Confidentiality of client files. Pasting a client's contract, medical records or deposition transcript into a consumer chatbot may disclose information relating to the representation to a third party under terms you have not reviewed. Depending on the account settings and terms, conversations may be retained or used to improve models.
• Fabricated citations. A chatbot can produce case names, reporter citations and quotations that look real and are not. Courts have sanctioned attorneys for filing briefs built on such citations, and many judges now require certifications about AI use in filings.
• No matter context. It does not know the other documents in the file, the prior drafts, the client's instructions or the procedural posture, so its answers can be fluent and wrong for this case.
• No audit trail. A browser conversation is not linked to a matter, is not visible to the supervising attorney and leaves no record of who relied on what output.
• Stale or general law. Without a search step against current sources, answers reflect training data, which may predate recent amendments and rarely reflects local rules.
• No source links. Even when an answer is right, nothing shows where it came from, so verifying it takes as long as researching from scratch.

No ABA Model Rule mentions any specific chatbot, but several apply directly, and ABA Formal Opinion 512 (July 2024) addresses generative AI tools in detail.

• Rule 1.1 (competence). Lawyers need a reasonable understanding of the capabilities and limits of the AI tools they use, and must check outputs with a level of review appropriate to the task.
• Rule 1.6 (confidentiality). Before entering client information into a self-learning tool, Opinion 512 indicates lawyers generally need the client's informed consent, and boilerplate language in an engagement letter is not enough.
• Rules 5.1 and 5.3 (supervision). Partners and supervising lawyers must set policies and train staff so that associates and paralegals use AI tools properly.
• Rule 1.4 (communication). In some situations clients should be told that AI is being used on their matter.
• Rule 1.5 (fees). Lawyers bill for actual time spent, and general overhead such as learning a tool is not billable to a client.

Courts add their own layer. A growing number of judges have standing orders on AI use in filings, so check the local rules and individual practice orders before filing anything an AI tool touched.

A firm does not need a long document. It needs clear answers to a few questions that everyone, including staff, can follow.

• Which tools are approved for client information, and which are approved only for non-confidential text.
• What may never be entered into an unapproved tool (client names, documents, facts, medical or financial records).
• Who must review AI output before it goes to a client, a court or opposing counsel.
• How citations are verified (every authority pulled and read in a primary source before filing).
• How AI use is recorded on the matter and reflected in billing.
• When the client is told about AI use, and how that is documented.

Review the policy once a year, and whenever the firm adds a tool or a court in your practice area issues a new order.

The gaps above are not reasons to avoid AI. They are a specification for what a tool used on client matters must provide.

• Contractual terms that client documents are not used to train models, with data encrypted in transit and at rest.
• Answers grounded in sources the tool actually searched, with links to the passage, and an honest statement when nothing relevant was found.
• Work organized by matter, with access limited to the people on that matter.
• An activity log that shows who ran what and when.
• Outputs labeled as drafts that require attorney review.