Despite the Plain Writing Act of 2010, which mandates that federal agencies communicate clearly with the public, many government websites still fall short. According to the latest Federal Plain Language Report Card, 8 of 21 federal agencies received failing grades in compliance or writing quality. Recognizing this challenge, WebFirst set out to discover how large language models (LLMs) could assist in simplifying government language, making it more accessible to everyone.
The Need for Plain Language
Rather than adopting AI for AI's sake, we began by identifying real use cases in content writing where AI could add tangible value. The Plain Writing Act requires federal agencies to use clear and straightforward language, but many documents and websites remain difficult to understand. While the Act doesn't specify a particular grade level, plain language guidelines generally recommend writing at a reading level between 7th and 9th grade. Legal jargon and convoluted sentences are often challenging for subject matter experts to recognize, and writing for the web requires specialized skills. We identified this as an area where AI could provide valuable assistance.
We identified a real need: simplifying complex government language using plain language best practices without losing essential information.
Evaluating AI Models for Text-to-Text Transformations
We conducted a comparative analysis of leading AI models including ChatGPT-4 and Gemini. We specifically focused on their ability to simplify text while adhering to plain language principles. We considered metrics such as readability scores, correct usage of plain language principles, and accuracy.
In our evaluation, we also used HuggingFace and Ollama to run top-performing open-source models, finding that these models effectively reduced the time needed to write plain language content for the web. However, ChatGPT-4 stood out for its higher overall compliance with plain language rules, even without extensive customization.
All the LLMs we evaluated were able to effectively transform complex original text into clearer, more understandable language by:
- Eliminating Jargon: Replacing complex terms with simpler words.
- Using Active Voice: Making sentences more direct.
- Breaking Down Sentences: Simplifying long, complicated sentences
Before-and-After Transformation
ORIGINAL TEXT: "Pursuant to the regulations stipulated under section 5(a) of the Administrative Procedures Act, stakeholders are hereby notified of the impending modifications to compliance requirements, effective immediately upon publication in the Federal Register."
We also employed Retrieval-Augmented Generation (RAG), a technique that combines document retrieval with language model generation to enhance the accuracy of AI outputs. By integrating the Federal Plain Language Guidelines directly into the model's training data, we ensured that the AI's suggestions were specifically referencing federal standards and domain-specific instructions.
Drupal Integration
After evaluating LLMs for text-to-text transformation we proceeded to incorporate our custom Plain Language GPT designed to provide Plain Language assistance directly in Drupal’s content editor. Using DrupalAI's suite of modules, we integrated our solution into Drupal’s content management workflow, enabling content editors to revise content for plain language compliance with a single click.
AI is a transformative technology with applications in medicine, agriculture, manufacturing, transportation, defense, and many other areas. It also holds substantial promise for improving government operations. Federal guidance has focused on ensuring AI is responsible, equitable, traceable, reliable, and governable. Third-party assessments and audits are important to achieving these goals.
- U.S. Government Accountability Office
Selecting Secure AI Platforms
Given the constraints of government cloud infrastructure, we focused on secure methods of deploying AI solutions. We evaluated hosting platforms like Azure AI and AWS Bedrock, which allow for full control over data flow within a secure environment. Choosing a self-hosted model enables agencies to mitigate data privacy concerns by keeping all data within system boundaries, adhering to security protocols and full compliance with FedRAMP.
References:
- Center for Plain Language. (2022). 2022 Federal Plain Language Report Card.
- GAO-21-519SP, "Artificial Intelligence: An Accountability Framework for Federal Agencies and Other Entities," June 2021.
