A Case Study of Teaching Public Interest Tech
by Tanina Rostain and Anna L. Stone
Civ Tech: Digital Tools and Access to Justice is an experiential class offered at Georgetown Law Center in collaboration with Mark O’Brien, the Executive Director of Pro Bono Net. In the class, students work in teams with public interest organizations to build web-based apps to increase access to the civil justice system. Since the class was launched, students have built dozens of apps, including apps that help users navigate court processes, understand and act on their rights, or access the law in other ways and apps that expand the scope and efficacy of the services organizations offer. Many of the tools from the class have been developed further and deployed; others serve as demos.
The Civ Tech Course
The course has several important learning objectives connected with the adoption of technologies in the access to justice field. One theme, which runs through their work on individual projects and in class, is the potential and limitations of technologies to improve access. Guest speakers, who are experts in the field, supplement and deepen students’ understanding of the civil justice landscape. In addition, students are assigned short readings and brief reflection papers to ensure they are aware of the opportunities and challenges of integrating tech in the access to justice ecosystem.
In the course, students acquire basic technical skills by learning to use a software platform that allows students to engineer legal expert systems. They also acquire basic UX, human, and community design skills through in class discussion and client collaboration. Among the more general skills they develop are professionalism, through supervised interactions with organizational clients, teamwork, and basic presentation skills.
The Course Projects
The centerpiece of the class is team projects in which students collaborate with organizations to build technological tools to improve access to the civil justice system by helping users navigate court processes, understand and act on their rights, or access the law in other ways. Students devote most of their effort over the course of a semester to working on one project. By the end of the semester, we expect students to have created a working tool that is ready (or near ready) to be deployed.
As they work on their projects, students learn how the technologies they are developing support their organizational partner’s service delivery model. Students working on an expungement app with a public defender’s office, for example, will learn about how the tool will support the office’s other services. Similarly, students working on a legal diagnosis app for social workers serving the elderly will learn how the tool enhances coordination between social workers and lawyers to better assist clients with their legal needs.
We devote in-class time to teaching students about the broader context for their projects. Over the course of the semester, civil justice advocates speak to the class about the access to justice crisis and the role of technologies in ameliorating it. Students may learn, for example, about tools to assist tenants to bring housing conditions complaints, debtors to file for bankruptcy, and low-wage earners to address wage theft. Students also hear about the limitations of digital tools to lowering barriers to access to justice, including the difficulties for users with low literacy and those with limited access to the internet to find and use digital tools and the challenges of finding sustainable business models to maintain and scale access to justice technologies. To encourage critical thought about these issues, we require students to submit several brief reflection papers over the course of the semester.
Students who don’t code find the class appealing because it does not require a programming background. We use a software platform developed for lawyers by a commercial company named Neota Logic. The software allows non-programmers to create web-based expert systems (i.e., apps) with customized interfaces, complex logic, automated document production, data collection and analysis, and other functions. All students master the platform within the first few weeks of the semester, and most are comfortable using its more complex functions by the end of the term. Building apps using Neota Logic requires embedding legal and non-legal rules into a reasoning engine and instills an “engineering” mindset that differs from the legal analytic skills taught in law school. Students learn to develop solutions for categories of problems, rather than addressing each one separately, and experience the satisfaction of building a tool that takes inputs and produces outputs.
Picking and Developing Projects
Over the years, the class has worked with a range of organizations: civil legal aid providers—which specialize in intimate partner violence protection, immigration, landlord-tenant, and consumer rights, among other areas—public defender offices, non-legal service providers, law school clinics, courts, and administrative agencies. The most important criterion for the selection of a legal tech project is that the organization has a vision of how the technology will complement or increase the efficacy of its services.
In addition, we consider the types of community needs the tool is intended to serve, whether the scope of the project is appropriate for the class, whether the functionalities requested are a good fit for the software we use, whether a very similar tool already exists in the organization’s ecosystem, and whether the organization has a deployment plan. Projects selected are either self-help tools, tools to be used by trusted intermediaries (such as social workers or tenant rights advocates), screening tools like an eligibility and triage app that support an organization’s service delivery model, or some combination. Organizations are typically based in the United States or Canada, and teams have collaborated remotely with organizations in states as far flung as Texas, Hawai’i, and Alaska.
To surface the most promising projects, we distribute a request for proposals (RFP) through civil justice networks, including the listservs of Legal Services Corporation grantees, Pro Bono Net, and the Self-Represented Litigants Network. In addition to setting forth application requirements, the RFP describes what the students will bring to the project and explains the expectations for the participating organization. Most importantly, the organization is expected to make a subject matter expert (SME) available to work with students on a weekly basis. In addition to making expectations clear from the outset, the RFP plays a forcing function, strengthening client motivation and commitment to the project.
When the course was first taught, we invited students to come up with projects themselves. A practical challenge we encountered was persuading organizations to adopt the tools students had built. More importantly, we wanted students to get away from the mindset, often associated with the tech world, that they could parachute into a setting, identify a problem, and fix it. Instead, consistent with the class’s goal to teach students about the civil justice system, we want students to appreciate that the organizations are the experts: they are embedded in an ecosystem of services and have a deep understanding of their clients’ needs.
Keys to Working with Client Organizations
- Designated subject matter expert + guarantee of 1+ hour/week
- Clearly defined need or problem to be addressed
- Concrete plan for deployment of the app
Scoping the Project
The students’ first challenge is to work with the client-organization to determine the scope of the app. Some clients have too high expectations for what the students can accomplish or what the platform can do. Others have too modest expectations. During the course of developing the tool, students and clients engage in an iterative process during which the students educate the clients about the technical capabilities of the software and the design principles taught in class. At the same time, clients educate students about the people they serve, their service delivery model, and their role in the larger civil justice ecosystem. This give and take includes a discussion of what functions the app should perform—Should it give guidance? Produce certain documents? Collect data?—and how applicable law should be embedded in the app. At the same time, client organizations are teaching students about the needs, interests, and goals of their target populations. The project scoping process mirrors the process that legal technologists and subject matter experts engage in to produce a successful legal digital tool.
Key Scope and Scale Questions
- Intended user— or end user of the legal system, trusted intermediary, or organizational client.
- Desired output(s) of the app
- Typical use case, including circumstances that will lead user to the app
- Jurisdiction(s) covered & variations in legal rules Adding additional jurisdictions with different laws or rules on an issue is typically beyond the scope of a one-semester course.
- Sustainability, including frequency of change in the applicable law(s) For sustainability, app components dealing with an area of law that changes frequently may need to be minimized or eliminated.
Design and Prototyping
UIX, Collaborative and Community-Centered Design Students design their apps using principles of human and community-centered design. In creating an app’s interface, students borrow concepts from UIX, including color theory and web page design, and work closely with the SME on questions of layout and flow. In general, the intended user is represented by the organization. To the extent feasible, students pilot their apps in collaboration with intended users.
Plain Language One of the biggest challenges students face in the legal design process is translating applicable legal concepts and rules into plain language. We ask students to be mindful of the potentially limited language abilities and legal system familiarity of the intended users of their app and instruct students to use “plain language”—language that is easy to read and understand and avoids technical or legal jargon. This turns out to be the biggest challenge for students (and many SMEs), who have difficulty communicating complex concepts in simple and clear terms. The technical language of law adds another level of difficulty. Many legal concepts are difficult to communicate in non-legal terms, and legal services providers may be resistant to using simpler language for fear that important legal subtleties will be lost.
Prototyping and Testing As soon as teams have a functioning app prototype, they work with their organizational client to test it with its intended users and make modifications based on the feedback they receive.
Decision-Making During the course of the semester students are encouraged to propose ideas to their client organizations. Consistent with the model of the class, client organizations have final say about all aspects of the app, including the issues addressed, the design of interface, and the language used.
End of the Semester and Beyond
Students are required to produce a working (i.e., deployable or nearly deployable) app by the end of the semester. Teams demonstrate their apps in a pitch-style competition, known as the “Iron Tech Lawyer Competition,” which is refereed by outside judges who are leaders in the access to justice and technology fields. The competition is attended by subject matter experts, legal technologists, and other interested parties and streamed online before a large audience. Beginning in 2020, the winner of the competition advanced to the international Iron Tech Lawyer Invitational to compete against teams and apps from other schools around the world.
At the end of the semester, students are required to submit “as built” documentation of their apps to assist organizations to maintain them and add features. After the semester is over, we work with Neota Logic to finalize the apps according to client specifications. Clients have final say about whether to deploy an app and share the intellectual property in the interface and rule structure.
Ongoing Challenges and Lessons Learned
Class Trade-Offs A major teaching challenge is imparting the skills needed to build a good tool in a single semester. Students need to master the software, which takes a few weeks, and work with the organization to learn the subject matter of the app, and design and pilot it. They also have to prepare for a major presentation. We have made a tradeoff between mastering the authoring tools in order to build a working app and learning human- and community-centered design. Our solution is to defer the collaborating organizations in the design process under the view that they are in the best position to say what tools will best serve the targeted population. The risk is that an organization will not be in a position to make that call.
Supervising Teams Teaching students to work in teams, an important element of this class, is time-intensive and requires close attention to group interactions and dynamics. Team supervision is enhanced through the work of teaching assistants.
Organizational Commitment From the beginning of the course, it was difficult to ensure that organizations, which have limited capacity, would devote the necessary time to the project. Although we have gone far to address this challenge by articulating expectations up front, the problem still arises occasionally.
Deployment Often, apps require final changes and finishing touches before they can be deployed. In addition to gathering the complete documentation from students at the end of the semester, planning for how to satisfy these final needs is crucial to enabling a successful handoff to the client organization and, ultimately, deployment of the app.
Sustainability Over time, apps need to be maintained and updated. Organizational needs, service delivery models, and governing legal frameworks all may change, necessitating changes to the app. This ongoing maintenance requires resources in the way of funding or technical expertise and can be a labor-intensive activity for client organizations.
Scaling Building tools that scale in the access to justice landscape is very challenging. For one, organizations want bespoke apps that fit into their organizational delivery model and are adapted to the clients they serve. More fundamentally, the law is highly disaggregated. As a result, a tool built in a particular city, county, or state, does not easily transfer to another location.
Ten years of teaching the course have yielded several lessons:
- Decide on teaching priorities; only so much can be done in one course.
- Solicit organizational buy-in through the RFP process.
- Encourage client organizations to drive the development process; active engagement by clients increases the likelihood the tool will be deployed and used to its fullest potential.
- Manage student and client expectations of scope and scale; the fancier the project, the more difficult it will be for the students to complete in one semester and for the organization to maintain in the long-term.
- Require students to submit complete app documentation and devise with their client organization a plan for handing off the app.
Example: The D.C. Name Change App
An example of a successful client organization and student team collaboration is the D.C. Name Change App, created in 2019 by a team of four students who worked with the nonprofit organization Whitman-Walker Health to create an app for transgender adults seeking to change their name in the District of Columbia. The app was named the Overall Winner of the 2019 Iron Tech Lawyer Competition.
Team Composition The student team consisted of four students: one fourth-year evening student and three second-year full-time students. The team was split equally by gender.
Students’ Prior Experience Within the team, one student had experience working in a consulting environment, one student had prior coding experience, and one student had experience working with non-profit and volunteer organizations. No student had prior experience working with Neota Logic, and no student had prior experience working with transgender or non-binary individuals, who were the target users of the app. The team drew upon its members’ experiences and skills throughout the project, particularly with regard to building a productive, professional relationship with the client organization.
Interpersonal Dynamics None of the team members had previously worked with each other, and a small number of interpersonal conflicts naturally arose between team members over the course of the semester. Given the relatively long-term nature of the team relationship—an entire semester—these conflicts gave the students an opportunity to practice professionalism in resolving conflicts, which the students did.
The student team was paired with Whitman-Walker Health, a Washington, D.C. community health center that specializes in HIV/AIDS care and care for people who are lesbian, gay, bisexual, transgender, or non-binary. Specifically, the student team worked with Whitman-Walker’s Legal Services Team, which seeks to meet the legal needs of Whitman-Walker’s health clients. Whitman-Walker’s project proposal, submitted in response to the RFP, called for a web-based application that would help users, primarily transgender and non-binary adults living in Washington, D.C., to complete the forms necessary to change their legal name.
Communication between Student Team and Client Organization
Initial Meeting: Effective and consistent communication between the student team and their client organization was essential to the project’s success. Like all student teams in the class, the team began building their relationship with Whitman-Walker with an introductory email, followed by an introductory video meeting. During this first video meeting, the team and the organization each identified who would be their primary contact person; the organization shared their vision for the app and described the typical use-case scenario; the team described what the organization could expect from the team in the coming weeks and months; and all participants agreed on action items.
Ongoing Communication: Following this initial video meeting, communication between the team and the organization consisted of email, phone, and video communication throughout the bulk of the semester. At a minimum, the students communicated with the organization at least once a week. Face-to-face meetings also occurred several times throughout the semester, either by video or in person, and they proved key to building trust and strengthening the relationship.
Scope, Design, and Prototyping
Scope of the Student Work The scope of the project was heavily influenced by the court procedures determining the process for obtaining a legal name change. In D.C., an individual seeking to change their legal name must submit a lengthy and complex Application for Change of Name, and the individual may also want to submit a Fee Waiver Application and a Motion to Seal their Application for Change of Name. Thus, the student team and Whitman-Walker quickly identified that the target users would be adults in D.C. who are transgender or non-binary and that the output of the app would be a completed Application for Change of Name, along with a completed Fee Waiver and Motion to Seal if the user desired. The possibility of expanding the app to serve users in Virginia and Maryland—nearby jurisdictions where Whitman-Walker also serves clients—was quickly set aside. Given the differing court procedures and forms in Virginia and Maryland, expanding the app’s scope to include these additional jurisdictions was beyond the students’ capabilities for one semester.
Design of the Solution The design of the app was guided by the organization’s already-developed Name and Gender Change Guide, a detailed resource to help individuals fill out the necessary forms on their own. The Guide served as a useful starting point for creating the app. The student team began the design process by translating the guide into an expert system. Then, students consulted with their SME on questions the guide did not answer.
In addition, the student team took the extra step of participating in one of Whitman-Walker’s monthly Name and Gender Change clinics to see the form completion process in action. Midway through the semester, three members of the student team attended the clinics to observe real clients working with pro bono attorneys to complete the same forms the students were tasked with helping users complete through the app. This observational experience helped the students better understand how users would interact with the app and, in turn, design a more user-centered app.
Prototyping the App Two weeks before the competition, the entire student team met with Whitman-Walker’s Legal Services Team in person to conduct usability testing. During this testing, members of the Legal Services Team utilized the prototype app from start to finish and provided feedback while the students observed and took notes. Then, the group reconvened to discuss overall impressions and priorities for changes to the app before the competition.
Before the student team left, the whole group of students and organization members took a group photo, which the students incorporated into their final presentation at the Iron Tech Lawyer Competition. The strong relationship the student team and client organization built together was evident at the competition at the end of the semester, where members of the organization, including the SME, attended the competition and were able to share in the students’ win.
The App By the end of the semester, the student team created a fully-functional web-based name change app for adults seeking to change their legal name in Washington, D.C. The app guides users through a series of questions to complete up to three PDF court forms. Questions and explanations of complex terms are written in plain language using vocabulary approved by the client organization.
Before starting, users are given clear guidance about how long the app will take to complete and what documents the user may need. The process is subdivided into sections with a progress bar and encouragement provided to the user between sections. Upon completion of the app, the user receives their completed form(s) in non-flattened PDF format by email, downloaded to the user’s computer, or both. The final screen displays next steps for the user and gives the user the option to have these instructions emailed or downloaded along with their completed forms.
Client Engagement The strong relationship the student team built with their client organization was evident at the students’ final presentation at the Iron Tech Lawyer Competition. Members of the organization, including the SME, attended the competition and were able to share in the students’ success.