An Interdisciplinary Approach to AI Ethics Training

An Interdisciplinary Approach to AI Ethics Training

Data Science & AI

May, 2023

Sina Fazelpour

Sina Fazelpour is an assistant professor of philosophy and computer science at Northeastern University. His research centers on questions concerning values in complex sociotechnical systems that underpin our institutional decision making. He is a core member of the Institute for Experiential AI and co-founded the Intelligence, Data, Ethics and Society (IDEAS) summer institute for undergraduate students.

Sina recently sat down with PITUNiverse Editor Kip Dooley to share progress on the IDEAS summer institute, where undergraduate students learn from world experts on data science, ethics, computer science, philosophy and law about responsible development of data science and AI. The IDEAS institute is supported in its second year in part through a PIT-UN Challenge grant.

What is PIT-UN?

Learn more

5 Keys to Institutionalizing PIT

Download now

Kip Dooley: Sina, you’re about to run the second cohort of an interdisciplinary summer institute on AI. How did the IDEAS institute come about? 

Sina Fazelpour: The motivations were twofold. First, I have both a technical background in engineering and a philosophical background in the values of technology. AI is a sweet spot for me as a practitioner and educator because AI systems very clearly create both benefits and burdens, whether in the context of allocating medical resources or hiring or some other domain. It is always going to be a complicated issue. Technologists working on AI need to be able to ensure that these systems simultaneously work in ways that respect privacy, lead to just and fair outcomes, and are robust in their performance. This is a very complex task, and we really don’t yet have good models for how to do it well. 

One of the key things missing from the puzzle is an interdisciplinary perspective. We cannot approach these problems from solely a technical perspective, nor solely a humanistic or philosophical perspective. A technical assessment without ethical considerations is insufficient, and you really can’t assess these systems well ethically without knowing at least some of the technical details. Interdisciplinarity is a key skill we need to cultivate for public interest technologists, but our institutions, generally speaking, are behind on this. 

When engineering students take ethics, it’s usually focused on on what not to do.

Most undergraduates interested in technology don’t receive the type of instruction that will prepare them to approach issues from an interdisciplinary perspective. Engineering students have to take an ethics course, but it’s usually focused on how you, as a professional engineer, can avoid breaking rules. They focus on what not to do. They don’t teach you what you ought to do in your practice as an engineer. What values should you consider when designing a product? What ethical processes should you embed in the entire process? We don’t train people how to do this, and that’s extremely problematic.

As a result, when we try to convene interdisciplinary teams (in academia or in industry), people often lack a shared language to even talk to each other. And perhaps even more fundamentally, they don’t know when they have to talk to each other. Engineers might come to a product launch thinking they are all done, only to find that some kind of ethicist or regulator is telling them how the product can or cannot be used. The engineers haven’t considered that throughout the design and development, they have made choices — their own choices! — that are permeated with certain values and ethical assumptions.

So the first motivation for the IDEAS institute was to make sure that we introduced this type of interdisciplinary way of thinking about values and technology at an earlier stage of development for our students, so that interdisciplinary thinking and dialogue is second nature for them by the time they graduate.

The second motivation was about broadening participation in the field of AI and technology development more generally. We know there are significant issues of underrepresentation of different groups, both in scientific disciplines and in the humanities. Both fields need to become more inclusive, and the environments more welcoming to different identities, value sets, and experiences. 

Why? Well, if you pay attention to the headlines, you’ll know that the harms of technology are not equally distributed. They disproportionately fall on members of historically disadvantaged groups. We want to make sure that people who are particularly affected by emerging technologies are among those making the decisions about how they are developed, deployed, and governed. This could mean making technical decisions, making philosophical decisions, legal decisions, regulatory decisions — technology touches every aspect of society, which is what public interest technology is trying to grapple with. We want to enrich the decision-making pipeline.

The IDEAS Summer Institute will take place in two locations this summer: Northeastern and UC San Diego.

For sourcing the guest speakers and creating this interdisciplinary program, did you already have connections with people in different disciplines? How did you bring together people from such a range of disciplines?

Coming from a very interdisciplinary background really helps. In my Ph.D. program at the University of British Columbia, I was in the Philosophy Department, but I was working with neuroscientists and computer scientists. My postdoc at Carnegie Mellon was in philosophy, but I had a secondary appointment in machine learning. So those relationships proved very helpful both in terms of guest speakers and in shaping the program. 

But to be honest, in the first year when funding was scarce, I just invited a bunch of my computer science and philosophy friends to come stay at my place for the week. It was really thanks to the generosity of my friends, who were willing to spend their own money to travel here and stay with me. 

 

We all need a little help from our friends. … How will the program be different this year? What do you hope to build on from the pilot?

On the final day last year, the students were so excited to take what they’d learned and to write a paper, make a video for social media, or design a product. I thought, “OK, the program needs to be two weeks.” The first week will provide the necessary technical background and also the philosophical background about fairness, justice, privacy, and in the second week they can work on group projects and presentations. 

The Network Challenge funding will allow us to do two full weeks. It will be more impactful in terms of training, because the students will actually get to do something with the theoretical background.

We’ll also look to enrich the mentorship piece this year. Last year, we just had guest faculty; this year we’ll also have graduate students who will serve as mentors. Throughout the two weeks, the students will have time to talk to their mentors about their projects and also ask questions about what life looks like in academia or industry. They’ll have the opportunity to build networks. 

We’ll also be inviting faculty from other PIT-UN schools, particularly ones that don’t have programs like this. Here at Northeastern, we have one of the highest densities of people working on the ethics of artificial intelligence. We want to share with others how to run these kinds of sessions, so they can create their own courses and programs and distribute this multidisciplinary ethics training across different types of institutions, not just the ones with a specialty like ours. 

 

To learn more about the IDEAS Institute, visit their website, or the website of Sina Fazelpour.

Data Science & AI

Learn more about Data Science & AI with these editorial features, case studies, resources and more.
See all Resources

Code for Charlottesville Teams up with Civil Rights Advocates

Code for Charlottesville volunteers present findings at the Data Justice Academy

Code for Charlottesville Teams Up
with Civil Rights Advocates

Data Science & AI 

May, 2023

Jonathan Kropko, professor of data science at the University of Virginia

Author: Jonathan Kropko is an Assistant Professor at the School of Data Science. His research interests include civic technology, remote environmental sensing, survival and time series analysis, and missing data imputation. He also leads Code for Charlottesville, the local chapter of Code for America that invites the community to volunteer on important issues.

The Problem

In the U.S., it is unconstitutional for someone to be tried multiple times for the same crime. So why then are people with criminal records punished again and again for past convictions — and even for past charges that did not result in conviction?

Anytime an individual charged with a crime appears in a district or circuit court, the charge creates a criminal record that can be found by the general public. In Virginia, these records can be accessed online in a matter of seconds, facilitating widespread criminal background checks in employment, housing, banking, and other decisions about whether to provide basic services. Schiavo (1969, p. 540) calls this practice “multiple social jeopardy” because although it is unconstitutional for a defendant to stand trial multiple times for the same charge, a person with a criminal record is punished by society over and over again through the withholding of basic services and opportunities. The result is a permanent underclass of people who are denied access to the resources and pathways they need to rebuild their livelihoods. 

A growing movement, led by legal aid societies such as the Legal Aid Justice Center in Charlottesville, Virginia, and nonprofit organizations such as Nolef Turns, advocates for these criminal records to be destroyed (through a process called criminal record expungement) or hidden from public view (what’s known as record sealing). Both expungement and record sealing have been shown to reduce recidivism, which is, ostensibly, an ultimate goal of the justice and corrections systems. 

Prior to 2021, only dismissals and cases of mistaken identity were eligible for criminal record sealing in Virginia. Even then, a qualifying individual had to complete a lengthy and costly petition process. Virginia enacted a law in 2021 that for the first time provided for automatic sealing of criminal records and extended eligibility for sealing to certain low-level convictions, such as possession of marijuana. The law goes into effect in 2025.

While the law represents real progress, it also comes with many restrictions and caveats: an individual can have no more than two records sealed over their lifetime; they must have no arrests or charges in the past three years; they must have no prior convictions; they must wait seven years with no additional convictions in order for the record to be sealed; and more.

All of which begs the question: How many people will actually qualify to have their records sealed once the law takes effect? Answering this question would help advocates decide where and how to focus their lobbying efforts, to ensure that the new law will in fact apply to the maximum number of people with records that deserve to be expunged or sealed.

The Project

Code for Charlottesville, a volunteer group of tech professionals and students that I lead, worked with the Legal Aid Justice Center and Nolef Turns to apply the tools of public interest technology to help answer this question. 

Our task was simple, but not easy: collect all public records from the Virginia district and circuit criminal courts between 2009 and 2020; anonymize the records; and then count the number of records that would qualify for automatic sealing or petition sealing. 

For any PIT project, it’s important to ask what data is available, how it was collected, and if there are any privacy concerns. 

Code for Charlottesville volunteers present findings at the Data Justice Academy
Code for Charlottesville volunteers present findings at the University of Virginia Data Justice Academy
University of Virginia School of Data Science

We used bulk data scraped from the web by Ben Schoenfeld, a computer engineer and civic tech enthusiast. While the current Online Case Information System 2.0 bans web scraping, Ben collected the data from version 1.0 of the system, which had no such restriction, and replaced individual defendants’ names and dates of birth with an anonymized numeric ID. This allowed us to use the entirety of a defendant’s record without knowing the defendant’s identity. Because the data was anonymized, we were confident that the solutions we built would not cause further harm to the people in the database.

In total, the data contains more than 9 million individual court records and more than 3 million different defendants. Code for Charlottesville volunteers built a decision tree-based classifier to translate all of the restrictions in the law into logical conditions that can be evaluated quickly by a code compiler. This function takes in all of a person’s court records and outputs a list that identifies which of the records would qualify to be automatically sealed, which would be eligible to be sealed by petition, and which would be ineligible for sealing.

The Impact

According to our findings, more than 1.4 million records from 2009 to 2020 will immediately qualify for automatic record sealing once the law is implemented in 2025. More than 1 million additional records will become eligible if the individuals with these records avoid any convictions for the rest of a wait period. And 3 million more cases will, immediately or pending a wait period, be eligible for sealing by petition

We used our model to calculate how many more people would be eligible for record sealing if specific restrictions were loosened or removed. We even broke these counts down to the level of the Virginia House of Delegates or Senate district so that the Legal Aid Justice Center could show a delegate or senator the results for their district, making the impact directly visible to the decision makers.

The LAJC used our results in discussions with the Virginia House and Senate to advocate for specific changes to the 2021 law that would expand record sealing access to even more people. This project demonstrates how public interest technology — even when the group of workers is small — can provide right-sized tech tools that support democracy and advance justice.

To learn more, visit Code for Charlottesville or contact Prof. Jonathan Kropko.

Data Science & AI

Learn more about Data Science & AI with these editorial features, case studies, resources and more.
See all Resources

GAEIA: A Global Collaboration to Grow Tech Ethics

Cal Poly's Digital Transformation hub

GAEIA: Building the Future of AI Ethics

Data Science & AI

May, 2023

Soren Jorgensen, cofounder of the Global Alliance for Ethics and Impacts of Advanced Technologies

Author: Søren Jørgensen is a Fellow at the Center for Human Rights and International Justice at Stanford University, and a co-founder of GAEIA. He founded the strategy firm ForestAvenue, which is based in Silicon Valley, Brussels, and Copenhagen, and previously served as the Consul General of Denmark for California.

Elise St. John, co-founder of the Global Alliance for Ethics and Impacts of Advanced technologies

Author: Elise St. John heads Academic Programs and Partnerships at California Polytechnic State University’s New Programs and Digital Transformation Hub, and is a co-founder of GAEIA. She builds and manages cross-disciplinary teams, and designs and leads research and innovation projects that utilize advanced technologies across sectors.

Since ChatGPT’s release in November 2022, public awareness of AI ethics and implications has exploded. As companies and lawmakers grasp for resources to meet this moment with clear and comprehensible strategies for weighing AI’s risks and rewards, what do we in the academy have to offer them?

In 2021, we (Søren Juul Jørgensen, Stanford, and Elise St. John, Cal Poly) launched the Global Alliance for Ethics and Impacts of Advanced Technologies (GAEIA), an interdisciplinary and multicultural collaboration to help companies and governments systematically consider the risks and benefits of AI. We’re excited to share with our PIT-UN colleagues some insights and resources from our journey with GAEIA, and possible directions for growth and expansion.

The Global Alliance for Ethics and Impacts of Advanced Technologies

Each year, GAEIA convenes a cohort of international researchers to collaborate with industry experts to investigate new, pressing ethical considerations in technology use and to develop methodologies and training tools for weighing risks and benefits. Our work is guided by a few key principles:

Changing cultures and norms within industries and companies is just as important as developing strong oversight and regulation of the tech industry. Diversity of geography, culture, race/ethnicity, gender, and values is of paramount importance in creating our methodologies and training tools.

Interdisciplinary collaboration is key to our work and to the future of ethical technology development, deployment, and governance. Here is what these principles have looked like in action.

 

Culture Change

I (Søren Jørgensen) worked in and alongside tech startups during the “move fast and break things” era of Silicon Valley’s early 2010s. Having experienced firsthand how damaging this ethos could be, I moved into a fellowship at Stanford, doing research and advising companies on ethics considerations. A German insurance company CEO said something to me in one of my early conversations at Stanford that really stuck with me: “Please, no more guidelines!”

Of course we need guidelines, but his point was that guidelines without culture change are just another set of rules for corporate compliance. How do you develop a company culture where people care about and understand the risks of technology? Our hypothesis with GAEIA is that companies need simple, iterative processes for collaborative ethical assessment and learning. 

Guidelines without culture change are just another set of rules for corporate compliance.

The first tool we developed is a simple template to iteratively assess the ethics of a technology by asking the kinds of questions that public interest technology prompts us to consider:

  • What is the problem we’re trying to solve with this technology?
  • How does the technology work, in simple terms?
  • How is data being collected and/or used?
  • Who is at risk, and who stands to gain?
  • What is our business interest here?
  • Is it fair? Is it right? Is it good?
  • What action should we take, and how will we communicate our actions?
  • How will we evaluate the impact and apply these insights?

To effectively pressure test this model, my colleague Elise St. John and I knew we needed a diverse, interdisciplinary global cohort of collaborators to mitigate against the kinds of bias and reductive thinking that cause so many tech-based harms in the first place.

The Need for Diversity

I (Elise St. John) joined Søren in 2021 to help organize and operationalize the first global network of collaborators, which would focus on the use of AI and advanced technologies in the financial sector. My background is in education policy research with a focus on issues of equity and the unintended outcomes of well-meaning policies. It lent itself quite well actually to the examination of unintended impacts of advanced technologies. At Cal Poly, I work in digital innovation and convene cross-disciplinary student groups to work on real-world public sector challenges through Cal Poly’s Digital Transformation (Dx)Hub. 

Images courtesy of Cal Poly

Cal Poly's Digital Transformation hub

When I reviewed the literature and explored the various academic groups studying tech ethics and the social impacts of financial technology at the time, it became apparent how very Western-centric this work was. Because public interest technology asks us to engage the voices and perspectives of those most exposed to and impacted by technological harms, we knew that the network we convened needed to be international and multicultural. This consideration is especially urgent vis-a-vis AI systems because they have the capacity to marginalize and silence entire populations and cultures, and to exacerbate existing inequalities, in totally automated and indiscernible ways. 

Our first cohort consisted of over 50 M.A.- and Ph.D.-level researchers representing Africa, the Americas, Asia, and Europe. Using the DxHub model, we broke them up into five groups, each of which worked with an industry adviser to consider real-world ethical dilemmas that companies are facing, using the GAEIA template. In biweekly meetings, the scholars and industry advisers discussed both new and potential ethical dilemmas that fintech services and novel data sources, for example, might inadvertently create. The advisers also spanned geographical regions, further diversifying the ethical frameworks and industry perspectives brought to the conversation, and then we also came together in monthly inspiration sessions to meet with other leading thinkers on ethics, AI, and fintech. 

Public interest technology asks us to engage the voices and perspectives of those most exposed to and impacted by technological harms.

The value of a truly global and diverse cohort was evident at several points. For example, one of the students introduced an ethical dilemma associated with “buy now/pay later” services. The consensus among many of the Western participants was that such services carry too much risk for users and are inherently prone to exploitation. A student from one of the African nations pushed back on this assessment, though, pointing out the opportunities that these systems could hold for the roughly 45% of people in sub-Saharan Africa who are unbanked. This opened up space for weighing the pros and cons of such a technology in different cultural and economic contexts, and it led to further conversations about the role of regulation vs. innovation, for example. These were very humbling and important moments, and they were exactly the kinds of conversations that need to become the norm in technology development, deployment, and governance.

We also had participants from Kenya, Brazil, and India, which are very exposed to climate disasters, develop a Global South working group. In our current cohort, students in Turkey and Ukraine who are living through natural disasters and war have also built connections and held separate meetings to explore how AI tools might provide swift and effective trauma relief in future crises.

Tech's Future Must Be Interdisciplinary

We intentionally recruited participants from across disciplines. Our two cohorts have featured M.A. and Ph.D. students from engineering, finance, law, philosophy, psychology, and more. Fundamentally, we want our students to be able to speak across many disciplinary languages. Technology is not just the domain of computer programmers. It is embedded in all aspects of society and the organizations where we work. Human resources managers have to understand how to communicate with engineers; product managers have to know enough about psychology to ask the right questions about enticement and deception; entrepreneurs need to be able to consult sociologists about the impacts of technologies on different communities. The list goes on. 

We believe that an interdisciplinary approach is not a “nice to have” but a “need to have” for businesses going forward. There’s a growing understanding of the potential risks that businesses face when they don’t have robust ethical decision-making processes: high fines (especially in the European Union), reputational risk among consumers and investors, and the demand from current and prospective employees that companies do no harm and live out good values. 

Having worked with hundreds of organizations during our careers, we can say with confidence that most of them don’t want to do bad things. They fundamentally want to understand risks and avoid them, which is why we’re designing the GAEIA resources and platform within the aspirational frameworks of learning and culture change, not corporate compliance. You can find good examples of how this approach has worked in the education sector. When educators are encouraged to develop genuine inquiry-oriented approaches to data use and systems change in response to accountability measures, they become invested in the accountability process and changing outcomes. Similarly, we want leaders and employees to be invested in ethical decision making, to set real metrics that not only ensure legal compliance but also lead to products and services that are profitable while at the same time aligning with the public interest.

What's Next for our Global Cohort

This work started as a project during the COVID-19 pandemic. At the outset, we didn’t know it would turn into a recurring cohort-based model and that we would further develop the model with the formation of GAEIA. In the first year, students were Zooming in from lockdown and quarantine and were sharing their diverse experiences as the waves of COVID-19 spanned the globe. 

The project’s goal was to break down institutional and sector-specific silos, and bring together a cross-disciplinary, global group of scholars to develop a pipeline of leaders versed in the ethics of advanced technology use. We got that and so much more. 

We are currently collaborating with people at the Center for Financial Access, Inclusion and Research at Tec de Monterrey (Mexico), who have expressed interest in forming a GAEIA chapter for undergraduates, and we are working now with Strathmore University Business School in Kenya on the development of a certification program. There is an emerging network not unlike PIT-UN that can help universities around the world build capacity and support for PIT research and curricula. 

We should also mention the inherent value of building a tech ethicist community across cultures and geographies. The students independently set up social hours on Zoom that were  structured around simple, fun aspects of culture like favorite foods and music. Students from China, Kenya, Germany, and the U.S. would show up on Zoom, whether it was 6 a.m. or 6 p.m. locally, with their favorite beverage. Getting to know more about each other’s lived realities, and bonding over simple human activities, even while far away, is the ground for understanding how AI and advanced technologies affect each of us in distinct ways.

Data Science & AI

Learn more about Data Science & AI with these editorial features, case studies, resources and more.
See all Resources

Grounding Principles for Understanding and Regulating AI

Grounding Principles for Understanding and Regulating AI

Data Science & AI

May, 2023

Author: Maria Fillippelli is the Data Director for the Southern Economic Advancement Project, and a former Public Interest Technology Census Fellow with New America. As a PIT Fellow, she developed and led a strategy to assist dozens of national, state, and local organizations and governments navigate the technical changes to the 2020 Census.

The full piece, excerpted below, is available on the New America website.

A few weeks ago my yoga instructor asked me after class about the hype surrounding ChatGPT and generative AI. Did I think it really was a watershed moment in humanity? It was early in the day, and my immediate response was that only history can determine if this is a watershed moment. However, I added, the actions we take now to understand and weigh the pros and cons of generative AI are incredibly important.

He nodded thoughtfully, and seemed to be gathering his thoughts for a follow-up question, but it didn’t come. As the day wore on, I realized that my answer was clear but probably insufficient. My yoga instructor wasn’t really looking for a single answer; he was, like many of us, looking for a framework to sort through the immense swirl of claims, counterclaims, hype, and critique about generative AI that has been unleashed since ChatGPT’s release in November 2022.

And I realized that I hadn’t seen much in the way of useful frameworks for experts and nonexperts alike to evaluate generative AI products…[continue reading on New America’s website].

Data Science & AI

Learn more about Data Science & AI with these editorial features, case studies, resources and more.
See all Resources

How Public Interest Tech Principles Can Shape the Future of Data Science and Artificial Intelligence

Higher Education and Generative AI

Public Interest Tech Principles Can Shape the Future
of Data Science and Artificial Intelligence

Data Science & AI

May, 2023

Public Interest Technologist Afua Bruce

Author: Afua Bruce is the founder of the ANB Advisory Group, co-author of The Tech That Comes Next and former Director of Engineering at New America Public Interest Technology. In early 2023, ANB Advisory Group conducted a scan of data science for impact programs at PIT-UN member institutions, and also conducted a review of data science projects that have received PIT-UN Challenge funding.

It has been more than a decade since Harvard Business Review declared the profession of data scientist to be the “sexiest job of the century.” Since then, we have seen industry embrace data science as businesses seek ways to differentiate themselves using insights and predictions based on data of their consumers, their markets, and their own organizations. Accordingly, research into data science has increased, and academic institutions have created a number of credentialed programs and research institutes for students and faculty. Data science’s ability to positively impact the speed of operations and efficiency of organizations has been proven. However, as many scholars, practitioners, and advocates have pointed out, that same speed and efficiency can also magnify social inequities and public harms.

Higher Education and Generative AI
An April 2023 PIT-UN webinar explored challenges and opportunities in higher education posed by generative AI

At the same time, the field of artificial intelligence has greatly expanded, as has its embrace by industry and the general public. AI now streamlines how organizations take notes, process payroll , recommend products to clients, and much, much more. Recent product releases and headlines about artificial general intelligence (the theoretical possibility that AI could perform any task humans can perform) have spurred a new round of conversations about how AI could transform human society — or destroy it altogether, depending on one’s perspective. 

With widespread use of AI, the workforce will certainly shift as some tasks and perhaps even entire jobs will be performed by AI systems. Many colleges have made significant investments in AI research programs. Many institutions have recognized the importance of training students in how to design and develop AI systems, as well as how to operate in a world where AI is prevalent. And once again, many scholars, practitioners, and advocates have warned that without more intentional and ethical designs, AI systems will harm, erase, or exclude marginalized populations.

The Intersection of Data Science, AI and Public Interest Technology

Data science and artificial intelligence are two separate, but related, computational fields. As Rice University’s Computer Science department describes:

While there is debate about the definitions of data science vs. artificial intelligence, AI is a sub-discipline of computer science focused on building computers with flexible intelligence capable of solving complex problems using data, learning from those solutions, and making replicable decisions at scale.

Data scientists contribute to the growth and development of AI. They create algorithms designed to learn patterns and correlations from data, which AI can use to create predictive models that generate insight from data. Data scientists also use AI as a tool to understand data and inform business decision-making.

Practically, at some institutions, data science and artificial intelligence programs are sometimes seen as competitors for talent and funding, sometimes seen as collaborators, and sometimes remain organizationally separate. As both data science and artificial intelligence garner more and more attention from universities, students, and employers, we must ask ourselves how to balance the promise and excitement of these fields with the need to develop the associated algorithms responsibly. When systems can automatically have an impact on who is eligible to be hired or promoted, who gets access to housing, or who can receive medical treatments, those designing the systems must understand how to approach problems with not just efficiency and profitability in mind, but also equity, justice and the public good. 

Public interest technology provides a framework to tackle these challenges. “By deliberately aiming to protect and secure our collective need for justice, dignity, and autonomy, PIT asks us to consider the values and codes of conduct that bind us together as a society,” reads an excerpt from PIT-UN’s core documents. 

What could it mean for designers and implementers of data science and AI to “advance the public interest in a way that generates public benefits and promotes the public good”? Public interest technology provides a way to ask, research, and address the following key questions:

  • How do technologists ensure the tools they design are deployed and governed responsibly within business, government, and wider societal contexts?
  • What data sets and training data are being used to design these systems? Do they represent the nuance of human populations and lived experience? Are they representative enough to ensure that analyses or predictions based on the data will be fair and just?
  • How do decisions made early in the data science life cycle affect the ultimate efficacy and responsiveness of systems?
  • How will acceptable accuracy rates be determined for different applications? 
  • Are there ways to turn on and off the algorithms as needed?
  • What accountability structures and auditing systems can be built to ensure the fairness of data science and AI algorithms across industries?

Examples of Public Interest Data Science and AI

Over the past several years, an increasing number of academic institutions have recognized the importance of applying data science and AI in the public interest and have created extracurricular clubs, classroom and experiential courses, and certificate and degree programs that train students to consider how data science and AI affect communities in different ways and how these tools can be designed and deployed in new, beneficial ways. 

A field scan by the ANB Advisory Group shows that students at PIT-UN institutions are learning vital historical context, working on interdisciplinary teams, and translating data insights into language that policymakers, community organizations, and businesses can understand.

For example, in Boston University’s BU Spark!, five program staff assign students to teams and manage semester-to-semester relationships with government agencies and nonprofit organizations. Students have used data science to conduct sentiment analysis of Twitter feeds for a national civil rights organization and regularly provide data analysis for the Boston City Council. Over 3,000 students have learned how to work with real-world, messy data, and how solving data problems can contribute to solving larger organizational or societal problems. In addition to technical courses, students learn critical sociological skills, such as how to understand race in the context of using census data. BU Spark! is one of many programs throughout PIT-UN members demonstrating that labs (including summer programs and practical courses) are an effective way for students to learn public interest tech ideas in real-world contexts and to practice co-design and co-development with affected community partners. 

Penn State’s “AI for Good, Experiential Learning & Innovation for PIT” program was one of a handful of PIT-UN grantees to train both college students and working professionals in the ethics and techniques of artificial intelligence. The program developed a new slate of experiential learning opportunities for college students, along with an online microcredential course for professionals in any sector. While it is important to train the next generation of technologists, we must also consider how to train today’s leaders and decision makers. 

Similarly, Carnegie Mellon University launched a Public Interest Technology Certificate program in 2022. Geared toward employees in all levels of government, the six-month program trained its first cohort in data management, digital innovation, and AI leadership “to create a more efficient, transparent, and inclusive government.” Training mid-career professionals while also building a PIT network that can inform and support their work can lead to real-world impact well beyond the walls of the university.

Key Lessons & Recommendations

These are just two of the many projects across PIT-UN applying a public interest framework to data science and AI challenges. And universities can do even more. Although the development and use of data science and AI differ, some of the application settings and opportunities to affect change have similar underlying challenges. Therefore, the following three recommendations can apply to both data science and AI programs.

1. Produce recommendations for policy work

As federal, state, and local policies and initiatives encourage the advancement of data, government agencies will seek not just support in accessing data, but also access to advanced data science tools to make data actionable. Miami Dade College, for example, worked with the nonprofit Code for South Florida, Microsoft, and the city of Miami to create a participatory web app that helps Miami residents become informed contributors to the city’s budget. In their 2019-2020 PIT-UN Challenge project, MDC created a GIS certificate course for underrepresented students to contribute to mapping the impacts of climate change.

Using data science to make clear policy recommendations or create policy products — especially in collaboration with other stakeholders — is a great way to provide students with experiential learning opportunities while also increasing the reach and impact of public interest tech’s core ideas. 

2. Define PIT competencies for data science and AI

As colleges and universities create and expand both data science programs and AI, both students and professors seek courses grounded in strong research and clear outcomes. Projects such as Georgia State’s Public Interest Data Literacy Initiatives have created individual courses that offer PIT frameworks for data science and AI. We are at a point where PIT-UN schools could collaborate to create an inclusive set of standard competencies. Such standardization could lend more credence and visibility to PIT degrees and could be a prototype for standards required of all data science and AI practitioners regardless of sector.

3. Structure meaningful internships & experiential programs

Students — and even faculty — seek practical experience that they can put on their resumes, describe to potential employers, and use to forge cross-sector partnerships. PIT-UN has consistently funded experiential learning projects to strengthen the pipeline of technologists who understand how to apply data science and AI in the public interest. 

Columbia University and Lehman College’s Public Interest Technology Data Science Corps placed college students in teams to use data science to support New York City agency projects to improve the lives of local residents. Ohio State University placed PIT fellows in state government to encourage young technologists to consider public service, while fostering a culture of collaboration between the public sector and academia. These are just two examples of how meaningful internships and experiential learning speak to the interests of students and faculty while growing PIT’s public reputation. 

Our Task Going Forward

The sustained interest in and excitement about both data science and artificial intelligence bodes well for the future of academic programs dedicated to these concepts. More significantly, the ways in which industries and community organizations operate will change and be changed because of advancements with these technologies. 

Making these changes more positive than negative, and actively reducing adverse disparities, will require sustained work, new ways of training practitioners, and usable recommendations and tools to shape a more just technology ecosystem. Public interest technology’s emphasis on equity and justice provides the necessary lens to guide the development and use of these technologies. As PIT-UN Program Manager Brenda Mora Perea reminds us, it is our job to keep these concepts at the center of all we do and to advocate for social responsibility at every stage of technology design, deployment, and governance. 

 

Data Science & AI

Learn more about Data Science & AI with these editorial features, case studies, resources and more.
See all Resources

Navigating the Generative-AI Education-Policy Landscape

Professor Wesley Wildman, boston university

Navigating the Generative AI Education Policy Landscape

Data Science & AI

May, 2023

Professor Wesley J. Wildman

Author: Wesley J. Wildman is Professor in the School of Theology and in the Faculty of Computing and Data Sciences at Boston University. His primary research and teaching interests are in the ethics of emerging technologies, philosophical ethics, philosophy of religion, the scientific study of religion, computational humanities, and computational social sciences.

Professor Mark Crovella

Author: Mark Crovella is a Professor and former Chair in the Department of Computer Science at Boston University, where he has been since 1994. His research interests center on improving the understanding, design, and performance of networks and networked computer systems, mainly through the application of data mining, statistics, and performance evaluation.

Like many institutions, universities are struggling to develop coherent policy responses to generative artificial intelligence amid the rapid influx of tools such as ChatGPT. Higher education is not known for its ability to respond nimbly to changes wrought by emerging technologies, but our experience thinking through and forming policy at Boston University — in dialogue not just with administrators and faculty colleagues, but also, crucially, with our students — points toward an opportunity to step back and reassess what our goals are as institutions of higher learning and how we can best achieve them. In this article, we describe the policymaking process at BU and the implications each of us is thinking through as instructors of writing and computer languages, two domains that generative AI is poised to disrupt in major ways. 

Generative AI has catalyzed a rare degree of intense discussion about pedagogy and policies.

A recent letter signed by over 27,000 leading academics and tech entrepreneurs calls for a pause on advanced AI development. (It’s important to note that plenty of their colleagues have opted not to sign, or have critiqued the letter). There is indeed reason to worry about both widening economic disruption caused by generative AI and the arrogant or naive belief that the market will self-regulate and nothing too terrible can happen. And while the letter does raise awareness about the dangers of AI, the “pause” the letter calls for is highly unlikely; companies stand to lose too much market share, and countries too much competitive advantage in research and development, to step out of the AI race willingly. Furthermore, it offers little in the form of concrete steps to move responsible AI forward. 

It is against this background that universities are struggling to develop coherent, effective policies for the use of generative AI for text, code, 2D and 3D images, virtual reality, sound, music, and video. As institutions, universities tend to be conservative, multilayered and unwieldy, and well-suited to implementing strategic change over the long-term. They are not so good at adapting to rapid technological change. But this area of policy is particularly urgent, because the assessment of learning has critically depended for a long time on humans performing functions that generative AI can now accomplish — sometimes better than humans, sometimes worse, but often plausibly and almost always faster.

In other words, generative AI has catalyzed a rare degree of intense discussion about pedagogy and policies.

Co-Creating Policy with Students at BU

At Boston University, where we teach the ethics of technology, and computer science, respectively, only a few individual university units have had enough time to devise unit-wide policies (most existing policies are for individual classes). Our unit — the Faculty of Computing and Data Sciences (CDS) — started with a student-generated policy from an ethics class (the Generative AI Assistance, or GAIA, policy), which the faculty then adapted and adopted as a unit-wide policy.

Screenshot of the Generative AI Assistance Policy, created by BU students and adopted by the Faculty of Computing and Data Sciences
Screenshot of the Generative AI Assistance Policy, created by BU students and adopted by the Faculty of Computing and Data Sciences

The GAIA policy is based on several student concerns, expressed as demands to faculty.

  • Don’t pretend generative AI tools don’t exist! (We need to figure them out.)
  • Don’t let us damage our skill set! (We need strong skills to succeed in life.)
  • Don’t ignore cheating! (We are competing for jobs so fairness matters to us.)
  • Don’t be so attached to old ways of teaching! (We can learn to think without heavy reliance on centuries-old pedagogies.)

The GAIA policy also makes demands of students. Students should: 

  1. Give credit and say precisely how they used AI tools. 
  2. Not use AI tools unless explicitly permitted and instructed.
  3. Use AI detection tools to avoid getting false positive flags. 
  4. Focus on supporting their learning and developing skill set. 

Meanwhile, instructors should: 

  1. Understand AI tools. 
  2. Use AI detection tools. 
  3. Ensure fairness in grading. 
  4. Reward both students who don’t use generative AI and those who use it in creative ways.
  5. Penalize thoughtless or unreflective use of AI tools. 

The GAIA policy also explicitly states that we should be ready to update policies in response to new tech developments. For example, AI text detectors that are used to flag instances of possible cheating are already problematic, especially due to false positives, and probably won’t work for much longer.

The GAIA policy is similar to other policies that try to embrace generative AI while emphasizing transparency and fairness. It doesn’t ban generative AI, which runs against the student demand that universities should help students understand how to use such tools wisely. It doesn’t allow unrestricted use of generative AI tools, which runs afoul of the student demand for equal access and fair grading in a competitive job market. It is somewhere in between, which works for now. There are only so many ways of being in between.

The Role of Instructors in the Age of Generative AI

New technologies often create policy vacuums, provoking public interest ethical conundrums — just think of websites for sharing bootlegged music, or self-driving cars. What’s fascinating about the policy vacuum created by generative AI is how mercurial it is. You can throw a policy like GAIA at it and six months later the policy breaks because, say, AI text generation becomes so humanlike that AI text detectors no longer reliably work.

The big breakthrough in AI that led to the current situation was development of the transformer (the “T” in GPT). This approach to deep learning algorithms on neural nets was revolutionary and massively amped up the capabilities of AI text generation. There will be other, similar technological breakthroughs, and it is impossible to predict where they will come from and the effects they will have. Policy targets for generative AI are leaping all over the place like pingpong balls in a room full of mousetraps. Nailing down relevant policy won’t be easy, even for experts.

Educators face the prospect of generative AI short-circuiting the process of learning to think.

New technologies often create policy vacuums, provoking public interest ethical conundrums — just think of websites for sharing bootlegged music, or self-driving cars. What’s fascinating about the policy vacuum created by generative AI is how mercurial it is. You can throw a policy like GAIA at it and six months later the policy breaks because, say, AI text generation becomes so humanlike that AI text detectors no longer reliably work.

The big breakthrough in AI that led to the current situation was development of the transformer (the “T” in GPT). This approach to deep learning algorithms on neural nets was revolutionary and massively amped up the capabilities of AI text generation. There will be other, similar technological breakthroughs, and it is impossible to predict where they will come from and the effects they will have. Policy targets for generative AI are leaping all over the place like pingpong balls in a room full of mousetraps. Nailing down relevant policy won’t be easy, even for experts.

Professor Wesley Wildman, boston university
Professor Wesley Wildman teaches a Data and Ethics class at CDS on Tuesday, February 14, 2023. Photo by Jackie Ricciardi for Boston University

Consider writing. For centuries, we’ve been using writing to help young people learn how to think and to evaluate how well they grasp concepts. Writing is valuable as a pedagogical tool not just because of its outputs (essays), but also because of the processes it requires (articulating one’s ideas, drafting, revising). GPTs allow students to generate the product while bypassing much of the process. Accordingly, instructors need to be more creative about assignments, perhaps even weaving generative AI into essay prompts, to ensure that the value of the writing process is not lost. The GAIA policy is not merely prohibitive. It rewards people who choose not to use generative AI in ways that shortcut the learning process, while also rewarding people who use it in creative ways that demonstrate ambition and ingenuity.

Now consider coding. Surprisingly, the transformer mechanism that works so well to produce human-level language also works well to produce computer “language,” that is, code. Each programming language that students struggle to master — Python, Rust, Java, and all the rest — is just another system of tokens to GPTs. Generative AIs have shown stunning ability to synthesize working code from plain-language descriptions of desired behavior.
 
Just as with writing, so here: Educators face the prospect of generative AI short-circuiting the process of learning to think — in this case, computational thinking, which involves analyzing a problem and breaking down its solution into steps that can be implemented on a computer. This is a sophisticated skill that typically takes years to acquire, and no data scientist can be effective without it.
 
Generative AI will revolutionize the development of software. In fact, we believe most programming in the future will be done using generative AI, yet we’re also convinced that computational thinking will remain a vital skill. To begin with, we’ll need that skill to learn how to craft prompts that elicit the right kind of code from a generative AI. Engineering prompts for generative AI is an emerging domain of inquiry. We need to learn how to do it as practitioners and instructors, and we need to teach our students how to do it.
 

Some Useful Historical Analogies

We believe this computer-language example can help universities grapple with the sharp challenge that generative AI poses to the traditional role of writing in education. Like the software stack, generative AI is enlarging the “writing stack,” promising to eliminate a tremendous amount of repetitive effort from the human production of writing, particularly in business settings. This new world demands writing skills at the level of prompt engineering and checking AI-generated text — unfamiliar skills, perhaps, but vital for the future and difficult to acquire.

In educational settings, instructors produce the friction needed to spur learning in more than one way. We once learned to program in machine language, then in assembly language, then in higher-level programming languages, and now in code-eliciting prompt engineering. Each stage had its own kind of challenges, and we nodded in respect to the coders who created the compilers to translate everything at higher levels back to executable machine language. Similarly, we learned to write though being challenged to express simple ideas, then to handle syntax and grammar, then to construct complex arguments, then to master one style, and then to move easily among multiple genres. Now, thanks to generative AI, there’s another level in the writing stack, and eliciting good writing through prompt engineering is the new skill. Friction sufficient to promote learning is present at each level.

Maybe, just maybe, generative AI is exactly the kind of disruption we need.

It’s not a perfect analogy. After all, high-level coders really don’t need to know machine language, whereas high-level writers do need to know spelling, syntax, and grammar. But the analogy is useful for understanding “prompt engineering” as a new kind of coding and writing skill.

In this bizarre policy landscape, how should universities chart a way forward? How should we handle generative AIs that produce high-quality text, computer code, music, audio, and video — and overcome existing quality problems in a matter of months? That have the potential to disrupt entire industries, end familiar jobs, and create entire new professions, and that are also vulnerable to replicating the bias of our cultures in uninterpretable algorithmic behavior that is more difficult to audit than it should be?

Given the massive questions now facing us, a “pause” on advanced generative AI would be nice. But we cannot pause the impacts of generative AI in the classroom, and we are not convinced that eliminating generative AI from the learning experience is the right path.

We recommend that university leaders and instructors step way back and ask what we are trying to achieve in educating our students. To the extent that universities are merely a cog in an economic machine, training students to compete against one another for lucrative employment opportunities, making them desperate to cut corners and even cheat to inflate their GPAs, then generative AI threatens extant grading practices and undermines the trust that employers and parents vest in universities to deliver valid assessments of student performance.

But if universities are about building the capacity for adventurous creativity, cultivating virtues essential for participating in complex technological civilizations, and developing critical awareness sufficient to see through the haze of the socially constructed worlds we inhabit, then maybe we reach a different conclusion. Maybe, just maybe, generative AI is exactly the kind of disruption we need, prompting us to reclaim an ancient heritage of education that runs back to the inspirational visions of Plato and Confucius.

When students tell us they need our support to help them figure out AI, and warn us not to get stuck in our well-worn pedagogies, we think they’re doing us a great favor. We ourselves need to figure out generative AI and rethink what we’re trying to achieve as educators.

 

Data Science & AI

Learn more about Data Science & AI with these editorial features, case studies, resources and more.
See all Resources

The Legacies We Create with Data and AI

Professor Renee Cummings

The Legacies We Create with Data and AI

Data Science & AI

May, 2023

Professor Renee Cummings

Renée Cummings is a Professor of Practice at the University of Virginia focused on the responsible, ethical and equitable use of data and artificial intelligence (AI) technology. She has spoken to groups around the country and the world, from local school districts to the EU Parliament about the risks and rewards of AI.

She recently sat down with PITUNiverse Editor Kip Dooley to reflect on a big year, and share the frameworks that her audiences have found most helpful for understanding data science and AI.

Kip Dooley: Renée, you’ve had a busy year of speaking engagements with everyone from local school districts to the World Economic Forum. What topics and areas of expertise have you been counseling people about?

Renée Cummings: So much is happening around AI and data science right now, and so quickly. The tools are rolling out with such frequency and ferocity that I have been called upon to discuss everything from generative AI to oversight, compliance, regulation, governance, and enforcement.

I always emphasize thinking about the “three R’s”: the risks, the rewards, and of course, the rights, whether talking about how we integrate AI into education systems, social systems, business — you name it. 

Data is like DNA. How it’s collected and used determines the kinds of algorithms we can design, which are more and more determining access to resources and opportunities.

In terms of your approach to data science and your professional journey, what do you think has prepared you to offer helpful advice at this moment?

I bring what I call an interdisciplinary imagination to data science. My work is not only in criminal justice, but also in psychology, trauma studies, disability rights, therapeutic jurisprudence, risk management, crisis communication, media, and more. My work is about future-proofing justice, fairness, and equity as we reimagine society, systems, and social relationships with AI. My work is also about public education, building awareness around the impact of data on society and democratizing data so we understand the power of data and how to use that power responsibly and wisely and in the interest of the public good as we design responsible AI.

I focus not only on doing good data science, but also on leveraging data science in ways that are equitable and ethical, using data science to build equitable and enduring legacies of success. How can we use this technology to ensure that groups and communities thrive? The goal is to build more equity into our systems, and in the ways in which we design data-driven solutions. It’s really about using data science to build more sustainable and resilient legacies.

 

“Legacy” is an interesting choice of word when talking about AI and data science. Tell me more about why you use that word in particular.

When we design an algorithm, we have the opportunity to use these tools of measurement as a means to enhance access, opportunity, and resources for communities — now, and for generations to come. Data is like DNA. How it’s collected and used determines the kinds of algorithms we can design, which are more and more determining access to resources and opportunities. 

Unfortunately, what we’ve seen for the most part are algorithms that deny legacies, that deny access to resources and opportunities for particular communities, because of issues like the lack of diversity in technology. What we find is that bias, discrimination, and systemic racism are amplified, and certain communities don’t get equal access to resources. What data does is shift power, particularly at the level of systems. Data is about power. Data doesn’t have the luxury of historical amnesia.

What are some examples that illustrate this idea, that data is about power?

We can start with the mere fact that most of the world’s data is owned by five companies. Those companies have created thriving legacies, billion-dollar legacies. They are the ones that governments need to negotiate with over tech regulation, compliance, and enforcement. Furthermore, they set the agenda for what we talk about. We’re all talking about generative AI now, and how it could change — or already is changing — the game, from industry to education. It’s all about power.

Looking at the mad rush to acquire data, from a criminal justice perspective, we’re starting to consider data brokers as cartels, traffickers, smugglers. Think of how companies scrape all kinds of data from the internet to feed large language models. This is creating new systems of power, placing a small group of individuals and companies at the helm of decision making around who has access to resources.

 

You’ve been studying these systems for a long time. How have the questions shifted with the sudden onset of generative AI and the explosion of generative AI applications? 

Generative AI is just a tool, and it can do us some good. But who has access to it? I just had a speaking engagement at a university in Kentucky where many students do not have internet access at home. So when we’re deploying technologies like generative AI, or we’re building smart cities, we’re only focusing on certain geographical spaces that have access to them. Is it going to widen the digital divide? The conversation happening in the U.S. and Europe about how to legislate generative AI is not engaging the Global South. 

We also have to ask whether or not it’s just a lot of hype, because we see the many contractual issues with adopting generative technologies into corporations, or the federal government, because of intellectual property rights. The [Federal Trade Commission] recently was very direct and instructive, talking about generative AI and deception and disinformation. 

I think that primarily it has amplified the questions we have been asking for a very long time, not created new questions. Although it does pose a new threat. And there’s just so much conversation, so much to keep track of, that a lot of people are overwhelmed at the moment. 

Professor Cummings at the World Economic Forum Global Technology Governance Retreat 2022 in San Francisco, June 20th - 23rd. © 2022 World Economic Forum
University of Virginia School of Data Science

For the people who are overwhelmed, what are some things you try to help them reorient toward in order to make the problems and the questions feel a little bit more manageable or at least digestible?

I often say that AI is a new language, and it’s important to become literate in that language. There’s a certain level of digital proficiency we’ll need to be able to function in society as these technologies continue to spread.

It’s also important to understand that this is not a new technology, it’s nearly 67 years old. The improvements in machine learning and deep learning and neural networks have advanced within the past 10 years and have brought forth very successful iterations, but it’s not a new concept. 

These tools can assist you and bring an extraordinary amount of effectiveness or even excellence in the way you do your work. But there are challenges: accountability, transparency, explainability. We’re not able to truly audit these technologies. We’ve got to enter into this space with a certain amount of sobriety instead of being totally overwhelmed.

I often tell people to just breathe and to play with the tools. Use curiosity. Be curious enough to know about the technology and how to use it, with the knowledge that it’s changing the world around you. How is technology changing your world? This is the backdrop we can use to discuss the need for more regulation and governance in the tech space more broadly. 

 

In this environment, where it feels like the public has little or no say in how these technologies are designed and governed, what are the areas or levers that you see as promising areas of intervention?

It’s important to remember that we have a very solid community of AI ethicists and activists working in this space who have the capability and competency to design rigorous and robust guardrails. But a lot of people, the public, don’t understand that AI ethics, tech ethics, and  data ethics even exist and that we all have rights in this digital space. Many of the technologies being developed and deployed impact our civil rights, our civil liberties, our human rights.

When we bring rights to the fore, people wake up. When people understand there’s a technology making decisions about them, and they don’t have an opportunity to participate in those decisions, they start to think about what they can do and what they need to do. They start to think about the lack of agency and autonomy. We all have a right to privacy, to self-actualization, self-expression, self-determination. We also have the right to equal opportunities. These are hard-won rights that people usually are not so willing to give up. 

Again, that concept of the “three R’s” — risks, rewards, and rights — can bring us back to these key questions. 

What always wakes my students up is that concept of legacy: What is the legacy you are designing and deploying?

To what extent can algorithms create equity? Where do you see positive gains that have been made, or possibilities, for algorithmic systems to protect rights and create equity?

One area is government corruption and procurement. Through algorithms, you can account for every dollar and track fraud and corruption through government systems. Every dollar that is stolen through corruption is a dollar that taxpayers don’t have access to, that children don’t have access to. 

Algorithms can help us visualize data around human trafficking and migration, and crisis intervention in times of war and national emergencies. We’ve seen really solid work being done around natural disasters like hurricanes and volcanic eruptions. There’s been some research looking at the effects of earthquake aftershocks in Haiti — encoding buildings and visualizing where and how destruction could take place. 

One other area I can point to, given my background in criminology, is organizations like the Innocence Project looking at ways to deploy algorithms to find cases where there could be wrongful convictions, or records that should be expunged. At UVA, through my data activism residency, we’re developing a tool called the Digital Force Index, which will help people see how much surveillance technology is being deployed in their communities. 

Unfortunately in policing and the criminal legal/criminal justice system, tools like predictive policing have really not delivered on their promises. We hope that tools like the Digital Force Index will spur a more informed, community-led conversation around police budgets, the right to know how much is spent on surveillance tools, where in communities they are being deployed, and whether these tools are truly enhancing public safety or simply vanity projects. The Digital Force Index is the heart of public interest technology.

 

What questions or best practices would you like to see technology designers take on as part of their responsibilities?

What always wakes my students up is that concept of legacy: What is the legacy you are designing and deploying? That brings them back to their social responsibility. Data scientists, whether we’re working on services, systems, or products on behalf of the collective, we are designing futures. What is your legacy? What is the legacy of your family, your community, your generation? 

They start to think about questions around diversity and inclusion, equity, trauma, and justice. How are we traumatizing certain groups with technology? How can we bring a trauma-informed and justice-oriented approach to the ways we’re using data? We have to understand that different communities experience data differently. We don’t want to do data science in ways that will replicate past pain and trauma.

Data carries memory, a troublesome inheritance for particular communities. Those painful past decisions are trapped in the memory of the data, opening some deep social wounds as we attempt to use data to resolve very pressing social challenges and social questions. If we use historical data sets to build tools like large language models, which have been developed with toxic data scraped off the internet, what we risk doing is retraumatizing, revictimizing, groups that have tried so hard to find ways to heal. I’m always trying to get students to ask how we can use data to help communities heal, thrive, and build resilient and sustainable legacies. 

 

Data Science & AI

Learn more about Data Science & AI with these editorial features, case studies, resources and more.
See all Resources

The Role of Public Interest Technologists in an Age of AI Hype

Dr. Suresh Venkatasubramianan

The Role of Public Interest Technologists in an Age of AI Hype

Data Science & AI

May, 2023

Dr. Suresh Venkatasubramanian, Brown University

Author: Suresh Venkatasubramanian is a Professor of Computer Science and Data Science at Brown University. He recently served as Assistant Director for Science and Justice in the White House Office of Science and Technology Policy, where he helped co-author the Blueprint for an AI Bill of Rights. 

 

Since ChatGPT’s release in November 2022, there’s been a lot of discussion about the potentially world-changing implications of generative AI. As a computer scientist who co-authored the Blueprint for an AI Bill of Rights, I don’t see generative AI as a completely new or unusually dangerous threat. Despite extraordinary claims by tech entrepreneurs and some AI researchers that ChatGPT points to an inevitable evolution of general or sentient AI that could enslave or kill us all, I am not worried at this point about AI sentience. I am worried, however, that a critical mass of people will be made to worry about AI sentience, which will distract from the manifold ways that AI-powered systems are already causing harm, especially to marginalized and vulnerable populations – and the role that humans need to play in the regulation and revision of these systems.

The White House Blueprint for an AI Bill of Rights
Screenshot from https://www.whitehouse.gov/ostp/ai-bill-of-rights/

Prior to ChatGPT, many of the companies deploying AI tools framed artificial intelligence as a distinctively nonhuman system. We were told that these systems could synthesize and sort much more data than the human mind ever could, making them neutral arbiters of information free from the limits of human capability and the errors of human bias. It took a lot of work by researchers, advocates, and journalists to show exactly how and why this claim doesn’t hold up. AI facial recognition tools routinely misidentify black and brown people; AI hiring algorithms often exclude women and other historically marginalized groups; social media algorithms are optimized to sensationalize, not to inform or connect people. They are error-prone, and they often amplify patterns of bias in ways that are hard for us to see or understand.

How do we address these harms? By this point, we actually understand the harms quite well. The five main principles we outlined in the Blueprint for the AI Bill of Rights — ensuring system safety and effectiveness, protecting us from algorithmic discrimination, preserving the privacy and limited use of our data, demanding that systems be visible and explainable, and ensuring that we always have human alternatives, consideration, and fallback — represent our best understanding of how to protect people from the harms of unchecked and misguided automated systems.

And lawmakers are taking action on these concerns. State legislatures across the U.S., and across its political spectrum, are starting to experiment with legislation. As I recently wrote with two colleagues for the Brookings Institution, these bills “seek to balance stronger protections for their constituents with enabling innovation and commercial use of AI.” Regulatory agencies have also come out strongly in support of AI regulation. As Federal Trade Commission Chair Lina Khan put it, “There is no AI exemption to the laws on the books.”

A Troubling Rhetorical Shift

There is good momentum in both government officials’ and the general public’s understanding of technological harms. Since ChatGPT’s release, a much wider segment of the population than ever can see what researchers have been saying for some time now: AI-powered systems often deceive, obfuscate, and make unexplained and untraceable errors. 

But I see the proponents of AI taking on a new rhetorical strategy that threatens to derail us. They are now saying the exact opposite of what they said before: AI is not unhuman after all, but it is actually on its way to sentience.

AI sentience is a compelling story. But a good story doesn't make something real.

This shift in rhetorical strategy away from “AI is decidedly not human, and that’s why it is good,” to “AI could be sentient, and we should all be afraid!” threatens to co-opt our genuine collective concern and drive us in directions that don’t make sense, both technically and in terms of mitigating harm. Since ChatGPT was released, at least eight more studies on the harms of AI systems to minority communities have been published. People are losing health care coverage because of biased AI systems. Human lives are at stake right here and right now, but what are we talking about? Hypothetical threats from sentient AI — a technology that does not yet exist. 

AI sentience is a compelling story, one that builds on Hollywood depictions like The Terminator, or 2001: A Space Odyssey. But a good story doesn’t make something real, and we have to wonder why companies like OpenAI and Microsoft and Google that are jockeying for AI market share are creating what is essentially a misinformation campaign about AI sentience. Why are they so keen, all of a sudden, to depict AI as possibly sentient – and themselves as the only ones who can protect us from it?

Our Role as Public Interest Technologists

It is our job, as experts and public interest technologists, to be transparent about what is known — and not known — about generative AI and to seek to understand these systems the same way we researched and exposed the prior iteration of AI systems used for decision making. Our collective confusion about the risks and rewards of generative AI stems in large part from experts and trusted spokespeople doing the opposite, by making overconfident and partially or entirely unfounded claims (not unlike ChatGPT itself) about AI sentience.

Let us try to understand the large language models that undergird generative AI — how they appear to do in-context learning, why their behaviors appear to manifest at scale, and most importantly what their limits (as with any automated system since the Turing machine) are. 

Meantime, it is incumbent upon us as academics, researchers, teachers, and university leaders, to build on our hard-earned bodies of knowledge to encourage a more sane and action-oriented public discourse around the future of AI, lest we lose our footing and fall down a rabbit hole of science fiction-inspired musings. We are humans, and sorting through the nuances and ambiguities of complex systems is core to the human endeavor. We, not computer programs, are the ones with sentience. Let us put our minds to good use. 

 

Data Science & AI

Learn more about Data Science & AI with these editorial features, case studies, resources and more.
See all Resources