This talk confronts the urgent challenge of building a responsible AI ecosystem in the UK. Shannon will explain why in the light of recent AI advances a responsible AI ecosystem is essential for UK innovation, and she will map the public, corporate and engineering ‘ecologies’ that must share new responsibilities in that ecosystem. Finally, Shannon will talk about the obstacles in the way of a healthy, mature and sustainable AI ecosystem in the UK, and how bridging divides across the Responsible AI landscape can help us overcome them.
← All talks 
Lead Turing Fest 2023
Who is Responsible for ‘Responsible AI’? The Ecologies of a Responsible AI Ecosystem
Auto-generated transcript - may contain errors. Tap a timestamp to jump the video.
I wanna talk to you today about responsible AI. I co direct a new program called BRIDGE, bridging responsible AI divides, that's funded by UKRI and that's looking to figure out how we can build a healthy and sustainable responsible AI ecosystem in the UK.
So I'm gonna talk today about why we're talking about an ecosystem in the context of responsible AI, and why we need it. So first of all, what do I mean by responsible AI? Responsible AI is something we've actually been working on for a long time.
It might be something you think is new, you might have heard about AI ethics and ethical issues around tools like ChatGPT only recently, but in fact, we've been working on responsible and ethical AI for years now, because these issues actually aren't really new.
We've had years of experience trying to develop algorithms, machine learning tools, and applications that are trustworthy, fair, just, accountable, transparent, safe, and beneficial. And it's hard work. We've made a tremendous amount of progress on this. But every new tool and every new advancement in artificial intelligence presents new challenges around this.
And you've heard of many of these challenges, of course, with ChatGPT, which have been all over the media. Right? And then there's some challenges we're still struggling with that you may not be as aware of. I mean, let's talk about trustworthy AI. Many of you may know that tools like ChatGPT will fabricate results that match the statistical pattern that underlies the algorithm and its model of our language world.
So there have been court cases where attorneys have submitted materials prepared by CHAT GPT that cite nonexistent precedents, nonexistent legal filings and cases. Judges are getting pretty concerned about this, rightly so. Imagine fabricated scientific research making its way into a scientific research article.
Journals and peer reviewers of scientific research are rightly concerned about this, right? So these tools can be incredibly useful, but we have to understand their limitations and the risks that they pose. We've had lots of examples, for example, of algorithmic bias and machine learning bias in everything from financial and lending algorithms to algorithms in health care.
One notorious algorithm used in US hospitals was found to be diverting needed medical care away from the sickest black patients towards white patients who were less in need. Not because the algorithm has any concept of race or racism, but because the training data that was used to build the algorithm mirrored the decades of medical neglect of black patients in the American health care system.
And the algorithm just learned to reproduce it because that's the data we gave it. We've had examples of algorithms in tools for benefit allocation and fraud detection in the public sector all over the world that have ruined the lives of innocent people. The Dutch government in two thousand twenty one had to resign over the child benefits scandal caused by an algorithm that for years was accusing tens of thousands of innocent Dutch families of fraud.
There were suicides. Children were taken from their homes. Hundreds of those children have not yet been returned. These are not trivial harms. Okay? And now you may hear lots of people worried about AI safety, about whether these tools will get out of our control.
Good news is you don't have to worry, the Terminator is not coming for you. But there are safety concerns around these tools. They're incredibly powerful and hard to predict. So we need responsible AI innovation. Okay. And I think we need to recognize that AI represents a further consolidation of social power in the hands of tech companies, developers, and users.
And we know that when we have social power without responsibility, that tends to degrade public trust and confidence in innovation. And that's a bad thing for a lot of reasons. Innovation without responsibility endangers the social license to operate. That's what we call it when we're comfortable with innovation, operating in our spheres and changing our environment and our institutions where we're willing to go along for the ride.
Right? When we lose the social license to operate the way, for example, that many mining operations in the '70s and '80s did when it became clear how much they were damaging the environmental health and the health of communities where they were operating, there were people sort of blocking the roads to the mines, right, because those companies had lost the social license to operate.
And we see a lot of resistance to AI applications in areas where that social license to operate is fragile. Innovation without responsibility inhibits adoption of AI. People who don't want to take unnecessary risks with their business or with their clients may not use it unless they're confident that these tools have been developed responsibly.
And the ones who will use it will tend to be the more reckless actors in this space, which is bad for everybody. So this breeds a vicious cycle of social harms. Right? Because as more reckless actors use the tools, which have not been shown to be safe or trustworthy or fair, more harm is done, which damages more public trust in the technology, right, and you have a vicious cycle.
And that incentivizes a short term race to the bottom where people want to get out in front and grab as much profit from this tool before people banning it or criticizing it to the point where it can't be used in a profitable way anymore, and so you get a race to the bottom.
And that impedes future public support for innovation. Lots of people are not as excited about what technology holds for them as they should be right now. Not as excited as many of you are probably in this room. Right? We want people to be excited about what technology holds for them in the future.
And the only way we get that excitement and that trust back is through responsible innovation. So there's two models of innovation. Right? We can move fast and break things. We've seen that model. How well do we think it went? Or we can innovate boldly but responsibly.
And we've done this before. Right? We've done this in aviation. We've done this in bioengineering and built incredibly strong, robust industries that innovate but keep people safe. So the cost to society for model one of innovation was pretty high. We're still dealing with the lasting damage to democratic norms, to trust in institutions, to social and civic cohesion in countries all over the world, and to the state of political rationality.
That was the cost we paid. Do we really wanna do that again with AI? I don't think so. So I want you to think about AI in a different way. I don't want you to think about AI as the way we used to talk about data as the new oil.
Right? That's a pretty bad metaphor for all kinds of reasons. No metaphor for AI is perfect. AI is a new kind of thing. In fact, it's not even one thing, it's many as I'll explain. But if we're gonna use a metaphor for it, let's talk about it as if it's something that's fundamental to the way we're rebuilding the world today.
In this sense, AI is more like the new steel, the way steel was in the industrial revolution. Right? Like steel, AI will change the way the world looks. Like steel, AI will allow us to rebuild the world at new scales. Think about how cities radically changed in a century because of the use of steel.
But you don't rebuild the world with materials you don't trust. So the future of AI, if we really think that it can be beneficial for us, which it can, depends upon us being able to trust it. And the social cost of not being able to trust it, therefore, could be catastrophic.
Not just because of the harm that it could do, but of the potential that will be lost. The potential to rebuild the world in more sustainable ways, in safer ways, in ways that promote human well-being and happiness. So responsible AI is the only other path for AI, the only path.
Okay. So how do we take that path? How do we walk it? Where does it take us? Again, this is where we have to start thinking about things in terms of an ecosystem. Why an ecosystem? Again, it's a metaphor. Right? Why this metaphor?
Well, I've mentioned already AI isn't one thing. Experts in AI actually go nuts when the media talks about AI as this sort of this one cohesive thing. There are all different kinds of techniques that have been used for AI. For decades, we had sort of formal logics built into expert systems that we called good old fashioned AI.
Then we had neural networks, the early days of machine learning, and then with the revolution in big data in the last ten years, a complete transformation of the old days of artificial neural networks to new machine learning models that are more powerful than we could have imagined.
And then now with what people are calling generative AI, large language models, large image models, we're seeing new capacities emerge from these technologies. And all of these different tools have different strengths and different weaknesses. So AI itself is a is a sort of ecosystem.
Right? But it's not just the algorithms and the data or the trained models. AI systems depend upon a complex shifting global web of people and organizations and materials. It's a global web of distributed and yet very tightly interwoven dependencies of actors. And that's very similar to what we see in the natural world with ecosystems, right, where everything is tightly coupled to everything else, but with degrees of complexity that are incredibly challenging to understand.
So a responsible AI ecosystem, if we began to map it, might start to look like this. We could talk about the ecologies within the larger ecosystem. There's a material ecology of AI. There's what I'll call a tech ecology, a corporate ecology, and a public ecology.
And by the way, like all maps, this is oversimplified. Right? Many of these spheres intersect in certain ways, overlap. But just to give you a sense of what I'm getting at, let's look at the material ecology of AI. Lots of people actually tend to forget about this.
You think about machine learning and AI as this thing that lives in code and not matter. But a lot of the physical world has to go into AI, from data suppliers, energy suppliers, human labor, human bodies go to work every day in countries like Kenya and the Philippines to label the data that makes AI run, or to mine the rare earth minerals that are required for the graphics chips that we're using to power it.
Many of those minerals are causing environmental damage as they come out of the ground, or political and social damage. Some of them are what we call conflict minerals. Tons of water gets used to develop the chips and train the models that we're relying on.
So there's a huge physical ecology here. And if you're interested in this, I strongly recommend that you visit this website, Anatomy of AI, that was developed in two thousand eighteen by the Microsoft and AI Now researcher, Kate Crawford and Vladan Johler. And they built this beautiful work of art, which is a schematic of, if you will, the material ecology of AI.
And you can see how detailed it is. Right? There's no way you can read that. That's because of how complex this ecosystem is. And there's a lovely essay that goes along with it that I strongly encourage you to read. Okay. But in addition to the material ecology, there's the tech ecology.
All the technologists and experts that go into building these systems, all of the researchers, the model developers, the engineers. And within that, right, even teams that are now devoted to expertise in things like machine learning fairness, trust and safety, responsible AI teams. I've worked on those in industry.
They're incredibly impressive and they're necessary for these tools to work. Professional tech societies and standards organizations like IEEE or ACM, AI research publication venues. Right? This is a partial list. The corporate ecology. Large platform companies and cloud providers we all know. Right? But also the VCs, the startups, the open source orgs, the research organizations, the logistics and supply chain companies, the third party apps and services, the consulting firms, the auditing firms, the corporate boards and lobbyists, and the users.
An immense ecosystem. The public ecology of AI, individual consumers and end users, impacted communities, including those who don't use the stuff. Right? Not using AI doesn't mean it won't determine whether you get a job, whether you get an organ transplant, whether you get housing, whether you can travel the roads safely.
So the public ecology really includes everybody. It includes public institutions like universities, which are grappling now with the impact of ChatGPT on education, NHS, courts, media, civil society, policy and advocacy organizations, academic societies, public research funding agencies like UKRI, local, regional, and national governments all over the world, regulators, intergovernmental bodies like the World Economic Forum and OECD.
We're all struggling to manage the transformations that AI is bringing about. And we're having to do it in coordination. And that's incredibly difficult. So how do you manage that challenge? Well, I think we can learn a lot from the study of ecology, which really grew out of, in the nineteen sixties, early work pointing out the risks of environmental loss and catastrophic damage, irreversible damage from industrial activity.
Right? And that spawned new sciences of ecological interest to try to understand these interlocking systems, these effects, these dynamics, and learn how we can manage them better and more sustainably. So here's some things we learned. We learned that a thriving ecosystem is balanced by the health of its component ecologies.
None can be sacrificed for another. So think about those sub ecologies. Right? You can't simply ignore what you're taking from the material world in order to build AI. Because if you run out of water, if you run out of those raw materials, if you break down the human labor chains, you don't have what you need to build the tech.
You can't ignore the public ecology. Remember what I said about the loss of social trust and the social license to operate? AI doesn't thrive without people willing to use it, willing to trust it. You can't thrive without the corporate structures, the the organizations that are necessary to operate at the scale that we're seeing for AI's potential.
So there's no one part of the AI ecosystem that we can lose or ignore or be unconcerned about the welfare or sustainability of. We need it all. And because of that, it's important to recognize that in ecosystems, symbiotic relationships add stability to this total system.
Right? Think about the ways in ecologies that you'll have one animal that does something for another animal, which does something for it in return. Right? So we need to think where are the symbiotic relationships in the responsible AI ecosystem? Are they operating the way they need to in order for that system to grow and flourish?
Right now, think we we don't. We have a lot of imbalanced relationships, right, where one part of the ecology is getting damaged at the expense of another. So healthy ecosystems also need to be regulated by a number of forces. They need constant adjustment though to dynamic environments and changes.
So they need to be regulated, but they need to be regulated in ways that are responsive and agile and can change according to need. Brittle ecosystems collapse. So how can we make a responsible AI ecosystem that's governed and regulated, but in a way that can move and adjust as needed?
Healthy ecosystems also develop mechanisms of resistance and resilience. So resistance is the ability to repel a shock to the system. Right? Resilience is the ability to adapt and accommodate to that shock. And you need both. An ecosystem which only has resistance is brittle because there will be some shocks it can't resist.
But an ecosystem that only adapts and accommodates and never resists harms is also likely to fail. So we need to think in the responsible AI ecosystem in these terms. What sorts of things are we not willing to adapt ourselves to? What sorts of harms are we not willing to accommodate ourselves to?
What sorts of things do we resist? And what sorts of things should spur us to become resilient, to change, to accommodate? And that's a decision that has to be made by the people who are most affected by these shocks to the system. And right now, it's only the large platform companies and a few relatively powerful, but not in comparison to them, national governments that have the power to decide what we accommodate and what we resist.
And a lot of people see this as a profound problem of political justice today. That those of us who are living with the costs of AI that's unsafe or untrustworthy don't have a say yet, not really, in what we resist and what we accommodate.
Responsibility for the health of ecosystems rests with agents with the power to change them, and particularly the power to damage them, and the knowledge that they can do otherwise. Think about an ecosystem that's being damaged by an animal that's overgrazing and or overpopulating, right, and causing stresses on the ecosystem.
Do we blame the animal for that? Do we say, damn those deer. Why would they do such a terrible thing? Right? No. We don't blame creatures that don't have the power to change their actions or don't have the knowledge that they can do otherwise than what they do.
People are different. But it also means that when we look for responsibility in the AI ecosystem, we have to look for the people who have two things, the knowledge and the power to do something different. And the knowledge and the power in the AI ecosystem is not distributed evenly, and neither are the benefits.
And that's partly why this ecosystem is out of balance. So what lessons can we learn then from the ecological metaphor around AI? First, we're learning that responsible AI policy and practice has to manage the health of all the AI ecosystems' ecologies. We cannot ignore the material dimension, for example.
I just attended a conference at the University of Bonn where they have a new center on sustainable AI. And they had a week long conference on sustainable AI where that's looking at physical sustainability, economic sustainability, political sustainability, and those are the kinds of conversations that we need to be having.
Right? So looking at the entire ecosystem, not ignoring any particular part or ecology. We have to build symbiotic relationships within and across those component ecologies, so that the public ecology of AI is strengthened, not weakened by what the corporate or tech ecologies are doing.
Right now, those kinds of relationships are not there. We need to guide and govern the system with coordinated, agile, and responsive regulation. Right? Whether that's government regulation, whether that's within the professions regulating themselves through standards or licensing or certification. You're seeing a lot of the professional societies around AI, including the leading AI conferences, now require research submissions to those conferences to demonstrate that the researchers have thought about the ethical implications of the work they're doing or the limitations, the ethical and social limitations of what they're doing.
That's really new. That's a form of regulation, right? That's computer scientists themselves getting together and saying, these are the standards we're gonna set for one another and follow. And we need to create new mechanisms of resistance and resilience to the shocks that inevitably come from AI innovation.
And we have to have the right balance, as I mentioned, between those two strategies. And the most important thing, the most important thing of all, we have to distribute the duties of care for this ecosystem to the most powerful actors across these ecologies.
What's the Spider Man motto? Right? Peter's uncle? We all know it. With great power comes great. Exactly. It sounds like a cliche, but, you know, some cliches are there because they're true. And right now, the responsibility is not being aligned with the power.
That has to change. But I think it will. So the challenges and opportunities that lie ahead for responsible AI. What do we need? We need actually some things. We need to build some things. We need to build it with research. We need to build it with the possibilities of the tech itself.
We need to build it in the regulatory environment. We need to build it through public education. Again, all these paths need to be used. So some of the things we need. We need better maps of the AI ecosystem's regional and global ecologies, and their key interdependencies and dynamics.
I showed you that map that Kate Crawford and Vladanjola made in two thousand eighteen. It's beautiful. It's incredible, but it's one of the only maps like that we have. And it's from five years ago. Right? Lots changed. So we're not investing yet in the mapping of this ecology.
Think about the way in environmental sciences. We focus much of our attention on understanding the dynamics within these ecologies, and we need that kind of attention to the responsible AI ecosystem. And part of what that gives us is ecological knowledge. An ecological knowledge that can be put in the hands of the most powerful actors who are acting upon vulnerabilities in the AI ecosystem whether they know it or not.
Right? Remember what I said with power comes responsibility, but to exercise power responsibly, you have to have knowledge. You have to have knowledge of what else you could do be doing that's better. Right? So we need to boost the ecological knowledge of those powerful actors and the way that they're affecting the ecosystem and create knowledge of what alternatives are possible.
But that won't happen unless we do this. And this is the hard part. Let's all admit it. Right? We need to realign the interests and incentives of powerful actors across the AI ecosystem with that ecosystem's health. If all you care about are the next quarterly earnings, if all you care about is what your shareholders will care about in the next four months And again, I'm not blaming corporations for caring about those things.
Those are the incentives we've set up. Right? In some cases, companies would be sued if they cared about anything else, or at least if they put anything else in front of those targets. We need to realign the incentives so that the parts of the ecosystem that are most powerful are incentivized to care for the ecosystem's long term sustainability, growth, and flourishing.
Just like you have to incentivize powerful actors in a physical ecosystem to take care of that environment. And right now, we're behind on this. We also need to bridge artificial divides between sectors and disciplines in the AI ecosystem that are blocking ecological understanding, communication and coordination.
Lots of what needs to happen from an ecosystem to work and grow and be healthy and sustainable is communication. In the physical environment, in the biological world, lot of that communication is nonverbal, right? But in the AI ecosystem, we'll communicate in other ways.
But to do that, we need to cross divides between disciplines, between sectors, between nations and communities, between different groups in society that are seeing different benefits and risks coming to them from these innovations. And these kinds of divides are what our new Braid program, Bridging Responsible AI Divides, that's funded by the UKRI Arts and Humanities Research Council.
It's part of the UKRI's larger fifty four million pound investment in responsible AI, and that we're delivering at the University of Edinburgh in partnership with Institute, also partners at the BBC, and many other people who are joining the effort. So we want to bridge these divides, develop this kind of communication, and foster the responsible AI ecosystem that we need.
Thank you.