Reflections on enacting the principles

At 2024 TrustCon, Thorn’s Vice President of Data Science, Rebecca Portnoff, hosted a panel including some of the generative AI leaders who made commitments this year to generative AI Safety by Design principles to combat child sexual abuse. 

Watch the recording below to hear reflections on how trust and safety leaders at Google and Invoke were able to secure commitments from their companies to put tangible Safety by Design principles in place to combat child sexual abuse. They share the learnings they have from the implementation process and progress they have made against these commitments–including the quick wins and the difficulties.

This video was originally recorded on July 23, 2024

Transcript (note: the transcript is auto-generated)

Rebecca Portnoff (Thorn) (00:00):

Everybody, thanks so much for coming to join our panel here today. We’re going to be discussing Safety by Design for generative ai, an initiative that Thorn and All Tech is Human conducted over the last year. We’re going to be sharing some insights on progress and providing overview of the principles, all that good stuff. So before we jump right into it, I want to give a moment for our panelists to introduce themselves. I’ll start first. My name is Dr. Rebecca Portnoff. I am Vice President of Data Science at Thorn. Thorn is a nonprofit that builds technology to defend children from sexual abuse. I lead the team of machine learning engineers that builds our technology to accelerate victim identification, stop re-victimization, and prevent abuse from occurring. I also lead our efforts more broadly engaging with the MLAI community, which brings us to this panel topic today. [cheering from another room] So, oh, they’re having fun. Cheer louder. No, I’m going to pass it over now to Emily.

Emily Cashman Kirstein (Google) (00:56):

Thanks, Rebecca. I’m Emily Cashman Kirstein. I lead child safety public policy at Google. Probably don’t need to explain what Google is, but I will say we are in the generative AI space with regard to our product Gemini, which folks may or may not have been aware of and playing in all the things. But what we’ll get to I think is we’re kind of on this panel to talk about the broad from building models to the entire lifecycle of generative ai. So thanks for having us.

Rebecca Portnoff (Thorn) (01:29):

Absolutely. Thank you, Emily.

Devon Hopkins (Invoke) (01:31):

Hi everyone. My name is Devon Hopkins and here representing Invoke. We are a generative AI platform for enterprise studios to deploy openly licensed models. We work with these studios to train models on their proprietary content and then deploy that to creative teams through end user application.

Rebecca Portnoff (Thorn) (01:51):

Awesome. Thanks so much, Devon.

Theodora Skeadas (ATIH) (01:53):

Thank you. Hi everyone. I’m Theodora Skeadas, Theo for short, and I’m here representing All Tech Is Human, which is a nonprofit community of over 10,000 people globally, folks who are working in the responsible tech movement. And we are a community of individuals that are engaged in upskilling younger people so they can get involved in this workforce as well as building best practices around a number of responsible tech issues. And we’re thrilled to have been a collaborator on this effort with Thorn.

Rebecca Portnoff (Thorn) (02:21):

Thanks so much Theo and Chelsea, please.

(02:41):

Awesome. Well thank you so much again all for coming to join us for this conversation. I want to kick us off by providing an overview of the issue that all the good folks here today came together to try to tackle collectively, the initiative, and then provide, I was going to say brief summary of the principles, but I do not know how brief I’ll end up being. So I’ll have a context or caveat right now. So starting with an issue overview, as I’m sure many of you in this room are aware, the unfortunate reality is that generative AI is being misused today to further sexual harms against children. We are seeing that bad actors are misusing generative AI to produce AI-generated child sexual abuse material, in some cases bespoke, specific-targeted content of certain children or survivors of child sexual abuse. We’re further seeing bad actors misuse this technology to broaden their pool of targets.

(03:33):

For example, using generative AI to sexualize benign images of children that they can then use to sexually extort them. We’re seeing this type of abuse manifest across all the different data modalities that exist: Image and text video is nascent, but it is occurring. The good news coming out from all of this is that we do still see a window of opportunity when it comes to taking action on this, that the prevalence of this type of material is small but consistently growing. And so that brings us to this initiative today. The goal of this collaboration, which Thorn and All Tech is Human led over the last year, was to bring together key stakeholders in the AI tech ecosystem, AI developers, providers, data hosting platforms, search engines and social platforms to collaboratively align, define, and then commit to a set of Safety by Design principles that cover the lifecycle of machine learning/AI from develop to deploy to maintain.

(04:29):

So now I’m going to pull up my phone because while I have lots of this memorized, not all of the exact words. So lemme provide some overview over the principles. Like I said, they were built across develop, deploy, and maintain. We have three sub principles for each of these sections, and I’m just going to jump right into it. For develop: develop, build, and train generative AI models that proactively address child safety risks. So breaking this down, the first sub principle is to responsibly source and safeguard training data sets from child sexual abuse material (CSAM) and a child sexual exploitation material (CSEM). Now models are a function of the training data and the training strategy that was used to build them. So the opportunity that we’re focusing on with this particular principle is the data–detecting, removing and reporting CSAM and CSEM from your training data, addressing the risk of compositional generalization where a model might combine concepts of adult sexual content with benign depictions of children to then produce abuse material and making sure that your data collection pipeline is avoiding training data sourced from sites that are known to host this type of abuse material.

(05:36):

The second sub principle is to incorporate feedback loops and iterative stress testing strategies in the development process. Now this is already best practice for AI development in general and it certainly applies to this particular issue space as well. Building out robust red teaming processes into development has to be a part of your model training strategy. Third, employ content provenance with adversarial misuse in mind. I think we are all aware here that content provenance is not a silver bullet, especially in the child safety space, but it is an important and can be an impactful pillar of responsible generative AI development when it’s done with adversarial misuse in mind. Because those of us in the trust and safety space know that these strategies that aren’t robust to that kind of adversarial misuse are not going to hold up in the short or the long term. Moving now to Deploy: release and distribute generative AI models after they have been trained and evaluated for child safety, providing protections throughout the process.

(06:32):

So the first sub principle here is to safeguard generative AI products and services from abusive content and conduct. Now I think that this principle really benefits from existing trust and safety strategies for protecting platforms, surfacing prevention messaging, setting up user reporting pathways, detecting at the inputs and outputs of generative AI systems for attempts to produce abuse material or abuse material that was successfully produced. These are all strategies that predate generative AI and have already been impactful in other spaces we’ve seen the proof is in the pudding. The second sub principle is to responsibly host models. This applies to both first party model providers, so those folks that only host models that are built in-house and third party model providers, those who are hosting models built by developers outside of their organization. And in either case it really comes down to this concept of elevating content moderation from the image, the video, and the text to the model itself.

(07:27):

So setting up the processes, systems and policies that are necessary to ensure that that content moderation happens at the model level so that you’re only hosting models that have first been evaluated and you’re taking down already hosted models that you have discovered can produce AIG-CSAM. The last sub bullet is to encourage developer ownership in safety by design. Now this is intended to highlight that there is an opportunity to create a pause point for developers, asking them to consider and to document what child safety measures they’ve taken before release. Now we think that model cards provide a really natural pathway for this pause point both from a closed-source and open-source development perspective. All right, I promise I’m almost done. We’ve moved our way now to Maintain, this is the last one, maintain model and platform safety by continuing to actively understand and respond to child safety risks.

(08:17):

The first sub principle here is to prevent services from scaling access to harmful tools. The unfortunate reality is that bad actors have built models specifically to produce AI generated child sexual abuse material. And as I said earlier, in some cases they are targeting specific children to produce AIG-CSAM that depicts their likeness. They’ve built services that are being used today to notify content of children and create more AIG-CSAM. We can and we should limit access to these harmful tools and we can do that by removing them from platforms and search results. The second sub principle is to invest in research and future technology solutions. Now, principle was written in mind knowing that in the child safety space, the threat is ever evolving. You can’t just sit on your laurels, bad actors are going to adopt new technologies. They’re going to pivot in response to your efforts.

(09:07):

So effectively combating this kind of misuse is going to require continued research and evolution to stay up to date with new threat vectors. Which brings us to the last one. Fight CSAM, AIG-CSAM, and CSEM on your platforms. Now while these specific principles were written with generative AI in mind, the reality is that this kind of misuse, it doesn’t occur in isolation, it occurs as part of a larger interconnected ecosystem with related harms. And so having a holistic approach is going to be necessary to have impact. Alright, I know that we have all been waiting eagerly to hear from our panelists here. So thank you for sitting through that mini lecture. I want to move now to talking about what it looked like to enact these principles that we’re describing. So we have nine principles, three for each of these sections, and I’m going to ask each of the representatives from Google and Invoke and OpenAI to share with us the progress that they have made to date taking action on one of those principles they get to pick since signing onto the commitments. So let’s start with Emily from Google. Emily, can you share with us progress that you made on one of those principles?

Emily Cashman Kirstein (Google) (10:11):

Definitely. And thank you Rebecca. That’s a wonderful summary. So I want to dig into the Deploy pillar, especially about safeguarding gen AI products, services from use of content and conduct. So from the Google perspective, as you can imagine, there’s a number of different ways we’re doing this, but to kind of put this within the three minute answer here. So first we have robust child sexual abuse and exploitation policies that cover everything from CSAM itself to the broader sexualization of minors to abuse instructions and manuals to content that supports sextortion and grooming of minors. Just to make a few examples. And that’s all under one policy. And here is where we clearly lay out what is policy violative and what warrant’s action and enforcement. So that’s one pillar of it. From there, of course we have the required product protections that include both protections at input, at output and all potential modalities in one place.

(11:17):

Again for all teams to see as they are developing gen AI products and testing them. And then from there, of course all of these protections are policies and protections are designed to prevent any imagery from being generated. But in the cases that happens, of course there’s also having reporting mechanisms so that a user can report to us. We can take immediate action for protection. So we’ve had these from the beginning to be very clear, it was very much a safety by design journey for Google in taking on generative AI and building from the ground up to ensure that we were responsible from the beginning. But this kind of gets into, and forgive me, I sort of ended up doing two pillars here.

Rebecca Portnoff (Thorn) (12:03):

Go for it. Go for it. Please

Emily Cashman Kirstein (Google) (12:04):

To maintain because these are fluid. This is not, like you said, this is all a continuum. And so what we have done is made sure that we have these policies, they’re continually evolving, that we’re keeping up to speed on both again, the policies and the protections. Because as we all know, and I’ve heard this in all the panels that I have been attending this week, obviously, especially when it comes to child safety, not only is generative AI, the technology shifting quickly, but the crime shifts quickly. We know this, we’ve known this, we were several years ago talking about just images and hashing and matching, went to video. We’re talking about things like CSAI match those sorts of things. And here we are in generative AI and making sure we’re on top of things that way so that we have made progress on those and making sure that we’re continually evolving through that. So I’ll stop talking.

Rebecca Portnoff (Thorn) (12:57):

No, thank you Emily. I can definitely hear the full systems approach that the Google team is taking to tackle this, and I think that reflects the reality of what we were going for with the principles that it’s not enough to do one particular intervention. There have to be layered interventions to be effective. So Devon, I first want to just give a shout out to Invoke because for those of you who don’t know, Invoke just joined to the commitments a couple of weeks ago. So thank you so much Devon for coming and representing. I’d love to hear from you as well.

Devon Hopkins (Invoke) (13:27):

Thank you for doing two pillars. I did that as well.

(13:31):

So again, we’ve been working with Thorn not very long, but in a short period of time we’ve made a lot of progress thanks to the great recommendations they put forward with their research. So we are a deployment solution for open licensed models. So I’ll speak mostly about the Deploy principle. So at the platform level, we’ve worked with Thorn to develop an understanding of the models that are out there in the open source ecosystem that are being used to generate this type of content. And we’ve created a hashing system so that if a user tries to upload that model into either their locally installed version of Invoke or to our hosted product, they will get a notification that says this is not allowed. It actually prompts ’em to seek help. We know that giving those notifications can direct people to self-help channels. So the Redirection self-help program is the one that we are pointing people towards.

(14:34):

We were also outputting a standardized metadata into all of the output images generated in Invoke. This is helping NCMEC look through images out there and do things like identify if there’s an increased rise in an unknown model that might be new, might be newly emerging in the community that creates CSAM. Flag that, and then we can put that back into the block list for all these platforms. I think what we found as we were putting these in place is that to have the impact that we wanted to have, we couldn’t just do this in our platform, but we also needed to be influencing and leading the open source community in doing this as well. So we started to do that. We’ve just launched the open model initiative, which is a coalition of a lot of the major players in the open source AI generation space. We’re getting support from the Linux Foundation to get that all stood up and we’re starting to have conversations about how can we really influence this at the core level, the foundation model level, where previously that may have been a failure. So we’re looking at the viability of removing children altogether from the core dataset that we use to train models because we know that one of the ways of at the Develop level, if we separate sexual content from child content, that makes it very, very difficult then to combine those concepts into images.

(16:05):

So we’re starting to get that alignment with those people. It’s been a good reception and I can speak a little bit more to that later.

Rebecca Portnoff (Thorn) (16:13):

Thank you so much, Devon. It’s great to hear both the efforts that you’re doing internally at Invoke and also what it looks like to be a leader in this space in this moment. So thank you so much for sharing. So we got a chance to hear all the good news about the progress that’s being made. I also want to make sure we have a chance to hear about some of the challenges that folks have encountered while they have been taking action on these principles. So starting again with Emily, are there any challenges that you can share with us or learnings?

Emily Cashman Kirstein (Google) (16:37):

Yeah, so I think about what’s working well, right? Taking that is I think when we were these safety requirements we had in place when we were going to sign on to these principles, going through the approval processes, the vetting for it, it really helped bring more attention internally both to the work of the product teams who are doing this work day in day out, developing these policies and forcing against these policies alight to their work, but also to the broader leadership and making sure that there’s attention paid to the importance of Google signing onto these principles from a broader ecosystem perspective and how important that was to the company that went really well and making sure that those teams who are working on this day to day and are often doing some of the hardest work at Google are getting again, broader recognition. So that was really important.

(17:35):

Challenges sort of goes back to that Maintain and knowing that the space is rapidly evolving. Gen AI is moving at the lightning speed and making sure that we do have processes in place to work in that evolution and know that there’s, here are the backstops, this is where we’re going to make sure that we’re continuing these protections moving forward, that the protections we have today are effective and the right ones for tomorrow. And so making sure that we do that, having processes in place and making sure that we are able to deliver on those commitments, both just as a company that we want to do and now we have this external facing commitment that we have to keep in mind and be accountable for. So I think having that in mind and going through the Maintain principle and things like that, I think the beauty of this is that it was not specific, right? It wasn’t there’s an opportunity in these principles to allow for innovation. We knew that this was going to be an interative process and needed to build in for that. And so I was really grateful for the foresight that Thorn and All Tech is Human had in making sure that we had that built in these processes.

Rebecca Portnoff (Thorn) (18:57):

Thanks Emily. And I think keeping pace is the main thing I’m hearing and certainly can appreciate the ongoing challenge of that. Definitely. Devon, any challenges you can share with us?

Devon Hopkins (Invoke) (19:06):

Yeah, I think going back to just the open source community being this coalition of different businesses with different goals, lots of different stakeholders, trying to get alignment around the direction that the community goes as a whole is, I don’t think it’s a new challenge to open source. I think that’s been around for decades, but specifically weighing trade offs that we might make that we know can mitigate the risks and reduce harm, but may also prevent honest and positive use cases. Other people out there that may want to create a children’s book that has depictions of children in it, but if we remove those from the dataset, we can’t do that anymore. Weighing the trade-offs of that, is that a valid, valid trade off to make it and making the decision when you don’t have a single person that’s making that decision? I think that’s been one challenge.

(20:01):

At just technical product level, I think we have implemented metadata tracking, but we know the limitations of metadata tracking and that that’s fairly trivial to strip out of an image. We’d love for there to be a really strong watermarking solution that does not reduce the quality of these images. We are an enterprise business selling to enterprise businesses, so we can’t have a solution that reduces the quality that’s just not viable and there just aren’t any that I’ve seen. If anyone here knows they come talk to me after, I’d love to know. So that’s something we’re looking for. We’re looking for someone in the community to take that and so we can implement that.

Rebecca Portnoff (Thorn) (20:44):

Thank you Devon, and come talk to me too. I’d be very curious to hear what you found, but definitely hearing themes of decision making where it comes to making decisions as a collective, as a collaborative, that there’s strength and value in that, but also challenges that come from it, and then the realities of technology development and not letting perfect be the enemy of good, but still wanting to make sure that what you’ve built is in fact good. So now that we’ve had a chance to hear about some of the progress and challenges from the companies who have been enacting these principles, I want to go back in time a little bit and talk through some of the experiences that we had building up alignment in the first place and joining the initiative and joining into the commitments. So Devon, I wanted to ask a question for Invoke from the perspective of open ai, deployer and developer for us with this initiative for Thorn, All Tech is Human. It was very important to make sure that we had representation across the open and closed source spectrum. We believe both that there are opportunities to prioritize child safety regardless of how you develop and deploy your models, and there’s also responsibility to do so. So with that significance, I guess I’ll ask, when it comes to the open source ethos, can you share with us your insights on where that ethos of open development made it easier to build alignment on actioning on these commitments where it might’ve made it more challenging? We’d love to hear your thoughts.

Devon Hopkins (Invoke) (22:00):

Sure. Yeah. It’s the open development ethos and values are openness, transparency, community-driven approaches, and that there’s a long precedence in open source software coming together around common challenges, aligning on standards, ensuring interoperability. That’s how email came about in the eighties. It was an open research paper from researchers at USC that called for feedback on these protocols that would get these disparate systems to talk to each other. So there’s just a historical precedent for this community coming together on these topics. I think we found that that culture really helped when we put together this open model initiative. We had a lot of these people just in various discord channels and we pulled them all together in one, in a steering committee and we said, Hey, here’s what we’re going to do. Thorn has given us this great guidance on these principles that we can put into place.

(23:07):

And the response was, yes, absolutely. We’re in, let’s do it. And I think, I don’t know if we were in a closed environment, if those businesses would’ve been that open to working with perceived competitors. I worked for one of the largest e-commerce marketplaces selling art, and when we saw fraud schemes coming up, we didn’t go call our competitors and tell ’em that we were seeing this. We kept our strategies close to the chest. I think a lot of businesses see their trust and safety as competitive advantage first, and it was just helpful to have these folks kind of see the greater good first and put aside the competitive piece to it, at least for the time being. I’d say on the flip side, the challenge is just the same other side of that coin. There’s lots of stakeholders, different business models, different personal goals. So it’s getting alignment on the best path forward as a community, but I think we’re making progress.

Rebecca Portnoff (Thorn) (24:15):

Thanks Devon. I’m glad to hear it and I can certainly appreciate that. That double-edged sword. I think that often in this space you find that things can cut both ways and trying to optimize for the one while reducing the other is the name of the game. So Emily, a question for you. Google is a behemoth, right? That’s an obvious statement, but it’s fact. Yes, it is. It’s also one of the few companies in this initiative that had a product or service that fit across every single part of the tech ecosystem that we were looking at. AI developer, let’s see if I can remember the list. AI provider, data hosting platform, social media platform and search engine. So my question for you is how did you and your team go about navigating this reality for building alignment, for committing to the principles where like Chelsea, maybe you ended up having to talk to the entire organization. Was your strategy to take it department by department? Did you just tackle it head on and then sleep really late for several months? What did that look like for you and your the team? No sleep allowed.

Emily Cashman Kirstein (Google) (25:16):

No. Again, no surprise, no trade secret. Google is large. There were a lot of people we had to talk to. For a company that with all of those things that you were talking about and every part of the company that either was building generative AI products or utilizing generative AI within existing products, we have to talk to everyone. And I’ve been at the company for almost three years. I thought I had a pretty good handle on who I needed to get in front of to make sure that we could make this process as smooth as possible, but I was wrong. I learned very quickly, I think similar to Chelsea, that there were work streams that I hadn’t thought of that needed to be, again, the downstream effects of all of this and making sure, one of the things that is quite Googly is making sure that everyone has visibility.

(26:10):

There’s no surprises and making sure we were doing that. So they were, as Rebecca I think can attest to, there were a core group of us that were the ambassadors, if you will, of this initiative. And we went from product area to product area to get consensus to make sure that folks were, again, had visibility to what was going on. And it helped that as a cross-functional, a cross-functional child safety team that includes myself. I don’t work on a product, I work for government affairs public policy, but there are folks from comms, legal, all of the product areas, and we all talk quite often and using that as kind of a core, making sure that we’re getting this out to the people that need to do that. So that was a big part of it. And then once we did have consensus from the different product areas, it was going into different leadership forums, which we were quite transparent with Rebecca on.

(27:12):

This might take a little bit of time. These are built in processes. So those ranged from things that were specific to child safety leadership forms specific to kids and families more broadly or generative AI more broadly. And one of the ways that we worked into those, I think successfully was Google already had generative AI broader principles, and we were able to layer in this is how these form principles layer into this. This is why it’s so important. And to clarify, it wasn’t convincing people why this work was important. It was, again, at such a large company where things are moving so fast, it was making sure we had things in front of the right people. And that did take a while. But as a foreign alum myself, I was over the moon to finally be able to call you, tell you we were in. And I think it felt like a real success in being able to get all of those groups together and just huge kudos to foreign and All Tech is Human for being really understanding partners in that because it did take some time. And both organizations got that. They understood, they get the technology, they understand where industry is coming from, but they’re also holding us accountable and making sure that we’re being held to a higher standard. So it was really an important process that we were really happy to be a part of.

Rebecca Portnoff (Thorn) (28:42):

Well, thank you Emily. It was definitely a good moment to have that call when you shared the good news. So you’ve really teed us up nicely for the next question. I’d love to bring Theo into this conversation around that concept, that reality of accountability. Now, I’ll first share a little bit that for me, it was a really easy decision to ask All Tech is Human to lead this initiative with us together. And it’s in part because of the way that All Tech is Human understands the pace of technology development. And I’ll just share here a quote from their website, tech innovation moves fast while our ability to consider its impact often moves slow. We need to reduce the gulf between these. Now we consider accountability to be a really important way in which we can go about reducing the gulf between these two things. So Theo, I’d love to hear your thoughts and your experience around ways in which companies can be held accountable, impactful ways that companies can be held accountable for taking action on these types of principles and these specific principles. If you have examples, guidance that you can share with us, we’d love to hear it.

Theodora Skeadas (ATIH) (29:43):

Absolutely. Thank you so much. So accountability can come from both within and there number of ways in which accountability can be generated by companies themselves, but also external pressures can impose accountability on companies directly and indirectly. So when looking at accountability that comes from within information sharing and transparency initiatives can be very important in this space. So for example, in terms of transparency initiatives, regular transparency disclosures are very helpful. Those can be issued on a quarterly or biannual or every six months or annual basis. Previously, I was working at Twitter and I supported our biannual transparency reports, which documented amongst a number of other things, state backed, state driven take down requests as well as enforcement issues and also platform manipulation issues. And those were ways to evidence transparency publicly. And a lot of the companies here have their own transparency reports. There is also information sharing.

(30:48):

So NCMEC and also CT, the global Internet forum for counter terrorism have hash sharing databases which enable information sharing across partner companies on really critical issues like child safety and counterterrorism respectively. But there are also ways to encourage accountability from about, so this can include for example, legislation and regulation that comes from legislative and regulatory bodies. For example, there’s a lot of bill activity that is ongoing right now in statehouses and also at the national levels in the US and also throughout the world on the issue of child safety because it’s so important. And this can also facilitate industry best practices and additional way there can be sharing of information with regulators. And this can happen in part through whistleblowing activity. But there are challenges to all of these mechanisms. So as both Chelsea and Emily mentioned, when it comes to working groups and bringing staff together to build information sharing advancements, often staff are burdened with a lot of responsibility.

(31:56):

It can be very hard to find the time and also to build the buy-in from an employer because there are a lot of different teams and different, and sometimes complimentary but not always work. And it can be hard to prioritize one effort from amongst many. And so building buy-in from the top so that the working group participation can yield effective and operable recommendations is something that can be challenging. Additionally, these initiatives are voluntary, which means that a transparency to quick disclosure if it’s a voluntarily provided one can be reci, as was the case of course with Twitter’s many transparency initiatives like the Twitter moderation research, which share state back information operations related information with independent researchers, including at the Stanford Internet of Observatory, tourists with fake units in Latin America and in Australia, but is no longer operable. So the voluntary nature of these things can make them a little bit less accountable in terms of legislative and regulatory endeavors.

(33:02):

Capacity is always an issue for regulators in the US and also abroad. And as we all know, our fraught political ecosystem makes policy making and passing bills very difficult even under the best of terms. And these are probably not the best of terms. And then as many of us know, whistleblower activities are challenged by a lack of safety mechanisms that enforce whistleblowers and can make sharing information with different regulatory bodies difficult. And so there are efforts from within and also that can yield better accountability. But all of these are challenged through a number of different issues.

Rebecca Portnoff (Thorn) (33:45):

Thank you so much for unpacking that, Theo. The different variety of options of the challenges that come from, I really like that framing of internal and external accountability. I think that the reality is that especially for these types of voluntary commitments, there will be companies that choose to not come to the table. There will be companies that come to the table but don’t move fast enough. There is a variety here of ways in which folks can end up responding to voluntary commitments. And so knowing there’s a variety of ways to help maintain accountability is really crucial for the success and the impact that we’re looking to have. So I’m going to pivot us to a topic that I’ve already heard several of you bring up. And I love it because it’s near dear to my heart as well, Maintain. Maintaining the quality of mitigations, maintaining them both in the face of an adversarial landscape, but also against your own tech stack that’s evolving and iterating. So starting again with Emily, if that’s all right, I’d love to hear a specific example. If you can share one on what kind of safeguards that your organization has put in place to ensure that your development, internal development doesn’t outpace these safety by design principles.

Emily Cashman Kirstein (Google) (34:48):

Definitely, and it’s a challenge and one that we are actively, we have developed processes before and are going to continue developing and redeveloping those processes. And one of the things that we’re doing as an example of this is we have one dedicated landing page with all of the product requirements as it relates to child sexual abuse and exploitation and generative AI that is regularly updated by our trust and safety policy team that goes through and makes sure all the teams have access to it as they are developing and testing products. And they keep that updated, which is really important to have that central piece there so that doesn’t, things don’t get lost. And then on top of it, those requirements that are listed there, they’re baked into approval processes, they’re baked into cross-functional check boxes and all of those things to make sure that this is really in the process.

(35:48):

And we take every opportunity we can, while this is already in the approval processes to continue to educate product teams from the beginning on these things. So every chance we get to get in front of a variety, again, no secret that Google’s big, getting in front of as many people as we can to reinforce not just these commitments, but the resources that we have internally to make sure that we can live up to these commitments. And conversely, and I think it’s really important is to know and be open to when product teams come back and say, this might not be working the way you think it is, how can we get to the same goal, the same outcome in a different way now that the tech stack is changing, now that there’s different kinds of protections, the threat has changed, things like that. And that’s really how protection evolution should go. We do need to continue to have those conversations and make sure that we are always thinking through the evolution of all of this and we remain committed to that.

Rebecca Portnoff (Thorn) (36:55):

Thank you, Emily. Yeah, strategy of being embedded, having that breadth across different departments while also still being responsive. Absolutely. Devon, any thoughts you can share with us about? Sure. Yeah.

Devon Hopkins (Invoke) (37:05):

So we are definitely a smaller team than OpenAI or Google. So for us it was a lot about putting together the right partnerships and that coalition that can support us in these goals. There’s a lot of great technology partners that we have that are tackling specific parts of this problem. So we’re working with a startup called Vera, which is really just focused on prompt monitoring and can they create tools that can help integrate with platforms to help ’em monitor prompts and flag back at bad actors. That’s great because they can work on that. We can put it into our system and we all benefit from it. I’d say that the key piece here though is making sure that these efforts are prioritized in the product roadmap. So having this at the business strategy level is very important. You can go into meetings and you can get a lot of head nods, but until it actually gets written down into whatever your strategic framework is, OKRs or whatever you’re not, that’s when you have a prioritization meeting and it doesn’t make it into the next sprint. So I think really making sure that we have quantitative measures around this and that we’re driving towards that was also a really important safeguard.

Rebecca Portnoff (Thorn) (38:18):

Yeah, I love the practicality of that. Make sure it’s on the spreadsheet that has your roadmap. Yeah, I love it. So I know that we all here are talking about a collaboration, right? That was what this whole last year and a half–I don’t know what is time–that was, what we were doing was collaborating together. And I think folks in trust and safety know how fundamental and core collaboration is to this field that when it comes to even with closed source competitors, that collaboration really is key to how trust and safety professionals are able to have impact. And so Theo, knowing that in a previous role you did a lot of this type of leadership role in collaborating and leading these collaborations that you manage the day-to-day operations for Twitter’s trusted and safety council, we’d love to hear your guidance on what is the secret sauce to making those types of cross industry, cross technology platform collaborations as impactful as possible.

Theodora Skeadas (ATIH) (39:09):

Yeah, absolutely. I’ll talk about the Twitter Trust and Safety Council and also a few other examples that come to mind. So the Trust and Safety Council, which was started around 2016 and ended at the end of 2022, was made up of a few permanent advisory groups and also some temporary advisory groups. So we included groups around child sexual exploitation, dehumanization, content governance, online safety and harassment, mental health and suicide and digital human rights. And we tried to be as transparent and accessible as we could as staff. So for example, we tried to regularly disclose with the members of the Trust and Safety Council, there were about 80 nonprofits that were members across the different advisory groups that how their feedback was being represented and ultimately implemented in product and policy developments. And then a few other examples that come to mind that help make best practices.

(40:05):

So we are all here together under the auspices of the Trust and Safety Professional Association, TSPA, which brings together different stakeholders so that we can have these incredible conversations. There’s the Integrity Institute, which is a group of about 400 individuals who all have trust and safety experience that works with governments and intergovernmental organizations. So cross-sectorally to elevate learnings so that they can do the work more effectively. Also, very briefly, I see we’re at time highlight two quick other examples. One is the National Democratic Institute, which I was working with earlier this year. We can convened a group of civil society organizations in member Kenya to talk about the issue of online gender based violence. And one of the main takeaways there was that enforcement, as we mentioned earlier, is a huge issue in this particular context. It’s the case that there’s a lot of legislation and regulation on tech facilitated gender based violence, but not internal capacity to enforce it, and therefore it doesn’t actually get enforced. So building capacity to make sure that enforcement happens is huge. And then as was mentioned yesterday, Barbara, I’m her chief of staff at Human Intelligence. We’re organizing aggregate bias county challenges, and we’re working with all the different stakeholders to think about how to scope challenges, where we are reflecting on how prompts can yield all kinds of biases and actuality issues and misdirection. And so really being open with partners about specific objectives and also the number, the detailed steps of how to work together, we have found to be very helpful in strengthening relationships across sectors.

Rebecca Portnoff (Thorn) (41:44):

Thank you so much, Theo. I appreciate the specificity there. And I know that we are up against time, so I just want to give every panelist thank you for their time. I was going to let people have closing thoughts. No one is rushing out the door. So I guess I’ll go ahead and do that if that’s all right. But yeah, Emily, any last closing thoughts?

Emily Cashman Kirstein (Google) (42:02):

Well, first of all, I won’t be offended, so go for it. But just to say Google was really proud to sign on to these principles at the launch. I was so happy to be on the webinar with you and Rebecca there. One of the thoughts I had on that webinar was that how important it’s to remember that we weren’t starting from scratch on this, that we were using the years of knowledge from having hash matching, having classifiers, having a more professionalized, trusted safety base and baseline and understanding put us in a really good position to be able to really live out safety by design that these principles, but you rightly pointed out to me and that webinar that because we’re starting from scratch doesn’t mean it was easy and it wasn’t. And for all of the reasons that we needed to get through extensive approval processes, but also because Thorn and Altech as human brought together across industry folks for in-depth, really tough conversations, granular conversations and working groups on how to do this, what’s happening in this space. And that was hard work and upleveling these to the principles. So certainly I’m grateful for the Google folks who were part of those working groups, but also again, to form and All Tech is Human for bringing industry together on this in a tiny way as an urgent moment, having the understanding of what the perspective is of tech, but again, also holding all of us to account and elevating the space when it comes to all the children that we’re all here to serve.

Rebecca Portnoff (Thorn) (43:38):

Thank you, Emily. Devon, any last thoughts?

Devon Hopkins (Invoke) (43:40):

Say data. I just think you said that the shared values among the trust and safety community and the open source community around collaboration, transparency, community driven approaches, let’s keep that energy going. Let’s keep that culture going. So I think we’re going to actually seek impact when we’re sharing best practices, when we’re sharing what’s working, when we’re creating these shared resources on the education side and the tech infrastructure side, I think we have this.

Rebecca Portnoff (Thorn) (44:13):

Thank you, Devon. Theo.

Theodora Skeadas (ATIH) (44:15):

Yeah. So definitely to the earlier, a lot of the stakeholders that come together may see themselves as competitors, which means that we don’t have to work hard to create a holding space, a safe space where organizations feel that they can come together and be honest and candid and share information that might make them a little vulnerable. They’re sharing issues that they’re dealing with directly. So that’s a huge part of the work as well.

Rebecca Portnoff (Thorn) (44:35):

Thank you, Theo. I’ll just close this here To say this was a lot of work to do. It was, it was a lot of work, a lot of late nights, a lot of effort. It’s the kind of work that all of us here are here to do. And so this is me saying, yes, it’s a lot of work, but I hope that you see reflected in this conversation, the own work that you’re doing in your own organizations and your institutions to make progress, to drive forward impact. And I want to say thank you to all of you for doing that. And that’s it.