Timnit_WAPO_20.html

<figure class="timnit-wapo-f">
    <img src="/LiterOCULAR/img/wapo1.jpg" class="img-fluid cover-img cover-timnit-wapo" alt="image of Timnit Gebru" title="Timnit Gebru speaking at a conference">
    <figcaption><span id="#" class="mention-group" about="GoogleAILab">Google AI</span> research scientist <span id="#" class="mention-person" about="TimnitGebru">Timnit Gebru</span> speaks on Sept. 7, 2018, at <span id="#" class="mention-event" about="TechCrunchDisruptSF">TechCrunch Disrupt SF 2018</span> at the Moscone Center in <span id="#" class="mention-location" about="SanFrancisco">San Francisco</span>. (<span id="#" class="mention-person" about="KimberlyWhite">Kimberly White</span>/<span id="#" class="mention-company" about="GettyImages">Getty Images</span>/<span id="#" class="mention-company" about="TechCrunch">TechCrunch</span>)</figcaption>
    </figure>

<h1 class="timnit-wapo-t"><span id="#" class="mention-company" about="Google">Google</span> hired <span id="#" class="mention-person" about="TimnitGebru">Timnit Gebru</span> to be an outspoken critic of <span id="#" class="mention-subject" about="AI">unethical AI</span>. Then she was <span class="timnit-wapo-t-2"><span id="#" class="mention-subject" about="Firing">fired</span> for it.</span></h1>
<p class="subheader timnit-wapo-s"><span id="#" class="mention-person" about="TimnitGebru">Gebru</span> is one of the most high-profile <span id="#" class="mention-subject" about="Identity">Black</span> <span id="#" class="mention-subject" about="Gender">women</span> in her field and a powerful voice in the <span id="#" class="mention-subject" about="AI">new field of ethical AI</span>, which seeks to identify issues around <span id="#" class="mention-subject" about="Bias">bias, fairness, and responsibility</span>.</p>
<p class="byline timnit-wapo-b"><span id="#" class="mention-person" about="NitashaTiku">Nitasha Tiku</span> | <span class="date-emoji">December 23, 2020</span> | <span id="#" class="mention-organization pub-emoji" about="TheWashingtonPost">The Washington Post</span></p>



<p class="dropcap-wapo"><span class="deco-drop">Two</span> months ago, <span id="#" class="mention-company" about="Google">Google</span> promoted <span id="#" class="mention-person" about="TimnitGebru">Timnit Gebru</span>, co-lead of <span id="#" class="mention-group" about="GoogleEthicalAI">a group focused on ethical artificial intelligence</span>, after she earned a high score on her annual employee appraisal. <span id="#" class="mention-person" about="TimnitGebru">Gebru</span> is one of the most high-profile <span id="#" class="mention-subject" about="Identity">Black</span> <span id="#" class="mention-subject" about="Gender">women</span> in her field and a powerful voice in the new field of <span id="#" class="mention-subject" about="AI">ethical AI</span>, which seeks to identify <span id="#" class="mention-subject" about="Bias">issues around bias, fairness, and responsibility</span>.</p>

<p>In his peer review of <span id="#" class="mention-person" about="TimnitGebru">Gebru</span>, <span id="#" class="mention-person" about="JeffDean">Jeff Dean</span>, the head of <span id="#" class="mention-group" about="GoogleAILab">Google Artificial Intelligence</span>, left only one comment when asked what she could do to have a greater impact, according to documents viewed by <span id="#" class="mention-organization" about="TheWashingtonPost">The Washington Post</span>: Ensure that <span id="#" class="mention-group" about="GoogleEthicalAI">her team</span> helps make a promising <span id="#" class="mention-technology" about="NLP">new software tool for processing human language</span> “consistent with <span id="#" class="mention-subject" about="GoogleAIPrinciples">our AI Principles</span>.”</p>

<p>In an email thanking <span id="#" class="mention-person" about="JeffDean">Dean</span> for his review, <span id="#" class="mention-person" about="TimnitGebru">Gebru</span> let him know that <span id="#" class="mention-group" about="GoogleEthicalAI">her team</span> was already working on <span id="#" class="mention-subject" about="GebruResearchPaper">a paper about <span id="#" class="mention-subject" about="Bias">the ethical risks around <span id="#" class="mention-technology" about="NLP">the same language models</span></span></span>, which are essential to understanding the complexity of language in <span id="#" class="mention-technology" about="SearchEngines">search queries</span>. On Oct. 20, <span id="#" class="mention-person" about="JeffDean">Dean</span> wrote that he wanted to see a draft, adding, “definitely not my area of expertise, but would definitely learn from reading it.”</p>

<p>Six weeks later, <span id="#" class="mention-company" about="Google">Google</span> <span id="#" class="mention-subject" about="Firing">fired</span> <span id="#" class="mention-person" about="TimnitGebru">Gebru</span> while she was on vacation.</p>

<p>“I can’t imagine anybody else who would be safer than me,” <span id="#" class="mention-person" about="TimnitGebru">Gebru</span>, 37, said. “I was super visible. I’m well known in <span id="#" class="mention-group" about="AICommunity">the research community</span>, but also the regulatory space. I have a lot of grass-roots support — and this is what happened.”</p>

<p>In an internal memo that he later posted online explaining <span id="#" class="mention-person" about="TimnitGebru">Gebru</span>’s <span id="#" class="mention-subject" about="Firing">departure</span>, <span id="#" class="mention-person" about="JeffDean">Dean</span> told employees that <span id="#" class="mention-subject" about="GebruResearchPaper">the paper</span> “didn’t meet our bar for publication” and “ignored too much relevant research” on <span id="#" class="mention-subject" about="Bias">recent positive improvements to the technology</span>. <span id="#" class="mention-person" about="TimnitGebru">Gebru</span>’s superiors had insisted that she and the other <span id="#" class="mention-company" about="Google">Google</span> co-authors either retract <span id="#" class="mention-subject" about="GebruResearchPaper">the paper</span> or remove their names. Employees in <span id="#" class="mention-group" about="GoogleResearch">Google Research</span>, the department that houses <span id="#" class="mention-group" about="GoogleEthicalAI">the ethical AI team</span>, say <span id="#" class="mention-subject" about="Bias">authors who make claims about the benefits of <span id="#" class="mention-technology" about="NLP">large language models</span> have not received the same scrutiny during the approval process as those who highlight the shortcomings</span>.</p>

<p>Her abrupt <span id="#" class="mention-subject" about="Firing">firing</span> shows that <span id="#" class="mention-company" about="Google">Google</span> is pushing back on the kind of scrutiny that it claims to welcome, according to interviews with <span id="#" class="mention-person" about="TimnitGebru">Gebru</span>, current <span id="#" class="mention-company" about="Google">Google</span> employees, and emails and documents viewed by <span id="#" class="mention-organization" about="TheWashingtonPost">The Post</span>.</p>

<p>It raises doubts about <span id="#" class="mention-subject" about="TechCompanies">Silicon Valley</span>’s ability to self-police, especially when it comes to <span id="#" class="mention-technology" about="AI">advanced technology</span> that is largely unregulated and being deployed in the real world despite <span id="#" class="mention-subject" about="Bias">demonstrable bias toward marginalized groups</span>. Already, <span id="#" class="mention-technology" about="AI">AI systems shape decision-making in <span id="#" class="mention-organization" about="LawEnforcement">law enforcement</span>, employment opportunity and access to health care worldwide</span>.</p>

<p>That made <span id="#" class="mention-person" about="TimnitGebru">Gebru</span>’s perspective essential in a field that is predominantly <span id="#" class="mention-subject" about="Identity">White</span>, <span id="#" class="mention-subject" about="Identity">Asian</span> and <span id="#" class="mention-subject" about="Gender">male</span>. <span id="#" class="mention-subject" about="Gender">Women</span> made up only 15 percent of <span id="#" class="mention-group" about="AICommunity">the AI research staff</span> at <span id="#" class="mention-company" about="Facebook">Facebook</span> and 10 percent at <span id="#" class="mention-company" about="Google">Google</span>, according to a 2018 report in <span id="#" class="mention-publication" about="WiredMagazine">Wired magazine</span>. At <span id="#" class="mention-company" about="Google">Google</span>, <span id="#" class="mention-subject" about="Identity">Black</span> <span id="#" class="mention-subject" about="Gender">women</span> make up 1.6 percent of the workforce.</p>

<p>Although <span id="#" class="mention-company" about="Google">Google</span> publicly celebrated <span id="#" class="mention-person" about="TimnitGebru">Gebru</span>’s work <span id="#" class="mention-subject" about="Bias">identifying problems with AI</span>, it disenfranchised the work internally by keeping it hierarchically distinct from other <span id="#" class="mention-technology" about="AI">AI initiatives</span>, not heeding <span id="#" class="mention-group" about="GoogleEthicalAI">the group</span>’s advice, and not creating an incentive structure to put in practice the ethical findings, <span id="#" class="mention-person" about="TimnitGebru">Gebru</span> and other employees said.</p>

<p><span id="#" class="mention-company" about="Google">Google</span> declined to comment, but noted that in addition to the dozen or so staff members on <span id="#" class="mention-person" about="TimnitGebru">Gebru</span>’s team, 200 employees are focused on <span id="#" class="mention-subject" about="AI">responsible AI</span>.</p>

<p><span id="#" class="mention-company" about="Google">Google</span> has said that it did not <span id="#" class="mention-subject" about="Firing">fire</span> <span id="#" class="mention-person" about="TimnitGebru">Gebru</span>, but accepted her “resignation,” citing her request to explain who at <span id="#" class="mention-company" about="Google">Google</span> demanded that <span id="#" class="mention-subject" about="GebruResearchPaper">the paper</span> be retracted, according to <span id="#" class="mention-person" about="JeffDean">Dean</span>’s memo. <span id="#" class="mention-company" about="Google">The company</span> also blamed an email <span id="#" class="mention-person" about="TimnitGebru">Gebru</span> wrote to an employee resource group for <span id="#" class="mention-subject" about="Gender">women</span> and allies at <span id="#" class="mention-company" about="Google">Google</span> working in <span id="#" class="mention-subject" about="AI">AI</span> as inappropriate for a manager. The message warned the group that <span id="#" class="mention-subject" about="Bias">pushing for diversity was no use until <span id="#" class="mention-company" about="Google">Google</span> leadership took accountability</span>.</p>

<p><span id="#" class="mention-person" about="RummanChowdhury">Rumman Chowdhury</span>, a former global lead for <span id="#" class="mention-subject" about="AI">responsible AI</span> at <span id="#" class="mention-company" about="Accenture">Accenture</span> and chief executive of <span id="#" class="mention-company" about="Parity">Parity</span>, a start-up that helps companies figure out <span id="#" class="mention-subject" about="Bias">how to audit algorithms</span>, said there is a fundamental lack of respect within the industry for work on <span id="#" class="mention-subject" about="AI">AI ethics</span> compared with equivalent roles in other industries, such as model risk managers in quantitative hedge funds or threat analysts in cybersecurity.</p>

<p>“<span id="#" class="mention-subject" about="Bias">It’s being framed as the AI optimists and the people really building the stuff [versus] the rest of us negative Nellies, raining on their parade</span>,” <span id="#" class="mention-person" about="RummanChowdhury">Chowdhury</span> said. “You can’t help but notice, it’s like the <span id="#" class="mention-subject" about="Gender">boys</span> will make the toys and then the <span id="#" class="mention-subject" about="Gender">girls</span> will have to clean up.”</p>

<p><span id="#" class="mention-company" about="Google">Google</span>, which for decades evangelized an office culture that embraced employee dissent, has fired outspoken workers in recent years and shut down forums for exchange and questioning.</p>

<p>Nearly 3,000 <span id="#" class="mention-company" about="Google">Google</span> employees and more than 4,000 academics, engineers and industry colleagues have signed a petition calling <span id="#" class="mention-person" about="TimnitGebru">Gebru</span>’s termination an act of retaliation by <span id="#" class="mention-company" about="Google">Google</span>. Last week, nine Democratic lawmakers, including Sens. <span id="#" class="mention-person" about="ElizabethWarren">Elizabeth Warren</span> (Mass.) and <span id="#" class="mention-person" about="CoryBooker">Cory Booker</span> (N.J.) and Rep. <span id="#" class="mention-person" about="YvetteDClarke">Yvette D. Clarke</span> (N.Y.), sponsor of the Algorithmic Accountability Act, a bill that would require <span id="#" class="mention-subject" about="TechCompanies">companies</span> to <span id="#" class="mention-subject" about="Bias">audit and correct race and gender bias in its algorithms</span>, sent a letter to <span id="#" class="mention-company" about="Google">Google</span> chief executive <span id="#" class="mention-person" about="SundarPichai">Sundar Pichai</span> asking the company to affirm its commitment to research freedom and diversity.</p>

<p>Like any good researcher, <span id="#" class="mention-person" about="TimnitGebru">Gebru</span> is comfortable in the gray areas. And she has been using her <span id="#" class="mention-subject" about="Firing">ouster</span> as an opportunity to shed light on <span id="#" class="mention-subject" about="AI">the black box of algorithmic accountability</span> inside <span id="#" class="mention-company" about="Google">Google</span> — annotating the company’s claims with contradictory data, <span id="#" class="mention-subject" about="Bias">drawing connections to larger systemic issues, and illuminating the way internal <span id="#" class="mention-subject" about="AI">AI ethics</span> efforts can break down without oversight or a change in incentives to corporate practices and power structures</span>.</p>

<p><span id="#" class="mention-subject" about="TechCompanies">Big Tech</span> dominates <span id="#" class="mention-subject" about="AI">AI research</span> around advancements in <span id="#" class="mention-technology" about="NLP">machine learning</span>, <span id="#" class="mention-technology" about="FacialRecognition">image recognition</span>, <span id="#" class="mention-technology" about="LanguageTranslation">language translation</span> — poaching talent from <span id="#" class="mention-organization" about="Academia">top universities</span>, sponsoring conferences and publishing influential papers. In response to <span id="#" class="mention-subject" about="Bias">concerns about the way those technologies could be abused or compound bias</span>, the industry ramped up funding and promotion of <span id="#" class="mention-subject" about="AI">AI ethics</span> initiatives, beginning around 2016.</p>

<p><span id="#" class="mention-subject" about="TechCompanies">Tech giants</span> have made similar investments in shaping policy debate around antitrust and online privacy, as a way to ward off <span id="#" class="mention-organization" about="USGovernment">lawmakers</span>. <span id="#" class="mention-person" about="SundarPichai">Pichai</span> invoked <span id="#" class="mention-company" about="Google">Google</span>’s <span id="#" class="mention-subject" about="GoogleAIPrinciples">AI principles</span> in an interview in 2018 with <span id="#" class="mention-organization" about="TheWashingtonPost">The Post</span>, arguing for self-regulation around <span id="#" class="mention-subject" about="AI">AI</span>.</p>

<p><span id="#" class="mention-company" about="Google">Google</span> created its <span id="#" class="mention-group" about="GoogleEthicalAI">Ethical AI group</span> in 2018 as an outgrowth of an employee-led push to prioritize fairness in the company’s <span id="#" class="mention-technology" about="AI">machine learning applications</span>. <span id="#" class="mention-person" about="MargaretMitchell">Margaret Mitchell</span>, <span id="#" class="mention-person" about="TimnitGebru">Gebru</span>’s co-lead, pitched the idea of a team of researchers investigating <span id="#" class="mention-subject" about="Bias">the long-term effects of <span id="#" class="mention-technology" about="AI">AI</span> and translating those findings into action to mitigate harm and risk</span>.</p>

<p>The same year, <span id="#" class="mention-person" about="SundarPichai">Pichai</span> released a <span id="#" class="mention-subject" about="GoogleAIPrinciples">broadly worded set of principles governing <span id="#" class="mention-company" about="Google">Google</span>’s AI work</span> after thousands of employees protested the company’s contract with <span id="#" class="mention-organization" about="USGovernment">the Pentagon</span> to analyze surveillance imagery from drones. But <span id="#" class="mention-company" about="Google">Google</span>, which requires privacy and security tests before any product launch, has not mandated an equivalent process for vetting <span id="#" class="mention-subject" about="AI">AI ethics</span>, employees say.</p> 

<p><span id="#" class="mention-person" about="TimnitGebru">Gebru</span>, whose family’s <span id="#" class="mention-subject" about="Identity">ethnic origins</span> are in <span id="#" class="mention-location" about="Eritrea">Eritrea</span>, was born and raised in <span id="#" class="mention-location" about="Ethiopia">Ethiopia</span> and came to <span id="#" class="mention-location" about="Massachusetts">Massachusetts</span> as 16-year-old after receiving political asylum from the war between the two African countries. She began her career as an electrical engineer at <span id="#" class="mention-company" about="Apple">Apple</span> and received her PhD from <span id="#" class="mention-organization" about="Stanford">the Stanford Artificial Intelligence Lab</span>, studying <span id="#" class="mention-technology" about="AI">computer vision</span> under renowned computer scientist <span id="#" class="mention-person" about="FeiFeiLi">Fei-Fei Li</span>, a former <span id="#" class="mention-company" about="Google">Google</span> executive and now co-director of <span id="#" class="mention-organization" about="Stanford">Stanford’s Human-Centered AI Institute</span>, which receives funding from <span id="#" class="mention-company" about="Google">Google</span>.</p>

<p><span id="#" class="mention-person" about="TimnitGebru">Gebru</span> did her postdoctoral research at <span id="#" class="mention-company" about="Microsoft">Microsoft Research</span> as part of a group focused on <span id="#" class="mention-subject" about="AI">accountability and ethics in AI</span>. There, <span id="#" class="mention-person" about="TimnitGebru">she</span> and <span id="#" class="mention-person" about="JoyBuolamwini">Joy Buolamwini</span>, then a masters student at <span id="#" class="mention-organization" about="MIT">MIT Media Lab</span>, co-wrote a groundbreaking 2018 study that found that <span id="#" class="mention-technology" about="AI">commercial facial recognition tools</span> sold by companies such as <span id="#" class="mention-company" about="IBM">IBM</span> and <span id="#" class="mention-company" about="Microsoft">Microsoft</span> were 99 percent accurate at identifying <span id="#" class="mention-subject" about="Identity">White</span> <span id="#" class="mention-subject" about="Gender">males</span>, but only 35 percent effective with <span id="#" class="mention-subject" about="Identity">Black</span> <span id="#" class="mention-subject" about="Gender">women</span>.</p>

<p>In June, <span id="#" class="mention-company" about="IBM">IBM</span>, <span id="#" class="mention-company" about="Microsoft">Microsoft</span> and <span id="#" class="mention-company" about="Amazon">Amazon</span> announced that they would stop selling the software to <span id="#" class="mention-organization" about="LawEnforcement">law enforcement</span>, which <span id="#" class="mention-person" about="JeffDean">Dean</span> credited to <span id="#" class="mention-person" about="TimnitGebru">Gebru</span>’s work. <span id="#" class="mention-person" about="TimnitGebru">She</span> also co-founded <span id="#" class="mention-organization" about="BlackInAI">Black in AI</span>, a nonprofit organization that increased the number of <span id="#" class="mention-subject" about="Identity">Black</span> attendees at <span id="#" class="mention-event" about="AIConference">the largest annual AI conference</span>.</p>

<p><span id="#" class="mention-person" about="TimnitGebru">Gebru</span> said that in 2018, <span id="#" class="mention-company" about="Google">Google</span> recruited her with the promise of total academic freedom. She was unconvinced, but the company was opening its <span id="#" class="mention-group" about="AICommunity">first artificial intelligence lab on the African continent in <span id="#" class="mention-location" about="Accra">Accra</span></span>, the capital of <span id="#" class="mention-location" about="Ghana">Ghana</span>, and she wanted to be involved. When she joined <span id="#" class="mention-company" about="Google">Google</span>, <span id="#" class="mention-person" about="TimnitGebru">Gebru</span> said, she was the first <span id="#" class="mention-subject" about="Identity">Black</span> <span id="#" class="mention-subject" about="Gender">female</span> researcher in the company. (When she left, there were still only a handful of <span id="#" class="mention-subject" about="Identity">Black</span> <span id="#" class="mention-subject" about="Gender">women</spam> working in research, out of hundreds.)</p>

<p><span id="#" class="mention-person" about="TimnitGebru">Gebru</span> said she was also drawn to working with <span id="#" class="mention-person" about="MargaretMitchell">Mitchell</span>. Both <span id="#" class="mention-subject" about="Gender">women</span> prioritized foresight and building practical solutions to prevent <span id="#" class="mention-subject" about="AI">AI risk</span>, whereas <span id="#" class="mention-subject" about="Bias">the operating mind-set in <span id="#" class="mention-subject" about="TechCompanies">tech</span> is biased toward benefits and “rapid hindsight,” in response to harm</span>, <span id="#" class="mention-person" about="MargaretMitchell">Mitchell</span> said.</p>

<p><span id="#" class="mention-person" about="TimnitGebru">Gebru</span>’s approach to <span id="#" class="mention-subject" about="AI">ethical AI</span> was shaped by her experiences. Hardware, for instance, came with datasheets that documented whether components were safe to use in certain situations. “When you look at this field as a whole, that doesn’t exist,” said <span id="#" class="mention-person" about="TimnitGebru">Gebru</span>, an electrical engineer. “It’s just super behind in terms of documentation and standards of safety.”</p>

<p><span id="#" class="mention-person" about="TimnitGebru">She</span> also leaned on her industry experience when collaborating with other teams. Engineers live on a product cycle, consumed with putting out fires and fixing bugs. <span id="#" class="mention-subject" about="Bias">A vague requirement to “make things fair” would only cause more work and frustration, she thought. So she tried to build institutional structures and documentation tools for “when people want to do the right thing</span>.”</p>

<p>Despite their expertise, <span id="#" class="mention-group" about="GoogleEthicalAI">the Ethical AI group</span> fought to be taken seriously and included in <span id="#" class="mention-technology" about="AI"><span id="#" class="mention-company" about="Google">Google</span>’s other AI efforts</span>, employees said.</p>

<p>Within the company, <span id="#" class="mention-person" about="TimnitGebru">Gebru</span> and her former colleagues said, there is little transparency or accountability regarding how decisions around <span id="#" class="mention-subject" about="AI">AI ethics</span> or diversity initiatives get made. Work on <span id="#" class="mention-subject" about="GoogleAIPrinciples">AI principles</span>, for instance, falls under <span id="#" class="mention-person" about="KentWalker">Kent Walker</span>, the senior vice president of global affairs, whose vast purview includes lobbying, public policy and legal work. <span id="#" class="mention-person" about="KentWalker">Walker</span> also runs an internal ethics board of top executives, including <span id="#" class="mention-person" about="JeffDean">Dean</span>, called the <span id="#" class="mention-group" about="AdvancedTechnologyReviewCouncil">Advanced Technology Review Council</span>, which is responsible for yes or no decisions when issues escalate, <span id="#" class="mention-person" about="TimnitGebru">Gebru</span> said. <span id="#" class="mention-group" about="GoogleEthicalAI">The Ethical AI team</span> had to fight to be consulted on <span id="#" class="mention-person" about="KentWalker">Walker</span>’s initiatives, she said.</p>

<p>“Here’s the guy tasked with covering <span id="#" class="mention-company" about="Google">Google</span>’s a--, lobbying and also … working on <span id="#" class="mention-subject" about="GoogleAIPrinciples">AI principles</span>,” <span id="#" class="mention-person" about="TimnitGebru">Gebru</span> said. “Shouldn’t you have a different entity that pushes back a little bit internally — some sort of push and pull?” What’s more, members of <span id="#" class="mention-person" about="KentWalker">Walker</span>’s <span id="#" class="mention-group" about="AdvancedTechnologyReviewCouncil">council</span> are <span id="#" class="mention-subject" about="Bias">predominantly vice presidents or higher, constricting diversity</span>, <span id="#" class="mention-person" about="TimnitGebru">Gebru</span> said.</p>

<p>In her conversations with product teams, such as a group working on fairness in <span id="#" class="mention-technology" about="AI">Machine Learning Infrastructure</span>, <span id="#" class="mention-person" about="TimnitGebru">Gebru</span> said she kept getting questions about <span id="#" class="mention-subject" about="Bias">what tools and features they could build to protect against the ethical risks involved with <span id="#" class="mention-technology" about="NLP">large language models</span></span>. <span id="#" class="mention-company" about="Google">Google</span> had credited it with the biggest breakthrough in improving <span id="#" class="mention-technology" about="SearchEngines">search results</span> in the past five years. <span id="#" class="mention-technology" about="NLP">The models</span> can process words in relation to the other words that come before and after them, which is useful for understanding the intent behind <span id="#" class="mention-technology" about="SearchEngines">conversational search queries</span>.</p>

<p>But despite the increasing use of <span id="#" class="mention-technology" about="NLP">these models</span>, there was <span id="#" class="mention-subject" about="Bias">limited research investigating groups that might be negatively impacted</span>. <span id="#" class="mention-person" about="TimnitGebru">Gebru</span> says she wanted to help develop those safeguards, one of the reasons she agreed to collaborate with <span id="#" class="mention-subject" about="GebruResearchPaper">the research paper</span> proposed by <span id="#" class="mention-person" about="EmilyMBender">Emily M. Bender</span>, a linguist at <span id="#" class="mention-organization" about="UniversityOfWashington">the University of Washington</span>.</p>

<p><span id="#" class="mention-person" about="MargaretMitchell">Mitchell</span>, who developed the idea of <span id="#" class="mention-subject" about="AIModelCards">model cards</span>, like nutrition labels for <span id="#" class="mention-technology" about="NLP">machine learning models</span>, described <span id="#" class="mention-subject" about="GebruResearchPaper">the paper</span> as “due diligence.” Her <span id="#" class="mention-subject" about="AIModelCards">model card</span> idea is being adopted more widely across <span id="#" class="mention-group" about="AICommunity">the industry</span>, and engineers needed to know how to fill out the section on harm.</p>

<p><span id="#" class="mention-person" about="TimnitGebru">Gebru</span> said her biggest contribution to both <span id="#" class="mention-group" about="GoogleEthicalAI">her team</span> and <span id="#" class="mention-subject" about="GebruResearchPaper">the paper</span> has been identifying researchers who study directly-affected communities.</p>

<p>That diversity was reflected in the authors of <span id="#" class="mention-subject" about="GebruResearchPaper">the paper</span>, including <span id="#" class="mention-person" about="MarkDiaz">Mark Diaz</span>, a <span id="#" class="mention-subject" about="Identity">Black and Latino</span> <span id="#" class="mention-company" about="Google">Google</span> researcher whose previous work looked at <span id="#" class="mention-subject" about="Bias">how <span id="#" class="mention-subject" about="TechCompanies">platforms</span> leave out the elderly, who talk about ageism in blog posts, but don’t share as much on sites such as <span id="#" class="mention-company" about="Twitter">Twitter</span></span>. For <span id="#" class="mention-subject" about="GebruResearchPaper">the paper</span>, <span id="#" class="mention-person" about="MarkDiaz">he</span> identified the possibility that <span id="#" class="mention-technology" about="DataSets">large data sets</span> from <span id="#" class="mention-technology" about="Internet">the Internet</span>, particularly if they are from a single moment in time, <span id="#" class="mention-subject" about="Bias">will not reflect cultural shifts from social movements</span>, such as the <span id="#" class="mention-subject" about="MeToo">#MeToo movement</span> or <span id="#" class="mention-organization" about="BLM">Black Lives Matter</span>, which seek to shift power through changes in language.</p>

<p><span id="#" class="mention-subject" about="GebruResearchPaper">The paper</span> identified <span id="#" class="mention-subject" about="Bias">four overarching categories of harm</span>, according to a recent draft viewed by <span id="#" class="mention-organization" about="TheWashingtonPost">The Post</span>. It delved into <span id="#" class="mention-subject" about="EnergyConsumption">the environmental effect of the computing power</span>, the inscrutability of <span id="#" class="mention-technology" about="DataSets">massive data sets</span> used to train <span id="#" class="mention-technology" about="NLP">these models</span>, <span id="#" class="mention-subject" about="MisdirectedResearch">the opportunity costs of the “hype” around claims that <span id="#" class="mention-technology" about="NLP">these models</span> can understand language, as opposed to identifying patterns</span>, and <span id="#" class="mention-subject" about="IllusionOfMeaning">the danger that the real-sounding text generated by <span id="#" class="mention-technology" about="NLP">such models</span> could be used to spread misinformation</span>.</p>

<p>Because <span id="#" class="mention-company" about="Google">Google</span> depends on <span id="#" class="mention-technology" about="NLP">large language models</span>, <span id="#" class="mention-person" about="TimnitGebru">Gebru</span> and <span id="#" class="mention-person" about="MargaretMitchell">Mitchell</span> expected that the company might push back against certain sections or attempt to water down their findings. So they looped in PR &amp; Policy representatives in mid-September, with plenty of time before the deadline for changes at the end of January 2021.</p>

<p>Before making a pre-publication draft available online, <span id="#" class="mention-person" about="TimnitGebru">Gebru</span> first wanted to vet <span id="#" class="mention-subject" about="GebruResearchPaper">the paper</span> with <span id="#" class="mention-group" about="AICommunity">a variety of experts</span>, including those who have built <span id="#" class="mention-technology" about="NLP">large language models</span>. She asked for feedback from two top people at <span id="#" class="mention-company" about="OpenAI">OpenAI</span>, an <span id="#" class="mention-subject" about="AI">AI research</span> lab co-founded by <span id="#" class="mention-person" about="ElonMusk">Elon Musk</span>, in addition to her manager at <span id="#" class="mention-company" about="Google">Google</span>, and about 30 others. They suggested additions or revisions, <span id="#" class="mention-person" about="TimnitGebru">Gebru</span> said. “I really wanted to send it to people who would disagree with our view and be defensive,” <span id="#" class="mention-person" about="TimnitGebru">Gebru</span> said.</p>

<p>Given all their upfront effort, <span id="#" class="mention-person" about="TimnitGebru">Gebru</span> was baffled when she received a notification for a meeting with <span id="#" class="mention-group" about="GoogleResearch">Google Research</span> Vice President <span id="#" class="mention-person" about="MeganKacholia">Megan Kacholia</span> at 4.30 p.m. on the Thursday before Thanksgiving.</p>

<p>At the meeting, <span id="#" class="mention-person" about="MeganKacholia">Kacholia</span> informed <span id="#" class="mention-person" about="TimnitGebru">Gebru</span> and her co-authors that <span id="#" class="mention-company" about="Google">Google</span> wanted <span id="#" class="mention-subject" about="GebruResearchPaper">the paper</span> retracted.</p>

<p>On Thanksgiving, a week after the meeting, <span id="#" class="mention-person" about="TimnitGebru">Gebru</span> composed a six-page email to <span id="#" class="mention-person" about="MeganKacholia">Kacholia</span> and <span id="#" class="mention-person" about="JeffDean">Dean</span> outlining how disrespectful and oddly secretive the process had been.</p>

<p>“Specific individuals should not be empowered to unilaterally shut down work in such a disrespectful manner,” <span id="#" class="mention-person" about="TimnitGebru">she</span> wrote, adding that researchers from underrepresented groups were mistreated.</p>

<p><span id="#" class="mention-person" about="MargaretMitchell">Mitchell</span>, who is <span id="#" class="mention-subject" about="Identity">White</span>, said she shared <span id="#" class="mention-person" about="TimnitGebru">Gebru</span>’s concerns but did not receive the same treatment from the company. “<span id="#" class="mention-company" about="Google">Google</span> is very hierarchical, and it’s been a battle to have any sort of recognition,” she said. “We tried to explain that <span id="#" class="mention-person" about="TimnitGebru">Timnit</span>, and to a lesser extent me, are respected voices publicly, but we could not communicate upwards.”</p>

<p>“How can you still ask why there aren’t <span id="#" class="mention-subject" about="Identity">Black</span> <span id="#" class="mention-subject" about="Gender">women</span> in this industry?” <span id="#" class="mention-person" about="TimnitGebru">Gebru</span> said.</p>

<p><span id="#" class="mention-person" about="TimnitGebru">Gebru</span> said she found out this week that <span id="#" class="mention-subject" about="GebruResearchPaper">the paper</span> was accepted to the Conference on Fairness, Accountability and Transparency, as part of its anonymous review process. “It’s sad, the scientific community respects us a lot more than anybody inside <span id="#" class="mention-company" about="Google">Google</span>,” she said. <span class="end-dot"></span></p>