While AI is becoming increasingly impactful and broad-reaching, its research and development has become centralized in a few. As AI progresses, developments towards superintelligence should aim to address maximal and accessible support of many, yet this is limited by the power of a few "big tech" corporations. Inevitably, without diverse involvement in AI development, we get less consideration of different future impacts, and large corporations are particularly skewed in their goals. In thinking about the implications of the current state of AI development, important ideas arise: the importance of open-source, transparency and interpretability, efficiency, and challenging pure scaling.
The following meta-research write-up was originally a report for the course Stanford ESF20A: Science as Culture, examining the double-sided nature of the role of science on humans and our role in the course of science.
The AI Divide: Power, Exclusivity, and Culture in AI
ABSTRACT
AI has grown to impact many around the world; at the same time, a select few hold power over the development of the pervasive technologies as the field becomes more exclusive. Currently AI models rely heavily on large amounts of data and compute power, centralizing influence among resource-rich, undiverse corporations. Their values have thus shifted the trends in AI towards scaling and English-centric applications that serve their interests at the expense of inclusive and accessible advancements that can support more minority languages and communities. The social and cultural contexts of these key players influence not only what technologies are prioritized but also how they are designed and deployed, embedding human biases into AI systems. This study critically examines the power dynamics within AI development, the sociocultural factors shaping its trajectory, and the implications of these trends on equitable technological progress.
INTRODUCTION
The spirit of the Silicon Valley is perhaps most captured by the motto of one of its most prominent figures, Meta’s Mark Zuckerberg: “move fast and break things”. It’s full of an addictive energy, of adventurous risk-taking—of destructive foolhardiness. The approach of many companies has been creating rapidly evolving technologies and releasing them out into the world where they have access to impact the lives of millions and billions. The damage they may cause, the havoc they may wreak all becomes a distant issue. While the rest of the world faces the repercussions of their creations, those at the top remain untouched. It’s tunnel vision for the next biggest, greatest product, and the technologies become embedded with the hustle mindset and self-interested motives.
With the rapid growth of AI, the field’s inhumanistic approaches become increasingly alarming. Even as AI’s reach expands, who is in power narrows. The values and priorities of those who shape technologies inevitably become reflected in their creations, from assumptions made in the algorithms to the choice of problems deemed worth solving. Limited diversity of perspectives can thus produce major blind spots. Even as algorithms are deemed the result of objective math and numbers, human judgements guide the direction of their progress.
In order to better understand the impact of AI, it is necessary to examine the influences on AI’s development. This paper delves into these valuable areas, exploring the leaders and course of AI development, the exclusivity and power dynamics of the field, and the values that drive advancements. First, I examine the rise of corporate dominance in the field of AI and how this has directed the course of AI. Next, I describe the demographic imbalances in AI. With the power hierarchy of AI established, I study the influence this has on the technologies themselves and the risks that can emerge from the exclusivity. By putting AI into the context of its creators, the intertwinement of technology and culture comes to light.
CORPORATE EXCLUSIVITY IN AI DEVELOPMENT
As AI has recently taken center stage as a prominent topic, several names appear again and again at the forefront of rapid developments. A select few Big Tech companies—including OpenAI, Google, and Meta—seem to have power over the technologies in the hands of billions. The power dynamics within the field of AI have become shaped by the context and influence of Big Tech.
Currently, the key components of developing quality AI systems are access to large volumes of training data for the algorithms to learn patterns from and tremendous amounts of computational power to process them. The process is extremely costly: the fourth iteration of ChatGPT, for instance, was trained on about 25,000 graphical processing units and 13 trillion words, resulting in a cost of over $100 million to train (Knight 2023: pg. 1). This presents a barrier of entry into the AI space. Through the history of their technological development, Big Tech companies have gained access to internet data by collecting information from their large user bases. The dominance of Big Tech, with their financial power and their reign over digital infrastructure and information, is compounded with the emergence of AI, where these resources are valuable. With exclusive access to such capital, Big Tech monopolizes AI development. The result is a “compute divide” between Big Tech and research centers like universities, concentrating AI research to the elite corporate world (Khanal et al. 2024: pg. 3). The corporate control over AI advancements brings differing motivations and interests from traditional innovation-centric values.
Perhaps the most significant result of Big Tech’s control over the course of AI development is the focus on scaling: relying on loading more and more data into AI models to yield improvements, instead of making any major algorithmic changes. Bender et al. (2022) describes this focus on making larger models, explaining how “the past 3 years of work in [natural language processing] have been characterized by the development and deployment of ever larger language models, especially for English." (pg. 610). The focus on scaling serves to secure the power of Big Tech, who hold the advantage of resources in data, computational power, and wealth. While the potential of scaling is restricted and prioritizes applications where big data is available, like high-resource languages such as English which dominate the internet, these limitations are accepted by Big Tech. Their market values target making profit rather than making a positive social impact with their technology. LaFrance (2024) describes this ideology of Big Tech companies as an “authoritarian technocracy.” The exclusivity in the creators of AI thus results in a similar exclusivity in who is best served by AI. Robertson (2017) describes how the sophistication and cost of innovation can generate a reliance on corporate funding for development; thus, creations “tend to both mirror and embody state and corporate ideologies and priorities” (pg. 82). This relationship forms a closed cycle in the path of AI development.
The pervasiveness of corporate interests in AI development is exemplified by the story of OpenAI. Despite starting as a nonprofit organization centered on serving the public, in order to hold any power, OpenAI had to initiate a for-profit arm which Microsoft invested billions of dollars into. This led to the deterioration of the company’s original values. The company CEO, Sam Altman, who was originally fired for reportedly “too heavily prioritizing the pace of development over safety” (LaFrance 2024: pg. 8), was then promptly rehired by Microsoft’s offer. The situation signaled the victory over the pursuit of scale over good, and the role of corporate values in establishing such a divide.
What is most concerning about the exclusivity in AI development by Big Tech is that the world becomes blind to their destructiveness and lack of morals through the narratives they present of AI. Big Tech holds control over media and policy. Big Tech companies are a significant source of funding for many news outlets, and hold the ability to selectively control the visibility of content online (Khanal et al. 2024: pg. 4). Big Tech presents AI through a lens of technological solutionism, promoting its positive potential and masking its flaws with terms like "algorithmic neutrality" and "technological objectivity" (Noble 2018). Consequently, an overly optimistic perception is built towards AI, a perception that preaches how it ‘democratizes’ access to different tools while they reign with absolute authority. In policy, corporate powers influence the debate on AI and push “policy toward a narrow focus on risks and safety” (Joyce & Cruz, 2024: pg. 4). Avoiding regulation and crafting their own ethical standards for AI, Big Tech is not only able to dominate in the realm of AI, but ensure that the world accepts and contributes to their technologies as well. They are able to selectively identify the problems and applications of value and present those to the public to shape the future of AI progress.
The corporate values of those in power over AI—i.e. Big Tech—prioritize their market interests and economic value over inclusivity, global support, and responsibility. Scaling and Western-centric applications are favored, and Big Tech’s power over public perception has enabled them to present AI as hopeful, diverting focus from risks and inequality.
DEMOGRAPHIC EXCLUSIVITY IN AI DEVELOPMENT
The dominance of AI by a select few companies has restricted the perspectives in the field through biases in the demographic representation within those companies. AI has become dominated primarily by wealthy left-leaning white men. At Google in the US, for instance, “only 2.5% of employees were Black, while 3.6% were Hispanic and Latinx,” a negligible increase despite claims of working towards diversifying the makeup of their workforce (Google 2018: pg. 5). Beyond race, there are also gender disparities in the field of AI. According to recent data from the National Center for Education Statistics, only 24% of computer science and mathematics degrees awarded were to women (National Science Foundation 2023: pg. 45). Even fewer make it into high tech roles in the workforce (Nix 2024: pg. 1).
The limited diverse representation in such positions in Big Tech, who, in turn, take the lead at the forefront of AI development, holds concerning implications for the course of AI development. Robertson (2017), in her investigations of technovation in Japan, describes her findings of how developers often “take for granted in their own gendered upbringing and everyday behavior—which is often resistant to change—is reproduced in the stereotyped forms they give, and the activities they assign” to their creations (pg. 90). Her analysis emphasizes the role of having diverse perspectives in innovation to prevent ‘renovative’ impacts that perpetuate harmful limited views. Big Tech doesn't just build machines and code, but the tools that reach everyday people around the world that reflect human values. As the bubble of Silicon Valley thinking brings little diversity in the views, perspectives, and values that inform AI development, risks emerge in the creations that reach many.
BIASES IN AI
Now that I have established how AI has become an exclusive field and the socio-cultural factors that have shaped the course of AI development, I will examine how that, in turn, directly shapes the impact of AI back on society through the biases within it.
The first place that AI algorithms pick up flawed human biases in their training is from the data that they learn from and draw out their internal representations of the world. Especially with the corporate, Western focus on scaling with more data, data has become a crucial aspect to shaping present AI models. Bias in data comes in two main forms: linguistic bias and selection bias.
Linguistic bias comes into play with the lack of language diversity present in online data. Over half of the internet, the greatest source of digital data for training AI, is in English, with a staggering gap in the remainder of the worlds’ languages (Williams 2021: pg. 1). Even the second most used language in the world, Chinese, represents only 0.3% of internet content. With the lack of sufficient information that can be supplied to models in diverse languages from data alone, models become strongly skewed towards English, leaving much of the world underserved. This is a disregarded problem to corporate biases with priorities of Western applications, of course. With the significant performance gap in minority languages, historically underserved global populations experience even more augmented disparities in their access to information and technology in the digital age.
The tangible harmful impact of linguistic bias is exemplified by relatively recent events in Myanmar. In 2017, the Myanmar military launched a campaign of ethnic cleansing against the Muslim Rohingya population, thousands being killed, tortured, displaced, and harmed in the storm of religious-based hatred. The key weapon of destruction? Facebook. The platform held a significant role in spreading provocative speech and enabling the event after the failure of their AI content moderation system in detecting hate speech (Human Rights Council Thirty-Ninth Session, 2018). AI, with its focus on scaling, leaves low-resource languages like Burmese behind. Western standards of hate speech established by Facebook further skewed the algorithm against the cultural nuances of Burmese hate speech. Facebook recognized the need for changes in their system as well as greater diversity in their workers. The event highlights how the priorities of those in power of AI development have neglected the needs of minority communities, and how this can have great consequences in the world.
Beyond the impact of linguistic bias is the role of representation bias in algorithms. These algorithms learn flawed patterns from humanly flawed data that are often unrepresentative of truth. Biases in AI can be observed in many aspects of the models; a famous harmful association they learn includes the famous “Man is to Computer Programmer as Woman is to Homemaker” (Bolukbasi et al. 2016). Such biases are picked up from poor representation from the data that they learn, and in this way these algorithms become not progressive but regressive, perpetuating existing human biases. Robinson (2017) describes such technologies as “renovative” rather than innovative (pg. 35). Beyond gender biases, many groups are systematically underrepresented and misrepresented in data. Socioeconomic gaps bring inequalities in data collection in communities with less access to technology; geographic divides in urban and rural areas introduce differences; and more disparities arise across age, race, marginalized communities, and global power. The benefits and risks of AI are not distributed equally; marginalized communities bear the costs of algorithmic decision-making without the gains. Data becomes a prominent source of bias in AI, and it is exacerbated by the neglect of developers with little consideration for these issues due to their corporate values and restricted backgrounds.
Perhaps more concerning than biases introduced into AI by data are the biases introduced into AI directly by their creators. In AI development, developers have much more control over setting the ethical standards of AI than one might expect, and the key is through a process called reinforcement learning with human feedback (Ouyang et al. 2022). After the initial training stage of AI models from the data, they undergo a stage of alignment with human involvement, where people manually determine the ethical standards by which the AI models will operate on once deployed and select what AI can and cannot say. The process shapes the model based on the views of those providing feedback to the model. Essentially, a select few get to “play God” on what is ‘right’ and ‘wrong’. This is warped by the skewed ideologies of those in power creating these models. For example, a recent Twitter post noted the responses of ChatGPT when asked to write a poem about the positive attributes of Donald Trump and Joe Biden, while ChatGPT refused to answer for the former, stating its reluctance to get involved in politics as a ‘neutral’ tool, it readily complied with poem for the latter (Baum & Villasenor 2023: pg. 4). The example demonstrates the role of subjective human views, and particularly, the human views of a select few, on the biases in AI algorithms.
CONCLUSION
AI has become dominated by Big Tech and corporate values that push the trends of development towards skewed priorities of scaling and socioeconomically and geopolitically prioritized applications. Just as AI has had a profound impact on humans, sociocultural factors have just as much impacted the course of AI development—human factors that bring their own human flaws and imperfections into the widespread pervasive tools. The solution to overcoming these limitations is embracing diversified views that normalize individual experiences into a comprehensive perspective; however, the exclusivity in the field of AI development has endangered such a possibility, raising an urgent need for change in the approach to AI. Diversifying the field of AI with a representation of backgrounds and interdisciplinary thought is a key step. This ensures that a broader range of perspectives, ethical considerations, and cultural nuances are integrated into the development process. To make AI more accessible, redirecting the focus from scaling with more data towards data-efficient algorithms that can support a broader range of applications is a crucial step. Not only does this shift promote diversity in who participates in and benefits from AI, but it also enhances AI's overall capabilities, driving innovation with creative and unique views. Technology, as it reaches the hands of people, can no longer be approached without recognizing its intertwinement with humans. By adopting this holistic view, AI development can achieve a more positive, inclusive, and sustainable integration with society.
REFERENCES
Baum, Jeremy, and John Villasenor. “The Politics of AI: ChatGPT and Political Bias.” Brookings, 2023, https://www.brookings.edu/articles/the-politics-of-ai-chatgpt-and-political-bias/.
Bender, Emily M., et al. “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜.” Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, Association for Computing Machinery, 2021, pp. 610–23, https://doi.org/10.1145/3442188.3445922.
Bolukbasi, Tolga, et al. Man Is to Computer Programmer as Woman Is to Homemaker? Debiasing Word Embeddings. arXiv:1607.06520, arXiv, 21 July 2016, https://doi.org/10.48550/arXiv.1607.06520.
Diversity and STEM: Women, Minorities, and Persons with Disabilities 2023 | NSF - National Science Foundation. https://ncses.nsf.gov/pubs/nsf23315/report/science-and-engineering-degrees-earned#overall-s-e-degrees-earned-by-women. Accessed 3 Dec. 2024.
Google. 2018.
Human Rights Council Thirty-Ninth Session 10-28 September 2018 Agenda Item 4 Human Rights Situations That Require the Council’s Attention Report of the Independent International Fact-Finding Mission on Myanmar* Summary. 12 Sept. 2018.
Joyce, Kelly, and Taylor M. Cruz. “A Sociology of Artificial Intelligence: Inequalities, Power, and Data Justice.” Socius: Sociological Research for a Dynamic World, vol. 10, 2024, p. 23780231241275390, https://doi.org/10.1177/23780231241275393.
Khanal, Shaleen, et al. “Why and How Is the Power of Big Tech Increasing in the Policy Process? The Case of Generative AI.” Policy and Society, Mar. 2024, p. puae012, https://doi.org/10.1093/polsoc/puae012.
Knight, Will. “OpenAI’s CEO Says the Age of Giant AI Models Is Already Over.” Wired, 2023, https://www.wired.com/story/openai-ceo-sam-altman-the-age-of-giant-ai-models-is-already-over/.
LaFrance, Adrienne. “The Rise of Techno-Authoritarianism.” The Atlantic, 30 Jan. 2024, https://www.theatlantic.com/magazine/archive/2024/03/facebook-meta-silicon-valley-politics/677168/.
Nix, Naomi. Job Rate for Women in Tech Has Hardly Budged since 2005, EEOC Finds. The Washington Post, 2024.
Noble, Safiya Umoja. Algorithms of Oppression: How Search Engines Reinforce Racism. NYU Press, 2018, https://doi.org/10.2307/j.ctt1pwt9w5.
Ouyang, Long, et al. Training Language Models to Follow Instructions with Human Feedback. arXiv:2203.02155, arXiv, 4 Mar. 2022, https://doi.org/10.48550/arXiv.2203.02155.
Robertson, Jennifer. “Robo Sapiens Japanicus.” University of California Press, 2017, https://www.ucpress.edu/books/robo-sapiens-japanicus/paper.
Williams, Demetrius. “Why the Most Popular World Languages Aren’t Reflected Online?” Toppan Digital Language, 10 June 2021, https://toppandigital.com/us/blog-usa/why-the-most-popular-world-languages-arent-reflected-online/.
Comments