Behind the Screen

1. Commercial Content Moderation: The Invisible, Essential Internet Labor

Commercial content moderators are professional people paid to screen content uploaded to the internet’s social media sites on behalf of the firms that solicit user participation.

Defining the work. Commercial content moderation (CCM) involves paid professionals reviewing user-generated content (UGC) on platforms like social media, news sites, and apps. Their job is to decide whether content adheres to site guidelines and legal standards, quickly evaluating thousands of items daily. Unlike earlier volunteer moderators, they are often deliberately kept discreet and undetectable.

Why it's essential. As social media platforms grew to global scale, the volume of UGC became staggering (e.g., hundreds of hours of video uploaded to YouTube per minute). Companies need CCM for brand protection, legal compliance (e.g., copyright, child abuse material), and maintaining a usable, appealing environment for users. Without it, platforms would be overwhelmed by objectionable, illegal, or off-topic content.

The paradox of invisibility. Despite being mission-critical, CCM workers and their labor are largely invisible to the public. This opacity is by design, allowing platforms to maintain an image of unfettered user expression while quietly controlling content flow. Unveiling this hidden labor is crucial to understanding how the contemporary internet is shaped and for whom.

2. From Volunteer Governance to Industrial-Scale Gatekeeping

What is new, however, is the industrial-scale organized content moderation activities of professionals who are paid for their evaluative gatekeeping services, and who undertake the work they do on behalf of large-scale commercial entities.

Early internet moderation. Content moderation isn't new; early online communities (MOOs, MUDs, BBSs) had users creating and enforcing rules, often through volunteer efforts. These spaces developed unique norms and self-governance structures, reflecting the community's identity and values. Access was limited, primarily to academics and tech enthusiasts.

The shift to commercial scale. The explosion of the World Wide Web and social media transformed the internet into a mass-market, commercial space. This necessitated a shift from volunteer, community-based moderation to organized, paid, industrial-scale operations. Large corporations, not users, now dictate the rules and employ gatekeepers to enforce them.

Gatekeeping for profit. This new era of CCM is driven by commercial interests. Moderators act on behalf of platforms and brands to curate content that maximizes user engagement and advertising revenue. This fundamentally changes the nature of online spaces, prioritizing corporate goals over community norms or ideals of free expression.

3. A Fractured Global Workforce: The Taxonomy of Moderation Labor

To understand the practice of commercial content moderation, it is necessary to gain a picture of how, where, and by whom the work of moderation takes place.

Diverse labor arrangements. CCM work is not monolithic but occurs under various organizational and geographic arrangements, often by design to reduce costs and accountability. Key types include:

• In-house: Workers on-site at the tech company, often contractors.
• Boutique: Specialized firms managing online presence for clients.
• Call Center: Large BPO operations offering moderation as one service.
• Microlabor Platform: Task-based work on sites like Mechanical Turk.

Global dispersion. Workers are often located far from the content's origin or destination, leveraging global labor pools for cost savings and specific linguistic/cultural skills. This dispersion makes the workforce difficult to track, organize, and advocate for.

Hybrid strategies. Large platforms often use a combination of these arrangements, creating complex layers of labor stratification. This allows for 24/7 coverage and cost efficiency but further obscures the human labor involved and complicates accountability.

4. Silicon Valley's Digital Factory: Precarious Labor Behind the Plush Facade

It’s factory work, almost. It’s doing the same thing over and over.

The contractor class. Even within the seemingly luxurious environment of Silicon Valley tech campuses, CCM workers often exist as a lower-status contractor class. Hired through third-party firms, they lack the benefits, job security, and status of full-time employees, despite working on-site and performing critical functions.

Rote, repetitive tasks. Despite the digital nature, the work is often described as akin to factory labor – repetitive, queue-based, and driven by productivity metrics (e.g., processing hundreds of tickets per hour). Tools are designed for efficiency, reducing complex judgments to quick clicks.

Limited mobility and recognition. Contractors face a revolving door, with limited-term contracts and little opportunity for permanent employment or advancement within the tech company. Their expertise and insights, gained from direct exposure to platform issues, are often undervalued and ignored by full-time staff and policy makers.

5. The Psychological Toll: Exposure to Humanity's Worst Expressions

I can’t imagine anyone who does [this] job and is able to just walk out at the end of their shift and just be done.

Constant exposure to trauma. Moderators routinely view disturbing content, including violence, pornography, hate speech, and child abuse material. This exposure is not fleeting but a daily reality, leading to potential psychological harm like burnout, desensitization, anxiety, and PTSD.

Difficulty compartmentalizing. Workers struggle to separate their job experiences from their personal lives. Disturbing images or themes can intrude on their thoughts and interactions outside of work, impacting relationships and well-being.

Lack of adequate support. Despite the known risks, psychological support for moderators is often insufficient, inconsistent, or inaccessible (especially for contractors lacking health benefits). Workers often rely on peer support, but even this can be challenging due to job insecurity, turnover, and the difficulty of discussing the content.

6. Brand Protection and Geopolitics: Moderation as Corporate Foreign Policy

There really is no free speech on commercial sites.

Corporate values dictate policy. Content moderation policies are not neutral but reflect the values and priorities of the platforms, primarily brand protection and profitability. Internal rules, often secret, prioritize avoiding PR crises and legal issues over abstract notions of free speech.

Editorial power. Moderators, guided by these policies, wield significant editorial power, deciding what content is seen by millions globally. This includes sensitive material like war zone footage or political advocacy, where decisions can have real-world geopolitical implications.

Uneven application. Policies can be applied unevenly, favoring large "partners" or reflecting the cultural and political biases of the company's headquarters (e.g., U.S.-centric views on nudity or conflict). This highlights how corporate interests shape the information landscape presented to users worldwide.

7. Outsourcing and Postcolonial Legacies: The Global Race to the Bottom

Outsourcing is not just a geographic concept denoted by physical-world spatial characteristics or constraints, but something else: a set of labor processes and configurations, and a designation of an available labor pool, as opposed to simply location.

Beyond geography. Outsourcing CCM is not merely moving jobs but leveraging global labor markets characterized by lower wages, fewer protections, and specific cultural/linguistic skills. This creates a "race to the bottom" where companies seek the cheapest labor pool, driving down wages and conditions globally.

Postcolonial connections. The rise of countries like the Philippines as BPO hubs is linked to historical colonial relationships, particularly with the U.S. Filipino workers' fluency in American English and familiarity with Western culture, a legacy of occupation, is commodified and sold as a valuable asset for moderation work.

Infrastructure and inequality. This global labor flow is supported by specific infrastructure developments (e.g., IT parks, fiber optics) often built through public-private partnerships in the Global South. These developments can exacerbate local inequalities, creating islands of high-tech work within areas lacking basic services, further highlighting the uneven benefits of globalization.

8. The Future of Moderation: AI, Regulation, and the Fight for Visibility

That’s a question we get asked a lot: When is AI going to save us all? We’re a long way from that.

AI limitations. While AI and automation are increasingly used (e.g., for detecting known child abuse material), they are far from replacing human moderators. Complex judgments involving context, nuance, and cultural understanding still require human cognitive skills. AI is also opaque and raises questions about bias and accountability.

Growing pressure for transparency and regulation. Scandals and public concern are forcing platforms to acknowledge CCM and face calls for greater transparency in their policies and practices. Governments, particularly in Europe, are enacting laws demanding faster content removal and holding platforms more accountable.

Worker organizing and advocacy. Despite challenges like global dispersion and NDAs, there are nascent efforts to organize moderators and advocate for better working conditions, pay, and psychological support. Legal challenges are also emerging, seeking to hold companies liable for the harm caused by the work. The fight for visibility and value for this essential labor continues.

Last updated: May 7, 2025