#Algorithmic Bias in AI

[thisisgraeme]

Transform Learning with AI in Education: Volume 1 - Insights for Educators

Kia ora! I’m excited to share Volume 1: AI in Education–Insights for Educators, a practical guide for educators and leaders in Aotearoa. Learn how to navigate AI tools, ensure ethical use, and apply culturally responsive frameworks to support all learners

Why We Wrote a Guide on AI in Education (And Why This is Just the Beginning) Kia ora! Over the past year, I’ve been working with my fellow AI enthusiast, Michael Grawe, on a project that’s been both exciting and challenging: a three-part guide series on Artificial Intelligence (AI) in Education, tailored specifically for educators in Aotearoa New Zealand. We just released Volume 1, and I’m…

[mostlysignssomeportents]

The surprising truth about data-driven dictatorships

Here’s the “dictator’s dilemma”: they want to block their country’s frustrated elites from mobilizing against them, so they censor public communications; but they also want to know what their people truly believe, so they can head off simmering resentments before they boil over into regime-toppling revolutions.

These two strategies are in tension: the more you censor, the less you know about the true feelings of your citizens and the easier it will be to miss serious problems until they spill over into the streets (think: the fall of the Berlin Wall or Tunisia before the Arab Spring). Dictators try to square this circle with things like private opinion polling or petition systems, but these capture a small slice of the potentially destabiziling moods circulating in the body politic.

Enter AI: back in 2018, Yuval Harari proposed that AI would supercharge dictatorships by mining and summarizing the public mood — as captured on social media — allowing dictators to tack into serious discontent and diffuse it before it erupted into unequenchable wildfire:

https://www.theatlantic.com/magazine/archive/2018/10/yuval-noah-harari-technology-tyranny/568330/

Harari wrote that “the desire to concentrate all information and power in one place may become [dictators] decisive advantage in the 21st century.” But other political scientists sharply disagreed. Last year, Henry Farrell, Jeremy Wallace and Abraham Newman published a thoroughgoing rebuttal to Harari in Foreign Affairs:

https://www.foreignaffairs.com/world/spirals-delusion-artificial-intelligence-decision-making

They argued that — like everyone who gets excited about AI, only to have their hopes dashed — dictators seeking to use AI to understand the public mood would run into serious training data bias problems. After all, people living under dictatorships know that spouting off about their discontent and desire for change is a risky business, so they will self-censor on social media. That’s true even if a person isn’t afraid of retaliation: if you know that using certain words or phrases in a post will get it autoblocked by a censorbot, what’s the point of trying to use those words?

The phrase “Garbage In, Garbage Out” dates back to 1957. That’s how long we’ve known that a computer that operates on bad data will barf up bad conclusions. But this is a very inconvenient truth for AI weirdos: having given up on manually assembling training data based on careful human judgment with multiple review steps, the AI industry “pivoted” to mass ingestion of scraped data from the whole internet.

But adding more unreliable data to an unreliable dataset doesn’t improve its reliability. GIGO is the iron law of computing, and you can’t repeal it by shoveling more garbage into the top of the training funnel:

https://memex.craphound.com/2018/05/29/garbage-in-garbage-out-machine-learning-has-not-repealed-the-iron-law-of-computer-science/

When it comes to “AI” that’s used for decision support — that is, when an algorithm tells humans what to do and they do it — then you get something worse than Garbage In, Garbage Out — you get Garbage In, Garbage Out, Garbage Back In Again. That’s when the AI spits out something wrong, and then another AI sucks up that wrong conclusion and uses it to generate more conclusions.

To see this in action, consider the deeply flawed predictive policing systems that cities around the world rely on. These systems suck up crime data from the cops, then predict where crime is going to be, and send cops to those “hotspots” to do things like throw Black kids up against a wall and make them turn out their pockets, or pull over drivers and search their cars after pretending to have smelled cannabis.

The problem here is that “crime the police detected” isn’t the same as “crime.” You only find crime where you look for it. For example, there are far more incidents of domestic abuse reported in apartment buildings than in fully detached homes. That’s not because apartment dwellers are more likely to be wife-beaters: it’s because domestic abuse is most often reported by a neighbor who hears it through the walls.

So if your cops practice racially biased policing (I know, this is hard to imagine, but stay with me /s), then the crime they detect will already be a function of bias. If you only ever throw Black kids up against a wall and turn out their pockets, then every knife and dime-bag you find in someone’s pockets will come from some Black kid the cops decided to harass.

That’s life without AI. But now let’s throw in predictive policing: feed your “knives found in pockets” data to an algorithm and ask it to predict where there are more knives in pockets, and it will send you back to that Black neighborhood and tell you do throw even more Black kids up against a wall and search their pockets. The more you do this, the more knives you’ll find, and the more you’ll go back and do it again.

This is what Patrick Ball from the Human Rights Data Analysis Group calls “empiricism washing”: take a biased procedure and feed it to an algorithm, and then you get to go and do more biased procedures, and whenever anyone accuses you of bias, you can insist that you’re just following an empirical conclusion of a neutral algorithm, because “math can’t be racist.”

HRDAG has done excellent work on this, finding a natural experiment that makes the problem of GIGOGBI crystal clear. The National Survey On Drug Use and Health produces the gold standard snapshot of drug use in America. Kristian Lum and William Isaac took Oakland’s drug arrest data from 2010 and asked Predpol, a leading predictive policing product, to predict where Oakland’s 2011 drug use would take place.

[Image ID: (a) Number of drug arrests made by Oakland police department, 2010. (1) West Oakland, (2) International Boulevard. (b) Estimated number of drug users, based on 2011 National Survey on Drug Use and Health]

Then, they compared those predictions to the outcomes of the 2011 survey, which shows where actual drug use took place. The two maps couldn’t be more different:

https://rss.onlinelibrary.wiley.com/doi/full/10.1111/j.1740-9713.2016.00960.x

Predpol told cops to go and look for drug use in a predominantly Black, working class neighborhood. Meanwhile the NSDUH survey showed the actual drug use took place all over Oakland, with a higher concentration in the Berkeley-neighboring student neighborhood.

What’s even more vivid is what happens when you simulate running Predpol on the new arrest data that would be generated by cops following its recommendations. If the cops went to that Black neighborhood and found more drugs there and told Predpol about it, the recommendation gets stronger and more confident.

In other words, GIGOGBI is a system for concentrating bias. Even trace amounts of bias in the original training data get refined and magnified when they are output though a decision support system that directs humans to go an act on that output. Algorithms are to bias what centrifuges are to radioactive ore: a way to turn minute amounts of bias into pluripotent, indestructible toxic waste.

There’s a great name for an AI that’s trained on an AI’s output, courtesy of Jathan Sadowski: “Habsburg AI.”

And that brings me back to the Dictator’s Dilemma. If your citizens are self-censoring in order to avoid retaliation or algorithmic shadowbanning, then the AI you train on their posts in order to find out what they’re really thinking will steer you in the opposite direction, so you make bad policies that make people angrier and destabilize things more.

Or at least, that was Farrell(et al)’s theory. And for many years, that’s where the debate over AI and dictatorship has stalled: theory vs theory. But now, there’s some empirical data on this, thanks to the “The Digital Dictator’s Dilemma,” a new paper from UCSD PhD candidate Eddie Yang:

https://www.eddieyang.net/research/DDD.pdf

Yang figured out a way to test these dueling hypotheses. He got 10 million Chinese social media posts from the start of the pandemic, before companies like Weibo were required to censor certain pandemic-related posts as politically sensitive. Yang treats these posts as a robust snapshot of public opinion: because there was no censorship of pandemic-related chatter, Chinese users were free to post anything they wanted without having to self-censor for fear of retaliation or deletion.

Next, Yang acquired the censorship model used by a real Chinese social media company to decide which posts should be blocked. Using this, he was able to determine which of the posts in the original set would be censored today in China.

That means that Yang knows that the “real” sentiment in the Chinese social media snapshot is, and what Chinese authorities would believe it to be if Chinese users were self-censoring all the posts that would be flagged by censorware today.

From here, Yang was able to play with the knobs, and determine how “preference-falsification” (when users lie about their feelings) and self-censorship would give a dictatorship a misleading view of public sentiment. What he finds is that the more repressive a regime is — the more people are incentivized to falsify or censor their views — the worse the system gets at uncovering the true public mood.

What’s more, adding additional (bad) data to the system doesn’t fix this “missing data” problem. GIGO remains an iron law of computing in this context, too.

But it gets better (or worse, I guess): Yang models a “crisis” scenario in which users stop self-censoring and start articulating their true views (because they’ve run out of fucks to give). This is the most dangerous moment for a dictator, and depending on the dictatorship handles it, they either get another decade or rule, or they wake up with guillotines on their lawns.

But “crisis” is where AI performs the worst. Trained on the “status quo” data where users are continuously self-censoring and preference-falsifying, AI has no clue how to handle the unvarnished truth. Both its recommendations about what to censor and its summaries of public sentiment are the least accurate when crisis erupts.

But here’s an interesting wrinkle: Yang scraped a bunch of Chinese users’ posts from Twitter — which the Chinese government doesn’t get to censor (yet) or spy on (yet) — and fed them to the model. He hypothesized that when Chinese users post to American social media, they don’t self-censor or preference-falsify, so this data should help the model improve its accuracy.

He was right — the model got significantly better once it ingested data from Twitter than when it was working solely from Weibo posts. And Yang notes that dictatorships all over the world are widely understood to be scraping western/northern social media.

But even though Twitter data improved the model’s accuracy, it was still wildly inaccurate, compared to the same model trained on a full set of un-self-censored, un-falsified data. GIGO is not an option, it’s the law (of computing).

Writing about the study on Crooked Timber, Farrell notes that as the world fills up with “garbage and noise” (he invokes Philip K Dick’s delighted coinage “gubbish”), “approximately correct knowledge becomes the scarce and valuable resource.”

https://crookedtimber.org/2023/07/25/51610/

This “probably approximately correct knowledge” comes from humans, not LLMs or AI, and so “the social applications of machine learning in non-authoritarian societies are just as parasitic on these forms of human knowledge production as authoritarian governments.”

The Clarion Science Fiction and Fantasy Writers’ Workshop summer fundraiser is almost over! I am an alum, instructor and volunteer board member for this nonprofit workshop whose alums include Octavia Butler, Kim Stanley Robinson, Bruce Sterling, Nalo Hopkinson, Kameron Hurley, Nnedi Okorafor, Lucius Shepard, and Ted Chiang! Your donations will help us subsidize tuition for students, making Clarion — and sf/f — more accessible for all kinds of writers.

Libro.fm is the indie-bookstore-friendly, DRM-free audiobook alternative to Audible, the Amazon-owned monopolist that locks every book you buy to Amazon forever. When you buy a book on Libro, they share some of the purchase price with a local indie bookstore of your choosing (Libro is the best partner I have in selling my own DRM-free audiobooks!). As of today, Libro is even better, because it’s available in five new territories and currencies: Canada, the UK, the EU, Australia and New Zealand!

[Image ID: An altered image of the Nuremberg rally, with ranked lines of soldiers facing a towering figure in a many-ribboned soldier's coat. He wears a high-peaked cap with a microchip in place of insignia. His head has been replaced with the menacing red eye of HAL9000 from Stanley Kubrick's '2001: A Space Odyssey.' The sky behind him is filled with a 'code waterfall' from 'The Matrix.']

Image: Cryteria (modified) https://commons.wikimedia.org/wiki/File:HAL9000.svg

CC BY 3.0 https://creativecommons.org/licenses/by/3.0/deed.en

 — 

Raimond Spekking (modified) https://commons.wikimedia.org/wiki/File:Acer_Extensa_5220_-_Columbia_MB_06236-1N_-_Intel_Celeron_M_530_-_SLA2G_-_in_Socket_479-5029.jpg

CC BY-SA 4.0 https://creativecommons.org/licenses/by-sa/4.0/deed.en

 — 

Russian Airborne Troops (modified) https://commons.wikimedia.org/wiki/File:Vladislav_Achalov_at_the_Airborne_Troops_Day_in_Moscow_%E2%80%93_August_2,_2008.jpg

“Soldiers of Russia” Cultural Center (modified) https://commons.wikimedia.org/wiki/File:Col._Leonid_Khabarov_in_an_everyday_service_uniform.JPG

CC BY-SA 3.0 https://creativecommons.org/licenses/by-sa/3.0/deed.en

[tigrikorn]

PSA: Product Specifications

[critical-skeptic]

The Illusion of Complexity: Binary Exploitation in Engagement-Driven Algorithms

Abstract:

This paper examines how modern engagement algorithms employed by major tech platforms (e.g., Google, Meta, TikTok, and formerly Twitter/X) exploit predictable human cognitive patterns through simplified binary interactions. The prevailing perception that these systems rely on sophisticated personalization models is challenged; instead, it is proposed that such algorithms rely on statistical generalizations, perceptual manipulation, and engineered emotional reactions to maintain continuous user engagement. The illusion of depth is a byproduct of probabilistic brute force, not advanced understanding.

1. Introduction

Contemporary discourse often attributes high levels of sophistication and intelligence to the recommendation and engagement algorithms employed by dominant tech companies. Users report instances of eerie accuracy or emotionally resonant suggestions, fueling the belief that these systems understand them deeply. However, closer inspection reveals a more efficient and cynical design principle: engagement maximization through binary funneling.

2. Binary Funneling and Predictive Exploitation

At the core of these algorithms lies a reductive model: categorize user reactions as either positive (approval, enjoyment, validation) or negative (disgust, anger, outrage). This binary schema simplifies personalization into a feedback loop in which any user response serves to reinforce algorithmic certainty. There is no need for genuine nuance or contextual understanding; rather, content is optimized to provoke any reaction that sustains user attention.

Once a user engages with content —whether through liking, commenting, pausing, or rage-watching— the system deploys a cluster of categorically similar material. This recurrence fosters two dominant psychological outcomes:

If the user enjoys the content, they may perceive the algorithm as insightful or “smart,” attributing agency or personalization where none exists.

If the user dislikes the content, they may continue engaging in a doomscroll or outrage spiral, reinforcing the same cycle through negative affect.

In both scenarios, engagement is preserved; thus, profit is ensured.

3. The Illusion of Uniqueness

A critical mechanism in this system is the exploitation of the human tendency to overestimate personal uniqueness. Drawing on techniques long employed by illusionists, scammers, and cold readers, platforms capitalize on common patterns of thought and behavior that are statistically widespread but perceived as rare by individuals.

Examples include:

Posing prompts or content cues that seem personalized but are statistically predictable (e.g., "think of a number between 1 and 50 with two odd digits” → most select 37).

Triggering cognitive biases such as the availability heuristic and frequency illusion, which make repeated or familiar concepts appear newly significant.

This creates a reinforcing illusion: the user feels “understood” because the system has merely guessed correctly within a narrow set of likely options. The emotional resonance of the result further conceals the crude probabilistic engine behind it.

4. Emotional Engagement as Systemic Currency

The underlying goal is not understanding, but reaction. These systems optimize for time-on-platform, not user well-being or cognitive autonomy. Anger, sadness, tribal validation, fear, and parasocial attachment are all equally useful inputs. Through this lens, the algorithm is less an intelligent system and more an industrialized Skinner box: an operant conditioning engine powered by data extraction.

By removing the need for interpretive complexity and relying instead on scalable, binary psychological manipulation, companies minimize operational costs while maximizing monetizable engagement.

5. Black-Box Mythology and Cognitive Deference

Compounding this problem is the opacity of these systems. The “black-box” nature of proprietary algorithms fosters a mythos of sophistication. Users, unaware of the relatively simple statistical methods in use, ascribe higher-order reasoning or consciousness to systems that function through brute-force pattern amplification.

This deference becomes part of the trap: once convinced the algorithm “knows them,” users are less likely to question its manipulations and more likely to conform to its outputs, completing the feedback circuit.

6. Conclusion

The supposed sophistication of engagement algorithms is a carefully sustained illusion. By funneling user behavior into binary categories and exploiting universally predictable psychological responses, platforms maintain the appearance of intelligent personalization while operating through reductive, low-cost mechanisms. Human cognition —biased toward pattern recognition and overestimation of self-uniqueness— completes the illusion without external effort. The result is a scalable system of emotional manipulation that masquerades as individualized insight.

In essence, the algorithm does not understand the user; it understands that the user wants to be understood, and it weaponizes that desire for profit.

[jcmarchi]

How Does AI Use Impact Critical Thinking?

New Post has been published on https://thedigitalinsider.com/how-does-ai-use-impact-critical-thinking/

How Does AI Use Impact Critical Thinking?

Artificial intelligence (AI) can process hundreds of documents in seconds, identify imperceptible patterns in vast datasets and provide in-depth answers to virtually any question. It has the potential to solve common problems, increase efficiency across multiple industries and even free up time for individuals to spend with their loved ones by delegating repetitive tasks to machines.    

However, critical thinking requires time and practice to develop properly. The more people rely on automated technology, the faster their metacognitive skills may decline. What are the consequences of relying on AI for critical thinking?

Study Finds AI Degrades Users’ Critical Thinking 

The concern that AI will degrade users’ metacognitive skills is no longer hypothetical. Several studies suggest it diminishes people’s capacity to think critically, impacting their ability to question information, make judgments, analyze data or form counterarguments. 

A 2025 Microsoft study surveyed 319 knowledge workers on 936 instances of AI use to determine how they perceive their critical thinking ability when using generative technology. Survey respondents reported decreased effort when using AI technology compared to relying on their own minds. Microsoft reported that in the majority of instances, the respondents felt that they used “much less effort” or “less effort” when using generative AI.  

Knowledge, comprehension, analysis, synthesis and evaluation were all adversely affected by AI use. Although a fraction of respondents reported using some or much more effort, an overwhelming majority reported that tasks became easier and required less work. 

If AI’s purpose is to streamline tasks, is there any harm in letting it do its job? It is a slippery slope. Many algorithms cannot think critically, reason or understand context. They are often prone to hallucinations and bias. Users who are unaware of the risks of relying on AI may contribute to skewed, inaccurate results. 

How AI Adversely Affects Critical Thinking Skills

Overreliance on AI can diminish an individual’s ability to independently solve problems and think critically. Say someone is taking a test when they run into a complex question. Instead of taking the time to consider it, they plug it into a generative model and insert the algorithm’s response into the answer field. 

In this scenario, the test-taker learned nothing. They didn’t improve their research skills or analytical abilities. If they pass the test, they advance to the next chapter. What if they were to do this for everything their teachers assign? They could graduate from high school or even college without refining fundamental cognitive abilities. 

This outcome is bleak. However, students might not feel any immediate adverse effects. If their use of language models is rewarded with better test scores, they may lose their motivation to think critically altogether. Why should they bother justifying their arguments or evaluating others’ claims when it is easier to rely on AI? 

The Impact of AI Use on Critical Thinking Skills 

An advanced algorithm can automatically aggregate and analyze large datasets, streamlining problem-solving and task execution. Since its speed and accuracy often outperform humans, users are usually inclined to believe it is better than them at these tasks. When it presents them with answers and insights, they take that output at face value. Unquestioning acceptance of a generative model’s output leads to difficulty distinguishing between facts and falsehoods. Algorithms are trained to predict the next word in a string of words. No matter how good they get at that task, they aren’t really reasoning. Even if a machine makes a mistake, it won’t be able to fix it without context and memory, both of which it lacks.

The more users accept an algorithm’s answer as fact, the more their evaluation and judgment skew. Algorithmic models often struggle with overfitting. When they fit too closely to the information in their training dataset, their accuracy can plummet when they are presented with new information for analysis. 

Populations Most Affected by Overreliance on AI 

Generally, overreliance on generative technology can negatively impact humans’ ability to think critically. However, low confidence in AI-generated output is related to increased critical thinking ability, so strategic users may be able to use AI without harming these skills. 

In 2023, around 27% of adults told the Pew Research Center they use AI technology multiple times a day. Some of the individuals in this population may retain their critical thinking skills if they have a healthy distrust of machine learning tools. The data must focus on populations with disproportionately high AI use and be more granular to determine the true impact of machine learning on critical thinking. 

Critical thinking often isn’t taught until high school or college. It can be cultivated during early childhood development, but it typically takes years to grasp. For this reason, deploying generative technology in schools is particularly risky — even though it is common. 

Today, most students use generative models. One study revealed that 90% have used ChatGPT to complete homework. This widespread use isn’t limited to high schools. About 75% of college students say they would continue using generative technology even if their professors disallowed it. Middle schoolers, teenagers and young adults are at an age where developing critical thinking is crucial. Missing this window could cause problems. 

The Implications of Decreased Critical Thinking

Already, 60% of educators use AI in the classroom. If this trend continues, it may become a standard part of education. What happens when students begin to trust these tools more than themselves? As their critical thinking capabilities diminish, they may become increasingly susceptible to misinformation and manipulation. The effectiveness of scams, phishing and social engineering attacks could increase.  

An AI-reliant generation may have to compete with automation technology in the workforce. Soft skills like problem-solving, judgment and communication are important for many careers. Lacking these skills or relying on generative tools to get good grades may make finding a job challenging. 

Innovation and adaptation go hand in hand with decision-making. Knowing how to objectively reason without the use of AI is critical when confronted with high-stakes or unexpected situations. Leaning into assumptions and inaccurate data could adversely affect an individual’s personal or professional life.

Critical thinking is part of processing and analyzing complex — and even conflicting — information. A community made up of critical thinkers can counter extreme or biased viewpoints by carefully considering different perspectives and values. 

AI Users Must Carefully Evaluate Algorithms’ Output 

Generative models are tools, so whether their impact is positive or negative depends on their users and developers. So many variables exist. Whether you are an AI developer or user, strategically designing and interacting with generative technologies is an important part of ensuring they pave the way for societal advancements rather than hindering critical cognition.

[slowandsweet]

Inspired by the landmark 1968 exhibition Cybernetic Serendipity, the first-ever international event in the UK dedicated to arts and emerging technologies, the event, titled Cybernetic Serendipity: Towards AI, will look back to look forwards, exploring the transformative potential of machine learning across the creative industries, from algorithms and text-to-image chatbots like ChatGPT to building virtual worlds and using AI to detect bias and disinformation. From AI chatbots to virtual companions and the never ending wave of deepfakes on our screens, artificial intelligence is an unavoidable part of culture nowadays. via Dazed and confused

[skannar]

I love good Audiobooks on new tech.

[codecrafted]

Data Science Ethics: Issues and Strategies Biased algorithms can cause serious societal harm. Learn how algorithmic fairness, diverse datasets, and transparent modeling can prevent discrimination in AI. This article also explains how data scientists can apply ethical principles using real case studies and practical frameworks taught in online data science courses in India.

[jtwb768-babbles]

Can Artificial Intelligence Be Queer? Exploring Identity, Design, and Digital Bias

Beyond Binary Systems—Queering the Machine Artificial intelligence, once confined to the pages of speculative fiction and the chalkboards of theoretical mathematics, has evolved into one of the most influential forces shaping 21st-century society. From chatbots to facial recognition, from content curation to predictive policing, AI systems mediate the way we engage with reality. But as these…

[mostlysignssomeportents]

Hypothetical AI election disinformation risks vs real AI harms

I'm on tour with my new novel The Bezzle! Catch me TONIGHT (Feb 27) in Portland at Powell's. Then, onto Phoenix (Changing Hands, Feb 29), Tucson (Mar 9-12), and more!

You can barely turn around these days without encountering a think-piece warning of the impending risk of AI disinformation in the coming elections. But a recent episode of This Machine Kills podcast reminds us that these are hypothetical risks, and there is no shortage of real AI harms:

https://soundcloud.com/thismachinekillspod/311-selling-pickaxes-for-the-ai-gold-rush

The algorithmic decision-making systems that increasingly run the back-ends to our lives are really, truly very bad at doing their jobs, and worse, these systems constitute a form of "empiricism-washing": if the computer says it's true, it must be true. There's no such thing as racist math, you SJW snowflake!

https://slate.com/news-and-politics/2019/02/aoc-algorithms-racist-bias.html

Nearly 1,000 British postmasters were wrongly convicted of fraud by Horizon, the faulty AI fraud-hunting system that Fujitsu provided to the Royal Mail. They had their lives ruined by this faulty AI, many went to prison, and at least four of the AI's victims killed themselves:

https://en.wikipedia.org/wiki/British_Post_Office_scandal

Tenants across America have seen their rents skyrocket thanks to Realpage's landlord price-fixing algorithm, which deployed the time-honored defense: "It's not a crime if we commit it with an app":

https://www.propublica.org/article/doj-backs-tenants-price-fixing-case-big-landlords-real-estate-tech

Housing, you'll recall, is pretty foundational in the human hierarchy of needs. Losing your home – or being forced to choose between paying rent or buying groceries or gas for your car or clothes for your kid – is a non-hypothetical, widespread, urgent problem that can be traced straight to AI.

Then there's predictive policing: cities across America and the world have bought systems that purport to tell the cops where to look for crime. Of course, these systems are trained on policing data from forces that are seeking to correct racial bias in their practices by using an algorithm to create "fairness." You feed this algorithm a data-set of where the police had detected crime in previous years, and it predicts where you'll find crime in the years to come.

But you only find crime where you look for it. If the cops only ever stop-and-frisk Black and brown kids, or pull over Black and brown drivers, then every knife, baggie or gun they find in someone's trunk or pockets will be found in a Black or brown person's trunk or pocket. A predictive policing algorithm will naively ingest this data and confidently assert that future crimes can be foiled by looking for more Black and brown people and searching them and pulling them over.

Obviously, this is bad for Black and brown people in low-income neighborhoods, whose baseline risk of an encounter with a cop turning violent or even lethal. But it's also bad for affluent people in affluent neighborhoods – because they are underpoliced as a result of these algorithmic biases. For example, domestic abuse that occurs in full detached single-family homes is systematically underrepresented in crime data, because the majority of domestic abuse calls originate with neighbors who can hear the abuse take place through a shared wall.

But the majority of algorithmic harms are inflicted on poor, racialized and/or working class people. Even if you escape a predictive policing algorithm, a facial recognition algorithm may wrongly accuse you of a crime, and even if you were far away from the site of the crime, the cops will still arrest you, because computers don't lie:

https://www.cbsnews.com/sacramento/news/texas-macys-sunglass-hut-facial-recognition-software-wrongful-arrest-sacramento-alibi/

Trying to get a low-waged service job? Be prepared for endless, nonsensical AI "personality tests" that make Scientology look like NASA:

https://futurism.com/mandatory-ai-hiring-tests

Service workers' schedules are at the mercy of shift-allocation algorithms that assign them hours that ensure that they fall just short of qualifying for health and other benefits. These algorithms push workers into "clopening" – where you close the store after midnight and then open it again the next morning before 5AM. And if you try to unionize, another algorithm – that spies on you and your fellow workers' social media activity – targets you for reprisals and your store for closure.

If you're driving an Amazon delivery van, algorithm watches your eyeballs and tells your boss that you're a bad driver if it doesn't like what it sees. If you're working in an Amazon warehouse, an algorithm decides if you've taken too many pee-breaks and automatically dings you:

https://pluralistic.net/2022/04/17/revenge-of-the-chickenized-reverse-centaurs/

If this disgusts you and you're hoping to use your ballot to elect lawmakers who will take up your cause, an algorithm stands in your way again. "AI" tools for purging voter rolls are especially harmful to racialized people – for example, they assume that two "Juan Gomez"es with a shared birthday in two different states must be the same person and remove one or both from the voter rolls:

https://www.cbsnews.com/news/eligible-voters-swept-up-conservative-activists-purge-voter-rolls/

Hoping to get a solid education, the sort that will keep you out of AI-supervised, precarious, low-waged work? Sorry, kiddo: the ed-tech system is riddled with algorithms. There's the grifty "remote invigilation" industry that watches you take tests via webcam and accuses you of cheating if your facial expressions fail its high-tech phrenology standards:

https://pluralistic.net/2022/02/16/unauthorized-paper/#cheating-anticheat

All of these are non-hypothetical, real risks from AI. The AI industry has proven itself incredibly adept at deflecting interest from real harms to hypothetical ones, like the "risk" that the spicy autocomplete will become conscious and take over the world in order to convert us all to paperclips:

https://pluralistic.net/2023/11/27/10-types-of-people/#taking-up-a-lot-of-space

Whenever you hear AI bosses talking about how seriously they're taking a hypothetical risk, that's the moment when you should check in on whether they're doing anything about all these longstanding, real risks. And even as AI bosses promise to fight hypothetical election disinformation, they continue to downplay or ignore the non-hypothetical, here-and-now harms of AI.

There's something unseemly – and even perverse – about worrying so much about AI and election disinformation. It plays into the narrative that kicked off in earnest in 2016, that the reason the electorate votes for manifestly unqualified candidates who run on a platform of bald-faced lies is that they are gullible and easily led astray.

But there's another explanation: the reason people accept conspiratorial accounts of how our institutions are run is because the institutions that are supposed to be defending us are corrupt and captured by actual conspiracies:

https://memex.craphound.com/2019/09/21/republic-of-lies-the-rise-of-conspiratorial-thinking-and-the-actual-conspiracies-that-fuel-it/

The party line on conspiratorial accounts is that these institutions are good, actually. Think of the rebuttal offered to anti-vaxxers who claimed that pharma giants were run by murderous sociopath billionaires who were in league with their regulators to kill us for a buck: "no, I think you'll find pharma companies are great and superbly regulated":

https://pluralistic.net/2023/09/05/not-that-naomi/#if-the-naomi-be-klein-youre-doing-just-fine

Institutions are profoundly important to a high-tech society. No one is capable of assessing all the life-or-death choices we make every day, from whether to trust the firmware in your car's anti-lock brakes, the alloys used in the structural members of your home, or the food-safety standards for the meal you're about to eat. We must rely on well-regulated experts to make these calls for us, and when the institutions fail us, we are thrown into a state of epistemological chaos. We must make decisions about whether to trust these technological systems, but we can't make informed choices because the one thing we're sure of is that our institutions aren't trustworthy.

Ironically, the long list of AI harms that we live with every day are the most important contributor to disinformation campaigns. It's these harms that provide the evidence for belief in conspiratorial accounts of the world, because each one is proof that the system can't be trusted. The election disinformation discourse focuses on the lies told – and not why those lies are credible.

That's because the subtext of election disinformation concerns is usually that the electorate is credulous, fools waiting to be suckered in. By refusing to contemplate the institutional failures that sit upstream of conspiracism, we can smugly locate the blame with the peddlers of lies and assume the mantle of paternalistic protectors of the easily gulled electorate.

But the group of people who are demonstrably being tricked by AI is the people who buy the horrifically flawed AI-based algorithmic systems and put them into use despite their manifest failures.

As I've written many times, "we're nowhere near a place where bots can steal your job, but we're certainly at the point where your boss can be suckered into firing you and replacing you with a bot that fails at doing your job"

https://pluralistic.net/2024/01/15/passive-income-brainworms/#four-hour-work-week

The most visible victims of AI disinformation are the people who are putting AI in charge of the life-chances of millions of the rest of us. Tackle that AI disinformation and its harms, and we'll make conspiratorial claims about our institutions being corrupt far less credible.

If you'd like an essay-formatted version of this post to read or share, here's a link to it on pluralistic.net, my surveillance-free, ad-free, tracker-free blog:

https://pluralistic.net/2024/02/27/ai-conspiracies/#epistemological-collapse

Image: Cryteria (modified) https://commons.wikimedia.org/wiki/File:HAL9000.svg

CC BY 3.0 https://creativecommons.org/licenses/by/3.0/deed.en

[bob3160]

Emoji Therapy & Tech Truths - Scammed -Tracked & Algorithmically Attacked

[ailifehacks]

Predictive Policing: AI-Driven Crime Forecasting and Its Ethical Dilemmas

Discover how AI-driven crime forecasting in predictive policing is reshaping law enforcement. Explore its ethical dilemmas, including bias, privacy concerns, Predictive policing refers to the use of AI algorithms to forecast potential criminal activities based on historical data. While this approach aims to enhance law enforcement efficiency, it introduces significant ethical concerns. What Is…

[firstoccupier]

When Big Tech Deletes History:

The Crisis of AI-Driven Censorship and Its Political Roots The recent removal of a 15-year-old YouTube channel—one that documented decades of protest, labor struggles, and political activism—raises urgent questions about the unchecked power of Big Tech and the dangers of AI-driven censorship. Google’s official reason cited “severe or repeated violations” of spam and deceptive practices…

[gedzolini]

Hiring Algorithmic Bias: Why AI Recruiting Tools Need to Be Regulated Just Like Human Recruiters

Artificial intelligence is a barrier for millions of job searchers throughout the world. Ironically, AI tends to inherit and magnify human prejudices, despite its promise to make hiring faster and fairer. Companies like Pymetrics, HireVue, and Amazon use it because of this. It may be harder to spot and stop systematic prejudice than bias from human recruiters if these automated hiring technologies are allowed to operate unchecked. The crucial question that this raises is whether automated hiring algorithms should be governed by the same rules as human decision-makers. As more and more evidence points to, the answer must be yes.

AI's Rise in Hiring

The use of AI in hiring is no longer futuristic, it is mainstream. According to a site Resume Genius around 48% of hiring managers in the U.S. use AI to support HR activities, and adoption is expected to grow. These systems sort through resumes, rank applicants, analyze video interviews, and even predict a candidate’s future job performance based on behavior or speech patterns. The objective is to lower expenses, reduce bias, and decrease human mistakes. But AI can only be as good as the data it is taught on, and technology can reinforce historical injustices if the data reflects them. One of the main examples is Amazon’s hiring tool. They created a hiring tool in 2014 that assigned résumé scores to applicants. The goal was to more effectively discover elite personnel by automating the selection process. By 2015, however, programmers had identified a serious weakness: the AI was discriminatory against women. Why? because over a ten-year period, it had been trained on resumes submitted to Amazon, the majority of which were from men. The algorithm consequently started to penalize resumes that mentioned attendance at all-female universities or contained phrases like "women's chess club captain." Bias persisted in the system despite efforts to "neutralize" gendered words. In 2017, Amazon discreetly abandoned the project. This exemplifies a warning about the societal repercussions of using obscure tools to automate important life opportunities, not just merely a technical error. So, where does the law stand?

Legal and Ethical Views on AI Bias

The EEOC (Equal Employment Opportunity Commission) of the United States has recognized the rising issue. To guarantee that algorithmic employment methods meet human rights legislation, the EEOC and the Department of Justice established a Joint Initiative on Algorithmic Fairness in May 2022. Technical guidance on the application of Title VII of the Civil Rights Act, which forbids employment discrimination, to algorithmic tools was subsequently released.

The EEOC’s plan includes:

Establishing an internal working group to coordinate efforts across the agency.

Hosting listening sessions with employers, vendors, researchers, and civil rights groups to understand the real-world impact of hiring technologies.

Gathering data on how algorithmic tools are being adopted, designed, and deployed in the workplace.

Identifying promising practices for ensuring fairness in AI systems.

Issuing technical assistance to help employers navigate the legal and ethical use of AI in hiring decisions.

But there's a problem. Most laws were written with human decision-makers in mind. Regulators are still catching up with technologies that evolve faster than legislation. Some states, like Illinois and New York, have passed laws requiring bias audits or transparency in hiring tools, but these are exceptions, not the rule. The vast majority of hiring algorithms still operate in a regulatory gray zone. This regulatory gap becomes especially troubling when AI systems replicate the very biases that human decision-makers are legally prohibited from acting on.If an HR manager refused to interview a woman simply because she led a women’s tech club, it would be a clear violation of employment law. Why should an AI system that does the same get a pass? Here are some reasons AI hiring tools must face the same scrutiny as humans:

Lack of Transparency

AI systems are often “black boxes”, their decision-making logic is hidden, even from the companies that deploy them. Job applicants frequently don’t know an algorithm was involved, let alone how to contest its decisions.

Scale of Harm

A biased recruiter might discriminate against a few candidates. A biased algorithm can reject thousands in seconds. The scalability of harm is enormous and invisible unless proactively audited.

Accountability Gap

When things go wrong, who is responsible? The vendor that built the tool? The employer who used it? The engineer who trained it? Current frameworks rarely provide clear answers.

Public Trust

Surveys suggest that public confidence in AI hiring is low. A 2021 Pew Research study found that a majority of Americans oppose the use of AI in hiring decisions, citing fairness and accountability as top concerns.

Relying solely on voluntary best practices is no longer sufficient due to the size, opacity, and influence of AI hiring tools. Strong regulatory frameworks must be in place to guarantee that these technologies be created and used responsibly if they are to gain the public's trust and function within moral and legal bounds.

What Regulation Should Look Like

Significant security must be implemented to guarantee AI promotes justice rather than harming it. These regulations are:

Mandatory bias audits by independent third parties.

Algorithmic transparency, including disclosures to applicants when AI is used.

Explainability requirements to help users understand and contest decisions.

Data diversity mandates, ensuring training datasets reflect real-world demographics.

Clear legal accountability for companies deploying biased systems.

Regulators in Europe are already using this approach. The proposed AI Act from the EU labels hiring tools as "high-risk" and places strict constraints on their use, such as frequent risk assessments and human supervision.

Improving AI rather than abandoning it is the answer. Promising attempts are being made to create "fairness-aware" algorithms that strike a compromise between social equality and prediction accuracy. Businesses such as Pymetrics have pledged to mitigate bias and conduct third-party audits. Developers can access resources to assess and reduce prejudice through open-source toolkits such as Microsoft's Fairlearn and IBM's AI Fairness 360. A Python library called Fairlearn aids with assessing and resolving fairness concerns in machine learning models. It offers algorithms and visualization dashboards that may reduce the differences in predicted performance between various demographic groupings. With ten bias prevention algorithms and more than 70 fairness criteria, AI Fairness 360 (AIF360) is a complete toolkit. It is very adaptable for pipelines in the real world because it allows pre-, in-, and post-processing procedures. Businesses can be proactive in detecting and resolving bias before it affects job prospects by integrating such technologies into the development pipeline. These resources show that fairness is a achievable objective rather than merely an ideal.

Conclusion

Fairness, accountability, and public trust are all at considerable risk from AI's unrestrained use as it continues to influence hiring practices. With the size and opacity of these tools, algorithmic systems must be held to the same norms that shield job seekers from human prejudice, if not more rigorously. The goal of regulating AI in employment is to prevent technological advancement from compromising equal opportunity, not to hinder innovation. We can create AI systems that enhance rather than undermine a just labor market if we have the appropriate regulations, audits, and resources. Whether the decision-maker is a human or a machine, fair hiring should never be left up to chance.

[jcmarchi]

Are AI-Powered Traffic Cameras Watching You Drive?

New Post has been published on https://thedigitalinsider.com/are-ai-powered-traffic-cameras-watching-you-drive/

Are AI-Powered Traffic Cameras Watching You Drive?

Artificial intelligence (AI) is everywhere today. While that’s an exciting prospect to some, it’s an uncomfortable thought for others. Applications like AI-powered traffic cameras are particularly controversial. As their name suggests, they analyze footage of vehicles on the road with machine vision.

They’re typically a law enforcement measure — police may use them to catch distracted drivers or other violations, like a car with no passengers using a carpool lane. However, they can also simply monitor traffic patterns to inform broader smart city operations. In all cases, though, they raise possibilities and questions about ethics in equal measure.

How Common Are AI Traffic Cameras Today?

While the idea of an AI-powered traffic camera is still relatively new, they’re already in use in several places. Nearly half of U.K. police forces have implemented them to enforce seatbelt and texting-while-driving regulations. U.S. law enforcement is starting to follow suit, with North Carolina catching nine times as many phone violations after installing AI cameras.

Fixed cameras aren’t the only use case in action today, either. Some transportation departments have begun experimenting with machine vision systems inside public vehicles like buses. At least four cities in the U.S. have implemented such a solution to detect cars illegally parked in bus lanes.

With so many local governments using this technology, it’s safe to say it will likely grow in the future. Machine learning will become increasingly reliable over time, and early tests could lead to further adoption if they show meaningful improvements.

Rising smart city investments could also drive further expansion. Governments across the globe are betting hard on this technology. China aims to build 500 smart cities, and India plans to test these technologies in at least 100 cities. As that happens, more drivers may encounter AI cameras on their daily commutes.

Benefits of Using AI in Traffic Cameras

AI traffic cameras are growing for a reason. The innovation offers a few critical advantages for public agencies and private citizens.

Safety Improvements

The most obvious upside to these cameras is they can make roads safer. Distracted driving is dangerous — it led to the deaths of 3,308 people in 2022 alone — but it’s hard to catch. Algorithms can recognize drivers on their phones more easily than highway patrol officers can, helping enforce laws prohibiting these reckless behaviors.

Early signs are promising. The U.K. and U.S. police forces that have started using such cameras have seen massive upticks in tickets given to distracted drivers or those not wearing seatbelts. As law enforcement cracks down on such actions, it’ll incentivize people to drive safer to avoid the penalties.

AI can also work faster than other methods, like red light cameras. Because it automates the analysis and ticketing process, it avoids lengthy manual workflows. As a result, the penalty arrives soon after the violation, which makes it a more effective deterrent than a delayed reaction. Automation also means areas with smaller police forces can still enjoy such benefits.

Streamlined Traffic

AI-powered traffic cameras can minimize congestion on busy roads. The areas using them to catch illegally parked cars are a prime example. Enforcing bus lane regulations ensures public vehicles can stop where they should, avoiding delays or disruptions to traffic in other lanes.

Automating tickets for seatbelt and distracted driving violations has a similar effect. Pulling someone over can disrupt other cars on the road, especially in a busy area. By taking a picture of license plates and sending the driver a bill instead, police departments can ensure safer streets without adding to the chaos of everyday traffic.

Non-law-enforcement cameras could take this advantage further. Machine vision systems throughout a city could recognize congestion and update map services accordingly, rerouting people around busy areas to prevent lengthy delays. Considering how the average U.S. driver spent 42 hours in traffic in 2023, any such improvement is a welcome change.

Downsides of AI Traffic Monitoring

While the benefits of AI traffic cameras are worth noting, they’re not a perfect solution. The technology also carries some substantial potential downsides.

False Positives and Errors

The correctness of AI may raise some concerns. While it tends to be more accurate than people in repetitive, data-heavy tasks, it can still make mistakes. Consequently, removing human oversight from the equation could lead to innocent people receiving fines.

A software bug could cause machine vision algorithms to misidentify images. Cybercriminals could make such instances more likely through data poisoning attacks. While people could likely dispute their tickets and clear their name, it would take a long, difficult process to do so, counteracting some of the technology’s efficiency benefits.

False positives are a related concern. Algorithms can produce high false positive rates, leading to more charges against innocent people, which carries racial implications in many contexts. Because data biases can remain hidden until it’s too late, AI in government applications can exacerbate problems with racial or gender discrimination in the legal system.

Privacy Issues

The biggest controversy around AI-powered traffic cameras is a familiar one — privacy. As more cities install these systems, they record pictures of a larger number of drivers. So much data in one place raises big questions about surveillance and the security of sensitive details like license plate numbers and drivers’ faces.

Many AI camera solutions don’t save images unless they determine it’s an instance of a violation. Even so, their operation would mean the solutions could store hundreds — if not thousands — of images of people on the road. Concerns about government surveillance aside, all that information is a tempting target for cybercriminals.

U.S. government agencies suffered 32,211 cybersecurity incidents in 2023 alone. Cybercriminals are already targeting public organizations and critical infrastructure, so it’s understandable why some people may be concerned that such groups would gather even more data on citizens. A data breach in a single AI camera system could affect many who wouldn’t have otherwise consented to giving away their data.

What the Future Could Hold

Given the controversy, it may take a while for automated traffic cameras to become a global standard. Stories of false positives and concerns over cybersecurity issues may delay some projects. Ultimately, though, that’s a good thing — attention to these challenges will lead to necessary development and regulation to ensure the rollout does more good than harm.

Strict data access policies and cybersecurity monitoring will be crucial to justify widespread adoption. Similarly, government organizations using these tools should verify the development of their machine-learning models to check for and prevent problems like bias. Regulations like the recent EU Artificial Intelligence Act have already provided a legislative precedent for such qualifications.

AI Traffic Cameras Bring Both Promise and Controversy

AI-powered traffic cameras may still be new, but they deserve attention. Both the promises and pitfalls of the technology need greater attention as more governments seek to implement them. Higher awareness of the possibilities and challenges surrounding this innovation can foster safer development for a secure and efficient road network in the future.

[theaspirationsinstitute]

Navigating the complexities of AI and automation demands more than just technical prowess; it requires ethical leadership. 

My latest blog post delves into building an "ethical algorithm" for your organization, addressing bias, the future of work, and maintaining human-centered values in a rapidly evolving technological landscape. Leaders at all levels will find practical strategies and thought-provoking insights. 

Read more and subscribe for ongoing leadership guidance.