AI Failures & Ethical Concerns

Real-World Case Studies

Case Studies

Why AI governance is crucial

  • Case Study 1: Amazon’s AI Hiring Tool (2018) – Gender Bias in Hiring

    What happened? Amazon trained an AI to screen job applicants. The AI penalized resumes that contained words like "women’s" (e.g., “women’s chess club”) because the model was trained on past hiring patterns favoring male candidates (Dastin, 2018).📌 Impact: The AI systematically discriminated against female applicants and was later shut down.📌 Lesson: AI systems inherit biases from historical data, reinforcing discrimination unless corrected.
  • COMPAS Algorithm (2016) – Racial Bias in Criminal Justice
    What happened? The COMPAS AI system used in U.S. courts to predict the likelihood of a criminal reoffending was found to assign higher risk scores to Black defendants compared to white defendants (ProPublica, 2016).📌 Impact: Many defendants were given harsher sentences based on biased AI predictions.📌 Lesson: AI used in law enforcement must be transparent and audited for bias.
  • Clearview AI (2020) – Privacy Violations in Facial Recognition
    What happened? Clearview AI scraped billions of photos from the internet to train facial recognition software—without user consent. The system was sold to law enforcement and private companies.📌 Impact: The company violated GDPR and privacy laws, leading to lawsuits and government bans.📌 Lesson: AI must respect personal privacy rights and follow legal guidelines.
  • The Future of AI Safety & Governance
    🔹 More Explainable AI (XAI): AI developers are working on interpretable AI models that explain their decisions.🔹 Stronger Regulations & AI Audits: Companies using AI for hiring, finance, or law enforcement must conduct fairness audits.🔹 AI & Human Rights Protections: AI ethics research will continue to protect workers, consumers, and vulnerable populations from algorithmic harm.
  • Call to Action: What Can You Do?
    🔹 Advocate for Stronger AI Regulations – Support AI fairness laws and corporate accountability.🔹 Demand Transparency in AI Hiring – Push for mandatory AI bias audits and fairness standards.🔹 Educate Yourself & Others – Stay informed on how AI impacts jobs, finance, and civil rights.Would you like a policy recommendation document or a petition draft to support AI fairness advocacy? 🚀
  • References
    Dastin, J. (2018). "Amazon Scraps Secret AI Recruiting Tool That Showed Bias Against Women." Reuters.ProPublica (2016). "Machine Bias: There’s Software Used Across the Country to Predict Future Criminals. And It’s Biased Against Black Defendants." ProPublica Investigative Report.Wachter, S., & Mittelstadt, B. (2021). "Why Fairness Cannot Be Automated: Bridging the Gap Between EU Non-Discrimination Law and AI." Computer Law & Security Review, 41.Raghavan, M., Barocas, S., Kleinberg, J., & Levy, K. (2020). "Mitigating Bias in Algorithmic Hiring: Evaluating Claims and Practices." Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (FAT).*

AI Safety

Key Risks & Challenges

AI safety focuses on ensuring that AI does not cause harm to people, society, or economies. Key concerns include

  • Bias & Discrimination

    🤖 How it happens: AI models learn from historical data, which can contain human biases. If the training data is biased, the AI system will make biased decisions. 💡 Example: AI hiring systems have been found to discriminate against women, older candidate...
    Learn more
  • Transparency & Explainability ("Black Box AI")
    🤖 How it happens: Many AI systems operate as black boxes, meaning users cannot see how decisions are made.💡 Example: AI-driven credit scoring companies refuse to explain why a customer was denied a loan, making it impossible to challenge unfair decisions...
    Learn more
  • Privacy & Data Protection
    🤖 How it happens: AI hiring tools and facial recognition systems collect personal data without proper consent.💡 Example: The use of AI-powered video interviews raises concerns about biometric data collection and surveillance (Wachter et al., 2021).
    Learn more
  • AI Alignment & Control
    🤖 How it happens: Advanced AI systems could act in ways that are harmful or unpredictable if not properly aligned with human values.💡 Example: Chatbots have been found to generate misinformation, offensive language, or biased content without human overs...
    Learn more