Least influence: Online groups

Change: I gained more influence with siblings and teammates as I got older.

• AIF article (as assigned in class).

Science shows teens need 8-10 hours and early school times make it hard.

• “She’s the type of person who never gives up, even when life gets messy.”

• “I don’t have to rush to become someone — I’m already becoming. Take your time.”

In high school, Maria went through a period where she felt alone even while surrounded by people. She learned to be comfortable with herself and not depend on others to define her worth. Now she grounds herself with small acts of care—like making her bed every morning to start fresh. “I don’t have to rush to become someone,” she told me. “I’m already becoming. Take your time.”

She’s building toward a peaceful life, a warm family, and a future where people—especially children—feel seen and supported.

Artificial intelligence is showing up pretty much everywhere in health care now. There are tools that read X-rays, predict which patients might be readmitted, decide who should get extra care management, and even suggest treatment plans. The idea is that these systems can help doctors work faster, make decisions more consistently, and maybe even reduce costs.

But AI isn’t automatically fair or neutral. It learns from data, and that data comes from real health-care systems that already treat some groups worse than others. If the data is biased, the AI usually ends up biased too. This is called algorithmic bias. It basically means the system makes systematic mistakes that hit certain groups harder like people of color, women, or people with lower incomes.

In medicine, this is a big deal because these tools affect real decisions: who gets referred to a specialist, who gets extra monitoring, and how test results are interpreted. Recent research shows that some medical AI systems work less accurately for people with darker skin tones, for minoritized racial and ethnic groups, or for women. That creates serious ethical and public health issues.

In this project, I look at how AI bias shows up in medicine, go over some real-world examples, and talk about what researchers say we should do to make medical AI more fair.

Existing Scholarship on the Topic

Researchers say algorithmic bias in healthcare AI can show up basically anywhere in the pipeline: when data is collected, when features are picked, when models are trained, and when they’re deployed and monitored.

Nororietal. argue that when datasets are unbalanced or leave out certain groups and when those datasets already reflect human prejudice algorithms end up repeating historical inequalities instead of fixing them.

Zou and Schiebinger explain that biomedical AI often performs worse on under-represented groups because the training data mostly comes from majority populations. So the model fits those majority groups best and doesn’t generalize well to everyone else. Mittermaier et al. review AI models in surgery and other medical areas and find that algorithms can worsen disparities related to race, ethnicity, gender, disability, and socioeconomic status if those issues aren’t carefully considered.

There are also several concrete examples of biased medical AI, like:

• Risk-prediction algorithms: Obermeyer et al. found racial bias in a widely used U.S. risk algorithm. At the same “risk score,” Black patients were actually sicker than white patients, so fewer Black patients got referred to high-risk care programs.

• Dermatology AI: Algorithms trained mostly on lighter skin tones perform worse on images of darker skin until they’re retrained using more diverse datasets.

• Pulse oximeters: These devices, which measure blood oxygen, have overestimated oxygen levels in patients with darker skin for decades. Recent research connects this device bias to broader concerns about biased health data and AI.

• Imaging AI: Deep-learning models for chest X-rays and cardiac imaging show performance gaps across race and sex, often tied to imbalanced training data.

On top of that, some clinical algorithms directly include “race corrections” (for example, in kidney function scores like eGFR or lung function tests). These have been criticized for reinforcing inequities instead of solving them.

To deal with these problems, several frameworks have been proposed. Chin et al. lay out principles for preventing bias in healthcare algorithms and promoting health equity. Abràmoff et al. suggest a “Total Product Lifecycle” approach, which maps where bias can appear and how to monitor it over time. Public-health organizations like the CDC, plus expert groups at Yale and other medical centers, stress transparency, accountability, and equity when using AI in health care.

Overall, the existing research makes it clear that AI bias in healthcare isn’t just a theory. It’s already happening and it’s already affecting patients.

Research Questions

Based on this scholarship, my project focuses on three main questions:

1. How does algorithmic bias arise in AI systems used in healthcare?

2. What real-world examples show biased medical AI harming or disadvantaging specific patient groups?

3. What strategies do researchers and policymakers recommend to detect, reduce, and prevent bias in medical AI?

Methodology

For this project, I didn’t collect any clinical data myself. Instead, I used library and web-based research. The main steps were:

• Database search: I searched PubMed, Google Scholar, and well-known public-health websites using keywords like “AI bias healthcare,” “algorithmic bias medicine,” “racial bias risk prediction algorithm,” “dermatology AI skin tone,” and “pulse oximeter bias.”

• Time frame: I focused on sources from around 2019–2025, since research on AI bias in healthcare really expanded during this period.

• Inclusion criteria:

• Peer-reviewed journal articles

• Reports from major health organizations (e.g., CDC, JAMA, Nature, Science)

• Expert commentary from academic medical centers

• Direct relevance to clinical algorithms, decision-support tools, diagnostic or monitoring devices, or AI-related health policy

• Exclusion criteria:

• Opinion pieces with no references

• Articles about AI bias in non-medical areas (like policing or hiring) unless they clearly connected to medical implications

At first, I gathered around 20 sources, then narrowed these down to about 12–15 that were the most recent, relevant, and methodologically solid.

Analysis

1. How Bias Arises in Medical AI

a. Biased and incomplete data

AI models usually learn from existing health records, images, billing claims, and other similar data. If certain groups are under-represented like people with darker skin or people from low-income neighborhoods the model will fit the majority group best and be less accurate for everyone else.

Historical health data also reflects past discrimination: unequal access to care, uneven insurance coverage, and clinician bias. When algorithms are trained on this data without any adjustment, they basically learn to repeat these patterns. Some authors from Harvard Medical School describe AI as more of a “mirror” of human bias than a neutral replacement for human decision making.

b. Problematic proxy variables

Bias can also come from what the algorithm is told to predict. In the Optum risk-prediction case, the model tried to estimate who needed extra care management by using health-care costs as a stand in for actual illness. But in the U.S., less money has historically been spent on Black patients than on white patients with the same level of sickness. So the model learned that Black patients were “lower risk,” and it sent fewer of them to high-risk care programs, even when they were just as sick.

c. Measurement and device bias

Sometimes the problem starts with the device that collects the data. Pulse oximeters estimate blood oxygen levels using how light passes through skin. Studies going back to the 1990s show that these devices tend to overestimate oxygen saturation more often in Black patients than in white patients. This means some Black patients may appear healthier than they really are. If AI systems use this kind of biased data, they can inherit and even amplify the problem.

d. Embedded race corrections

Some clinical algorithms build race straight into their formulas. For example, certain kidney function scores (eGFR) and lung function tests include “race adjustments” that assume Black patients naturally have different baseline function. Critics argue these adjustments often don’t have a strong biological basis and can delay diagnosis or treatment for minoritized patients.

Case 1: Risk-Prediction Algorithm for High-Risk Care

Obermeyer et al. studied a commercial algorithm used on millions of U.S. patients to decide who should be enrolled in intensive, “high-risk” care programs. They found that, at a given risk score, Black patients were actually much sicker than white patients. Even so, Black patients were enrolled in these programs at much lower rates. The authors estimate that fixing the bias in the model could increase the share of Black patients receiving extra care from about 17.7% to 46.5%.

This example shows how a decision that seems neutral—like using cost instead of illness as the target—can bake structural racism into AI.

Case 2: Pulse Oximeters and Skin Tone

During COVID-19, pulse oximeters became very important because people used them at home to track their oxygen levels. Multiple studies and a 2024 review from Johns Hopkins confirm that these devices overestimate oxygen saturation in patients with darker skin, causing “occult hypoxemia” (real low oxygen levels that the device misses).

If AI models rely on these biased readings, people with darker skin are more likely to have serious respiratory problems go under-treated.

Case 3: Dermatology AI and Skin-Tone Diversity

Researchers created the Diverse Dermatology Images (DDI) dataset to test how dermatology AI works on darker skin tones. When top-performing algorithms were tested on DDI, their accuracy dropped by about 27–36% compared with earlier test results. They did the worst on dark skin and rare diseases.

However, when the models were fine-tuned using the more diverse dataset, the performance gap mostly closed. In some cases, the AI even outperformed dermatologists on dark-skin images. This example shows both the harm of non-diverse training data and the benefits of deliberately including diversity.

Case 4: Imaging Foundation Models

A 2022 study looked at a chest-radiography foundation model and found clear differences in both learned features and performance across sex and race. For some labels—like “no finding” or “pleural effusion”—performance dropped for women and Black patients. That raises questions about using these models in practice without checking how well they work for different subgroups.

Similarly, cardiac MRI segmentation models have shown lower accuracy for certain racial groups when trained on unbalanced datasets. But when researchers apply fairness-focused training methods, they can reduce these gaps.

3. Impacts on Patients and Health Equity

All of these cases point in the same direction: marginalized groups tend to get less accurate predictions, fewer resources, or slower care when AI systems are biased. Black and other minoritized patients may be labeled “lower risk,” referred less often to specialty care, or have their symptoms underestimated because of biased devices.

AI tools can seem objective or “smart,” so doctors and hospital administrators might trust their outputs a lot. The CDC and other organizations warn that if bias isn’t addressed, AI could actually make existing health disparities worse instead of better.

Findings and Discussion

Finding 1: Bias is systemic, not just random.

Across different types of tools—risk models, imaging systems, and devices—bias usually comes from deeper inequalities in health care: who can get care, whose data is recorded, and who is included in studies. It’s usually not that developers are intentionally trying to be racist or sexist; it’s that the system they’re building on is already unequal. This means technical fixes alone won’t solve everything. We also need broader changes that focus on health equity.

Finding 2: Data diversity and quality matter a lot.

Almost every example involves skewed or low-quality data in some way. We need datasets that are diverse and well-annotated, including good information on skin tone and demographics. Recent work on standardized skin-tone scales is one step toward making it easier to check whether models are fair across skin tones.

Finding 3: Bias can be measured and reduced.

The good news is that bias isn’t always invisible. In the Optum risk algorithm and the DDI dermatology example, once researchers measured how different groups were being treated or classified, they could change the model. In the Optum case, switching the target from cost to actual health needs cut racial bias by about 84%. In dermatology, retraining on diverse images greatly improved performance for darker skin.

Finding 4: Ethical and regulatory frameworks are emerging, but not consistent.

Guidelines from places like JAMA, Health Affairs, Yale, and the CDC talk about transparency, including stakeholders, and ongoing monitoring. But civil-rights groups like the ACLU point out that regulation still doesn’t fully cover many AI tools, and companies often don’t share enough about their training data or methods. So there’s still a gap between what experts recommend and what actually happens in practice.

Limitations of this project:

This project is based only on existing literature, not on new experiments or interviews. It also mostly focuses on the U.S. and other high-income countries, where AI is being adopted faster and data is easier to access. I didn’t talk directly to clinicians or patients, and I didn’t test any AI models myself, so I’m relying on how other researchers designed their studies and interpreted their results.

Conclusion

AI has a lot of potential to improve health care. It can speed up diagnosis, help flag high-risk patients, and support doctors who are overloaded with work. But, as current research shows, medical AI is not automatically fair. When it’s trained on biased data, uses bad proxy variables, or gets deployed without proper testing, it can quietly hurt the same groups that already face barriers in health care.

To move toward fairer medical AI, researchers and policymakers recommend:

• Collecting and using diverse, high-quality data

• Auditing algorithms for performance across different groups before and after deployment

• Avoiding simplistic race-based corrections that lack a solid scientific basis

• Involving affected communities, clinicians, and ethicists in design and evaluation

• Creating stronger regulatory standards that focus on equity, not just accuracy

In the end, it’s not enough for AI in medicine to be technically impressive. It also has to be fair. Recognizing and fixing bias is a key step if we want AI to improve health for all patients—not just the ones who look like the people in the training data.

Abràmoff, M. D., et al. (2023). Considerations for addressing bias in artificial intelligence for health care. npj Digital Medicine.

Chin, M. H., et al. (2023). Algorithm bias and racial and ethnic disparities in health care. JAMA Network Open.

Daneshjou, R., et al. (2022). Disparities in dermatology AI performance on a diverse, curated clinical image set. npj Digital Medicine (preprint on arXiv).

Dankwa-Mullan, I., et al. (2024). Health equity and ethical considerations in using artificial intelligence in health care. Preventing Chronic Disease (CDC).

Haider, S. A., et al. (2024). A systematic review on AI-driven racial disparities in healthcare. Journal of Medical Systems.

Jain, A., et al. (2023). Awareness of racial and ethnic bias and potential solutions in clinical algorithms. Annals of Internal Medicine.

Mittermaier, M., et al. (2023). Bias in AI-based models for medical applications. npj Digital Medicine.

Norori, N., et al. (2021). Addressing bias in big data and AI for health care. Journal of Biomedical Informatics.

Obermeyer, Z., et al. (2019). Dissecting racial bias in an algorithm used to manage the health of populations. Science, 366(6464), 447–453.

Public Health Insights. (2024). Reflecting our biases through AI in health care. Harvard Medical School.

Pulse Oximeters’ Racial Bias. (2024). Johns Hopkins Bloomberg School of Public Health.

• Zou, J., & Schiebinger, L. (2021). Ensuring that biomedical AI benefits diverse populations. Nature Medicine.

• • Yale School of Medicine. (2023). Eliminating racial bias in health care AI: Expert panel offers guidelines.