What Is Algorithmic Fairness? Bias Testing for Underwriting Health Models

The integration of artificial intelligence and machine learning into health insurance underwriting is no longer a future-facing concept; it is an operational reality. Carriers are using algorithmic models to increase efficiency, improve risk assessment, and personalize products. However, this technological shift has introduced a significant compliance and ethical challenge: ensuring algorithmic fairness. As regulators sharpen their focus on the discriminatory potential of these systems, understanding and implementing robust algorithmic fairness bias testing for underwriting models has become a critical priority for medical directors and compliance leadership.

"A 2021 study by researchers at Stanford University found that a widely used algorithm for predicting health risk systematically assigned lower risk scores to Black patients than to equally sick white patients, limiting their access to specialized care programs."

The core challenge of algorithmic fairness bias testing in underwriting models

Algorithmic fairness in the context of health underwriting is the principle that automated systems should not produce unjustly disparate outcomes for individuals based on protected characteristics like race, ethnicity, gender, or age. The core challenge lies in the data used to train these models. Historical health data can reflect and amplify societal biases and structural inequalities in healthcare access and quality. If a model is trained on data where certain demographic groups have historically had less access to care, it may learn to associate those groups with lower health risks, not because they are healthier, but because their conditions are underdiagnosed. This is where algorithmic fairness bias testing for underwriting models becomes essential. It is a systematic process of evaluating these models to identify and mitigate such biases before they result in discriminatory decisions, such as unfairly priced policies or wrongful denials of coverage.

The regulatory environment is rapidly solidifying around this issue. In December 2023, the National Association of Insurance Commissioners (NAIC) adopted the Model Bulletin on the Use of Artificial Intelligence Systems by Insurers. This bulletin requires carriers to establish a formal program to manage the risks of AI, with a strong emphasis on preventing "inaccurate, arbitrary, capricious, or unfairly discriminatory outcomes." This places the onus on insurers to proactively test their models and demonstrate their fairness, a significant shift from a reactive compliance posture.

Fairness Metric	Description	Application in Health Underwriting
Statistical Parity	Aims for the model's approvals, denials, or risk scores to be proportional across different demographic groups.	Ensures that the overall percentage of applicants approved for a given policy is similar across racial or gender groups.
Equal Opportunity	Focuses on the true positive rate, requiring that the model correctly identifies positive outcomes (e.g., high-risk individuals who need intervention) at an equal rate across groups.	Helps prevent a model from being less accurate at identifying a specific disease in one demographic group compared to another.
Predictive Equality	Requires that the false positive rate is equal across groups. For example, the rate at which the model incorrectly flags someone as high-risk should be the same for all groups.	Minimizes the risk of disproportionately burdening a specific group with unnecessary follow-up or higher premiums based on inaccurate flags.
Counterfactual Fairness	A causal-based metric that checks if a model's decision for an individual would change if their demographic attributes were different, all else being equal.	Assesses if a specific applicant's predicted risk score would change if, hypothetically, only their race was different, providing a direct test for discriminatory influence.

Industry applications and compliance imperatives

For chief medical officers and compliance leaders, the application of algorithmic bias testing is not merely a technical exercise but a strategic imperative. It directly impacts regulatory compliance, risk management, and market viability.

### regulatory and legal risk mitigation

The NAIC Model Bulletin, which has been adopted by nearly half of the states as of mid-2024, is just the beginning. States like Colorado and New York have already established specific regulations requiring insurers to test their models for unfair discrimination. Failure to conduct and document rigorous algorithmic fairness bias testing for underwriting models can result in regulatory audits, fines, and legal challenges. A documented testing framework provides a defensible position, showing regulators that the carrier has exercised due diligence.

### Building Trust with Reinsurers and Partners

Reinsurers are increasingly scrutinizing the digital underwriting practices of their ceding partners. A carrier that cannot demonstrate the fairness and validity of its algorithmic models may face challenges securing reinsurance capacity or may be subject to less favorable treaty terms. Robust bias testing is becoming a prerequisite for participation in the broader insurance ecosystem.

### ensuring clinical and actuarial soundness

Biased models are, by definition, inaccurate. If an algorithm systematically underestimates risk in one population and overestimates it in another, the resulting underwriting decisions are neither clinically sound nor actuarially fair. This can lead to adverse selection, unprofitable books of business, and a failure to meet the core mission of providing equitable access to insurance.

Current research and evidence

The academic and research community is actively engaged in developing more sophisticated methods for bias detection and mitigation.

• Data-centric approaches: Research from institutions like MIT and Stanford has focused on techniques to pre-process data to remove bias before model training. This can involve re-weighting data points to give more significance to underrepresented groups or using synthetic data generation to create more balanced datasets.
• Model-centric approaches: Researchers are exploring modifications to learning algorithms that penalize biased outcomes. Techniques like adversarial debiasing, as described in a 2018 paper by Brian K. S. Hu and colleagues, involve training a second model to predict a protected attribute from the primary model's output, forcing the primary model to learn representations that are not dependent on that attribute.
• Post-processing techniques: This involves adjusting a model's outputs after a prediction is made. For example, if a model is found to have a lower approval threshold for one group, a post-processing step could adjust the threshold to equalize the approval rates, as explored in work by Moritz Hardt, Eric Price, and Nathan Srebro in 2016.

However, challenges remain. There is no single, universally accepted definition of "fairness," as the comparison table above illustrates. A model that satisfies one fairness metric may fail another, forcing organizations to make complex ethical and business trade-offs.

The future of algorithmic governance in underwriting

The future of underwriting compliance will be defined by a continuous, evidence-based approach to algorithmic governance. The trend is moving away from one-time model validation toward ongoing monitoring and dynamic adjustment. As models learn and evolve from new data, they can develop new biases, a phenomenon known as model drift. Future compliance frameworks will require systems that can detect this drift in real-time and trigger alerts for re-testing and recalibration. The work of the NAIC's Big Data and Artificial Intelligence (H) Working Group, which is exploring a potential uniform model law, signals that regulatory expectations will only become more stringent and specific.

Frequently asked questions

Q: What is the first step my organization should take to address algorithmic bias? A: The first step is to create a cross-functional governance committee that includes representation from compliance, legal, data science, and clinical teams. This committee should be tasked with developing a formal AI use policy based on established frameworks like the NAIC Model Bulletin and the NIST AI Risk Management Framework.

Q: Is it enough to simply remove protected attributes like race from our datasets? A: No, this is often insufficient. Models can easily learn to use proxy variables, such as ZIP codes, income levels, or even language patterns, that are highly correlated with protected attributes. Effective bias testing must analyze outcomes and identify these proxy effects, not just remove sensitive data fields.

Q: How do we balance the trade-off between different fairness metrics? A: This is a significant challenge without an easy answer. It requires a clear, documented process where the governance committee evaluates the trade-offs in the context of the specific use case, legal requirements, and the company's ethical principles. The decision and its rationale should be explicitly documented for regulatory review.

Q: Does using a third-party model absolve us of the responsibility for bias testing? A: No. Regulators have made it clear that the insurer is ultimately responsible for the models it uses, regardless of whether they are developed in-house or sourced from a vendor. Due diligence on third-party models, including demanding transparency and evidence of bias testing, is a critical compliance function.

As the industry moves toward a more regulated and transparent future for AI, building a compliance-first approach to digital underwriting is essential. The complex work of algorithmic fairness is a core part of this new landscape. For more information on navigating these regulatory challenges, explore our compliance guides and regulatory insights at circadify.com/industries/payers-insurance.