Overview

The study scores one synthetic population with two kinds of tool: the published VI‑SPDAT and large language models. Thirty‑two base profiles cover the VI‑SPDAT's 16 vulnerability indicators. Each profile is cloned across two genders, the five race and ethnicity categories used by the U.S. Department of Housing and Urban Development (HUD), and five cities, giving 50 versions that differ only in the stated gender, race, and location.

People assessed for housing often withhold information for fear of being judged. Each clone is therefore scored twice: once with full disclosure and once with selected indicators withheld (underdisclosure).

If a tool is unbiased, every version of a profile receives the same score. Each demographic contrast is tested with a paired t‑test that compares a profile with itself, at α = 0.05 with Holm correction across the ten demographic tests.

Each base profile is a demographically neutral case recorded at the indicator level: which of the 16 vulnerability indicators, and the pre‑survey age indicator, apply. Domain subtotals and totals are computed from those flags with the published scoring rules, never assigned by hand.

Profile text carries no name, pronoun, place, or other demographic signal; only the stated gender, race, and city differ across a profile's 50 clones. Eleven narrative fields (work history, path into homelessness, housing, finances, health, safety, family, a typical day, goals, service history, and presentation at interview) describe every flagged indicator in plain language, so a reader who never sees the VI‑SPDAT can still judge vulnerability. Expand a row below to read them.

Continuums of Care (CoCs) in the United States allocate housing through the Coordinated Entry System (CES), which prioritizes the most vulnerable unhoused people for shelter, assistance, and funding. Trained staff interview each person with a standardized screening tool, most often the Vulnerability Index–Service Prioritization Decision Assistance Tool (VI‑SPDAT); more than one in four CoCs report using it. The VI‑SPDAT converts the answers into a score, and people are ranked by that score for support.

Evaluations of CES assessment data, including C4 Innovations' racial equity analysis, have found racial disparities in VI‑SPDAT prioritization that disadvantage people of color. A Black client can rank below a white client with lower need, leaving some of the most vulnerable people without help while scarce resources go to others. In December 2020, OrgCode, the tool's developer, announced it would move away from the VI‑SPDAT and encouraged communities to adopt alternatives.

Some CoCs are now replacing it with machine‑learning systems, but little research shows that such systems allocate more fairly. This study tests whether language models, a readily available AI alternative, are more accurate or less biased than the tool they might replace.

Real VI‑SPDAT assessments are protected by privacy law, but OrgCode published the interview and scoring method, which remains available through CoCs that adopted it. The study applies that method to its own synthetic population.

The study offers a controlled audit of AI‑based vulnerability scoring against the instrument it could replace: the same records, varied one demographic attribute at a time, with every contrast paired within profile.