|
||
|
||
We have spent almost three decades building the architecture of digital inclusion—accessibility standards, assistive technology exemptions, disability-inclusive procurement rules. Quietly, and largely outside the scope of that architecture, a second system has been built alongside it: one that automates suspicion, and that treats the bodies and voices of persons with disabilities as anomalies to be flagged.
This is not a hypothetical harm. It is an immediate, physical one. AI-driven surveillance and behavioral analytics—deployed in airports, transit systems, retail stores, workplaces, and increasingly in general public infrastructure—are trained overwhelmingly on neurotypical, able-bodied movement and speech. The consequence is mechanical and predictable: bodies and voices that fall outside that training distribution are read not as different, but as wrong.
Behavioral-detection systems—marketed under labels such as threat assessment, anomaly detection, or risk scoring—are built to flag deviation from a statistical norm. That norm is not often disclosed, rarely audited, and almost never validated against the full range of human neurology and physiology. As the AI Now Institute’s 2019 report on disability and AI documents, this is not a hypothetical gap: AI systems are trained on data that reflects existing patterns of exclusion, and those historical patterns of marginalization become embedded in the statistical baseline itself.
The consequences are already visible in deployed systems, not just in theory. AI-assisted lie-detection pilots at borders—including the EU-funded iBorderCtrl trial and the U.S. Department of Homeland Security-backed AVATAR kiosk—have drawn documented criticism for treating reduced eye contact, fidgeting, and speech disfluency as indicators of deception, with independent reviewers reporting high false-positive rates. For a person with Parkinson’s, tremor risks being read the same way. For someone on the autism spectrum, or a person with a speech disorder, atypical affect or disfluency risks being caught in the same net. For a person who uses a wheelchair, travels with a mobility-assistance dog, or wears a prosthetic limb, published research on computer-vision detection of assistive devices confirms that unfamiliar silhouettes and movement patterns still produce frequent misclassifications, particularly in low-light or cluttered conditions—precisely the everyday environments these systems are deployed in.
None of this requires malicious design. It requires only ordinary indifference—the same indifference that, before the accessibility movement forced the issue, produced buildings with no ramps and websites with no alt text. The difference now is that the barrier is not a staircase. It is a probability score, generated in real time, often invisible to the person it is scoring, and frequently feeding into a human decision—a security stop, a denied transaction, a flagged file—that the person has no practical way to contest.
Legal scholarship on chilling effects—from Monica Youn’s account of how the concept originated in efforts to suppress disfavored groups, to Jonathon Penney’s more recent work tracing how surveillance disproportionately represses already marginalized and disempowered communities—has long recognized that surveillance’s psychological toll is not distributed evenly. It falls hardest on those for whom the cost of being misread is highest, and the recourse is thinnest. Persons with disabilities meet both conditions.
Consider what the mere possibility of algorithmic misclassification does to a person navigating public space. They face a choice with no good option. They can attempt to mask—suppress a tremor, script a scripted affect, avoid a mobility aid that draws algorithmic attention—a strategy that is exhausting at best and physiologically impossible at worst. Or they can withdraw: skip the airport, avoid the public building, decline the job interview conducted under monitored video, stay away from the protest or the public meeting where their presence might be flagged as anomalous.
This is the ceiling I want to name plainly: an automated ceiling on accessibility, layered invisibly on top of the physical and attitudinal ceilings the disability rights movement has spent decades dismantling. A ramp does not help if the sensor at the top of it scores your gait as suspicious. A screen reader does not help if the system behind the counter has already flagged your voice pattern as untrustworthy before you finish your sentence.
This is not solely a human rights conversation to be left to disability advocates, and it is not solely an AI-ethics conversation to be left to model auditors. It is an Internet governance conversation, squarely within this community’s remit, because the infrastructure enabling this harm is the same infrastructure this community builds, standardizes, and regulates: identity and biometric verification protocols, edge-deployed video analytics, behavioral-scoring APIs sold as commodity services, and the data-sharing agreements that move a flagged score from a private vendor into a public authority’s hands without the accountability either would face alone.
Three interventions follow directly from that framing:
First, disability must be treated as a mandatory dimension of algorithmic auditing, not an edge case discovered after deployment. Bias audits that test only for race, gender, or age miss the training-distribution gap that harms users with disabilities most acutely—precisely because disability is heterogeneous by nature and cannot be captured by a single demographic proxy. Testing protocols for any behavioral or biometric system procured by a public authority should be required to include atypical movement, speech, and affect profiles before certification, not after complaint.
Second, the WCAG framework that has anchored digital accessibility for the web era needs an explicit extension into physical-digital surveillance systems. We built enforceable standards for how a screen should behave when a user with disabilities encounters it. We have built almost nothing for how a camera, a microphone array, or a risk-scoring model should behave when it encounters the body of a person with a disability. That gap is where the next decade of disability-rights litigation is heading, and it would be far cheaper—in both human and institutional terms—to close it through standards bodies now than through courts later.
Third, any jurisdiction deploying algorithmic behavioral assessment in a public function needs a contestability mechanism that does not depend on the flagged individual first proving they were flagged. Chilling effects thrive on uncertainty; uncertainty thrives on opacity. A person with a disability cannot mask what they do not know is being measured, and cannot appeal a score they are never shown.
Every accessibility standard this community has fought to enforce—from Section 508 to WCAG 2.2 AA to UNCRPD-aligned procurement rules—was built on a single premise: that digital and physical infrastructure should adapt to the full range of human variation, not the other way around. AI surveillance, left unaudited, quietly reverses that premise. It asks persons with disabilities to conform to a model’s expectations or be flagged, monitored, or excluded.
We are not short of the technical means to fix this. Disability-inclusive training data exists. Contestability frameworks exist in other regulatory domains and can be adapted. What is missing is the will to treat disability as a first-order design constraint for surveillance systems, the same way this community has learned—through hard, incremental struggle—to treat it as a first-order design constraint for the web itself.
The alternative is a future in which the accessibility gains of the last twenty years are quietly undone by a layer of infrastructure most users with disabilities will never see, cannot contest, and did not consent to being measured against.
Sponsored byVerisign
Sponsored byWhoisXML API
Sponsored byIPv4.Global
Sponsored byDNIB.com
Sponsored byCSC
Sponsored byVerisign
Sponsored byRadix