California Acknowledges Unreported High-Risk AI Systems in Government Use
California's state government has acknowledged using high-risk artificial intelligence systems—including at least one category that was not disclosed in mandatory reports filed last year, according to a CalMatters investigation.
The admission comes amid growing scrutiny of how governments deploy automated systems that can significantly impact residents' lives, from benefit eligibility determinations to criminal justice applications. High-risk AI systems typically refer to algorithms that make or meaningfully influence consequential decisions about individuals.
The disclosure highlights ongoing challenges with compliance under California's AI transparency requirements, which mandate that agencies report their use of automated systems. The failure to include certain systems in prior filings raises concerns about the adequacy of existing inventory processes.
State officials indicated that the unreported systems have since been identified and added to the official inventory. However, the incident underscores the difficulty governments face in tracking their own AI deployments as these technologies become more embedded in public sector operations.
Experts have long warned that without robust oversight mechanisms, high-risk AI systems can perpetuate biases, produce inaccurate outcomes, or operate without appropriate human review. California's acknowledgment may prompt renewed calls for stronger enforcement of AI reporting requirements and independent auditing of government algorithms.