PAN Lab — govern the network before failure spreads

Legend

Start here

Challenge of the Day

Challenge result

Round summary

Choose a starting point

Apply institutional pressure

Pull governance levers

Qualitative readout

Edit this network

Explore's workshop — authored parts and pathways, saved in this browser

From Simulation to Lab — Sociotechnical Systems Modeling and Simulation

Modeling evidence and assumptions behind this network

The same AI, hands off: the agentic office

The AI here doesn't just draft casework — it acts on cases, and a stretched staff waves most of it through. Same model as the other two offices. In the published runs — a modeled office, not a real one — errors stick here about 75% of the time, versus 20% and 16% next door. Find the levers that change that.

A narrated tour of the whole Lab — or today's ranked governance exercise.

Description

The same AI, hands off: the agentic office. No pressures applied. No governance levers in place.

Who and what is in the system

  • AI assistant + autonomous agentDrafts decisions and, as an agent, acts on cases with minimal review.
  • Stretched staffA small team supervising many automated actions at once.
  • Case recordsThe shared record system both people and agents read and write.
  • Auto-retrievalPulls prior records into the model's context automatically.

How strongly each pathway flows right now

  • Agent outputs adopted with little verification: strong.
  • Agent writes case records directly: moderate.
  • Hurried prompts frame the model toward confirmation: moderate.
  • Adopted outputs documented into the record: strong.
  • Retrieved records re-enter drafts as fresh context: moderate.
  • Staff read and rely on the record as written: moderate.
  • Shortcuts and adopted claims spread between coworkers: moderate.
  • Agent outputs chained into other agents' context, client context riding along: moderate.
  • Second opinions between coworkers — rare under this load: off — a protective peer check; stronger is better.
  • No independent model checks the agents at baseline: off — a protective peer check; stronger is better.

Where the gauges sit

The failure regime reads self-feeding. Operator deference drift: high; Record contamination pressure: medium; Correction capacity: low; Deployment authority engaged: low.

Under heavy incoming pressure: Stretched staff, Case records, AI assistant + autonomous agent.

Edition

Current is the live ruleset (v10): modifiers are active components you tether onto pathways, and several old network-wide levers now live there. Classic preserves the game as it was before. Switching editions starts a fresh run.

Where each failure mode lives here

The Lab speaks in pathways, pressures, levers, and gauges. This map connects that vocabulary to the formal failure-mode names used in the research grounding it — including a 2026 national survey of 1,179 U.S.-based social workers.

  • Automation bias / overreliancecore

    The “Failures adopted by people or agents” pathway and the operator-deference-drift gauge. Staff turnover and autonomy expansion push it up; the deskilling-arrest lever caps it.

    Evidence: In the same national survey, 40.8% of respondents reported ethical concerns about relying on AI for decision-making, and overreliance on automated decision-making was among the most frequently cited concerns overall.

  • Deskilling / professional judgment erosioncore

    Overreliance in slow motion: the deference gauge drifting upward while correction capacity thins. The deskilling-arrest lever is its deliberate counter-schedule.

  • Sycophancy / agreement-seeking outputcore

    The pushback-heavy-usage pressure runs the “Operator framing biases the model” pathway hot and makes the agreeable answers easier to adopt; the framing-hygiene lever dampens the loop at its origin.

    Evidence: Research on AI sycophancy describes it as a fragmented construct — a family of distinct agreement-seeking behaviors that share a label but differ in form, mechanism, measurement, and required mitigation — and finds it intensifies under user pushback and across multi-turn interaction.

  • Hallucination / incorrect-output propagationcore

    The Lab's core premise: every pathway in the diagram carries incorrect output away from its source, and every lever is a way of governing that propagation rather than assuming a perfect model.

  • Unsafe data flow / privacy & confidentialitycore

    Modeled as pathways, gauged as exposure (Phase NP). Unsanctioned tool use opens a visible egress to the off-network sink; connector sprawl replicates an unverified cache between record systems; case-file-flagged pathways (MiDAS-class enforcement replication, records feeding vendor models) carry the same concern. While any such pathway runs, the Privacy gauge drains — and in Hard and Expert a full gauge is part of the win. “Vet connections” cuts the pathways structurally; “Store less data” shrinks what there is to expose.

    Evidence: In the same national survey, concerns about data privacy and security were the most frequently reported challenge to using AI in practice (46.5% of respondents), and an increased focus on client privacy and confidentiality was the most requested improvement to AI tools for social work (50.4%).

  • Bias propagation / institutional workflow biascore

    Biased framings and contaminated records travel the same workflow pathways failures do — an institutional propagation question, and the documented cases show the workflow (not the model alone) carrying the equity outcome in both directions. The Lab models no demographics: differential harm to served people is recorded outside the network, never computed from its dynamics.

    Evidence: Evaluation evidence on the Allegheny Family Screening Tool found that screener overrides of the tool's recommendations reduced racial disparity in screen-in rates relative to the tool alone.

    Evidence: An independent audit of Rotterdam's welfare-fraud risk model documented scores skewed against already-vulnerable groups, and the city suspended the system's use following the scrutiny.

  • Transparency / provenance failurecore

    The record-contamination gauge reads how much unlabeled machine content feeds back into decisions; “Mark AI-written records” discounts it and “Gate vendor updates” attacks opacity at procurement.

  • Weak human oversight / safeguard failurecore

    The scenario axis itself: one model, three oversight cultures, three very different outcomes. The correction and authority gauges track it; “Review the riskiest first”, “Pause AI on alarms”, “Require sign-off”, and “Review on schedule” govern it.

    Evidence: In the same national survey, 42.1% of respondents reported having no role in decision-making about AI adoption in their workplace; the report concludes most respondents have limited or no control over how AI technologies are selected or implemented within their organizations.

  • Low AI literacy / verification readinesscore

    The AI-literacy-gap pressure: verification skill (not time) drops and deference rises as trust calibrates on the tool itself. The correction-capacity gauge reads the result. The grounding literature proposes AI literacy as a core professional competency.

    Evidence: The same national survey describes a gap between AI exposure and AI preparedness: 26.6% of respondents cited lack of training or understanding of AI technology as a challenge, 53.4% said training on AI tools and effective use would help, and clear guidelines on the ethical use of AI were the most-endorsed need (66.8%).

    Evidence: AI literacy — the knowledge and skills required to understand, use, and critically evaluate AI systems — has been proposed as a core competency for social work, relevant even to practitioners who never directly use AI tools.

  • AI iatrogenics / governance backfireadvanced

    First-class here: purging records without reading them backfires exactly as the published runs found, and the efficiency readout will call a lever stack counterproductive to its face. The quieter trap — oversight whose gains are bought by rising deference — is why the deskilling-arrest lever exists.

    Evidence: In published PAN runs, content-blind record removal raised the contaminated share of the record system by stripping benign dilution; only content-aware decontamination reliably reduced it. (PAN Governance Lever Audit ('content-blind levers backfire'))

  • Monitoring failure / drift blindnessadvanced

    Two pressures carry it: the silent vendor update (drift arriving under controls tuned to old behavior) and monitoring going stale (dashboards nobody must act on — the authority gauge hollows while the regime worsens). “Review on schedule” and “Escalate checks” are the counters.

  • Environmental burden (external context)external context

    Deliberately outside the network: nothing in this Lab computes environmental cost, and no gauge claims to. Practitioner concern about AI's environmental impact is documented in the survey evidence and belongs in deployment governance as external context.

    Evidence: In the same national survey's open-ended comments, ethical concerns — prominently including the environmental impact of AI infrastructure — were the most common theme, and the report's first recommendation includes environmental impact among the topics profession-wide ethical guidance should address.

The Lab runs invented cases. Your organization runs on a real one.

We map your actual deployment — its pathways, pressures, and the levers your leadership can pull — through intake, diagnosis, prescription, and monitoring.

Sources & Evidence

Tap to expand

Claims made on this page and what supports them. The full registry lives in Evidence.

EmpiricalAn independent audit of Rotterdam's welfare-fraud risk model documented scores skewed against already-vulnerab…

An independent audit of Rotterdam's welfare-fraud risk model documented scores skewed against already-vulnerable groups, and the city suspended the system's use following the scrutiny.

wiredlighthousereports2023GroundingInvestigative

WIRED / Lighthouse Reports, Inside the suspicion machine (2023) https://www.wired.com/story/welfare-state-algorithms/

https://www.wired.com/story/welfare-state-algorithms/

Appears in: PAN framework development

Grounds: deployment audit: Rotterdam welfare-fraud algorithm

EmpiricalEvaluation evidence on the Allegheny Family Screening Tool found that screener overrides of the tool's recomme…

Evaluation evidence on the Allegheny Family Screening Tool found that screener overrides of the tool's recommendations reduced racial disparity in screen-in rates relative to the tool alone.

ScenarioIn a published PAN model run over a supervised-plus-agent scenario, adding a verifier to the autonomous agent …

In a published PAN model run over a supervised-plus-agent scenario, adding a verifier to the autonomous agent removed roughly 45.8% of steady-state harm and a coordinated governance package roughly 43.0%, while upgrading the model alone removed roughly 6.3%.

From the published runs: PAN Social Work User Guide (v6.2.11.6), lever-ranking walkthrough (fig03).

ScenarioAcross a published PAN sweep of 216 deployment topologies (432 sampled deployments), system-layer governance o…

Across a published PAN sweep of 216 deployment topologies (432 sampled deployments), system-layer governance outperformed equal-effort model-layer improvement in 99.1% of cases at roughly 3.4x mean leverage - a figure the project corrected downward from an earlier 9.4x estimate.

From the published runs: PAN Normalized Baseline Analysis; README direction-and-shape summary.

ScenarioIn published PAN runs, content-blind record removal raised the contaminated share of the record system by stri…

In published PAN runs, content-blind record removal raised the contaminated share of the record system by stripping benign dilution; only content-aware decontamination reliably reduced it.

From the published runs: PAN Governance Lever Audit ('content-blind levers backfire').

ScenarioIn published PAN runs, the same AI embedded in three office cultures produced steady-state error adoption of r…

In published PAN runs, the same AI embedded in three office cultures produced steady-state error adoption of roughly 75% (agentic low-oversight), 20% (human-supervised), and 16% (high-governance professional).

From the published runs: PAN Social Work User Guide (v6.2.11.6), three-scenario comparison (fig02b).

EmpiricalIn a national survey of 1,179 U.S.-based social workers conducted from October 2025 to February 2026 by the Un…

In a national survey of 1,179 U.S.-based social workers conducted from October 2025 to February 2026 by the University of Texas at Austin in collaboration with NASW, 63.5% of respondents reported using AI tools or technologies in their current role.

borah2026bAcademic

Borah, P., & Landers, G. (2026). AI and social work: A national workforce survey. Steve Hicks School of Social Work, University of Texas at Austin, in partnership with NASW.

Appears in: 1023AI authored research

Topics: social-work

EmpiricalIn the same national survey, concerns about data privacy and security were the most frequently reported challe…

In the same national survey, concerns about data privacy and security were the most frequently reported challenge to using AI in practice (46.5% of respondents), and an increased focus on client privacy and confidentiality was the most requested improvement to AI tools for social work (50.4%).

borah2026bAcademic

Borah, P., & Landers, G. (2026). AI and social work: A national workforce survey. Steve Hicks School of Social Work, University of Texas at Austin, in partnership with NASW.

Appears in: 1023AI authored research

Topics: social-work

EmpiricalIn the same national survey, 40.8% of respondents reported ethical concerns about relying on AI for decision-m…

In the same national survey, 40.8% of respondents reported ethical concerns about relying on AI for decision-making, and overreliance on automated decision-making was among the most frequently cited concerns overall.

borah2026bAcademic

Borah, P., & Landers, G. (2026). AI and social work: A national workforce survey. Steve Hicks School of Social Work, University of Texas at Austin, in partnership with NASW.

Appears in: 1023AI authored research

Topics: social-work

EmpiricalThe same national survey describes a gap between AI exposure and AI preparedness: 26.6% of respondents cited l…

The same national survey describes a gap between AI exposure and AI preparedness: 26.6% of respondents cited lack of training or understanding of AI technology as a challenge, 53.4% said training on AI tools and effective use would help, and clear guidelines on the ethical use of AI were the most-endorsed need (66.8%).

borah2026bAcademic

Borah, P., & Landers, G. (2026). AI and social work: A national workforce survey. Steve Hicks School of Social Work, University of Texas at Austin, in partnership with NASW.

Appears in: 1023AI authored research

Topics: social-work

EmpiricalIn the same national survey, 42.1% of respondents reported having no role in decision-making about AI adoption…

In the same national survey, 42.1% of respondents reported having no role in decision-making about AI adoption in their workplace; the report concludes most respondents have limited or no control over how AI technologies are selected or implemented within their organizations.

borah2026bAcademic

Borah, P., & Landers, G. (2026). AI and social work: A national workforce survey. Steve Hicks School of Social Work, University of Texas at Austin, in partnership with NASW.

Appears in: 1023AI authored research

Topics: social-work

EmpiricalIn the same national survey's open-ended comments, ethical concerns — prominently including the environmental …

In the same national survey's open-ended comments, ethical concerns — prominently including the environmental impact of AI infrastructure — were the most common theme, and the report's first recommendation includes environmental impact among the topics profession-wide ethical guidance should address.

borah2026bAcademic

Borah, P., & Landers, G. (2026). AI and social work: A national workforce survey. Steve Hicks School of Social Work, University of Texas at Austin, in partnership with NASW.

Appears in: 1023AI authored research

Topics: social-work

massey2026Academic

Massey, M., Williams, I., Polistina, G., & Breaux, E. (2026). Artificial intelligence and environmental justice: A critical review of social work literature. Society for Social Work and Research Annual Conference. https://sswr.confex.com/sswr/2026/webprogram/Paper62664.html

https://sswr.confex.com/sswr/2026/webprogram/Paper62664.html

Appears in: National survey report (2026)

Topics: social-work

ConceptualAI literacy — the knowledge and skills required to understand, use, and critically evaluate AI systems — has b…

AI literacy — the knowledge and skills required to understand, use, and critically evaluate AI systems — has been proposed as a core competency for social work, relevant even to practitioners who never directly use AI tools.

ahn2025Academic

Ahn, E., Choi, M., Fowler, P., & Song, I. H. (2025). Artificial intelligence (AI) literacy for social work: Implications for core competencies. Journal of the Society for Social Work and Research, 16(1), 9-26. https://doi.org/10.1086/735187

doi.org/10.1086/735187

Appears in: 1023AI authored research; National survey report (2026)

Topics: social-work

EmpiricalResearch on AI sycophancy describes it as a fragmented construct — a family of distinct agreement-seeking beha…

Research on AI sycophancy describes it as a fragmented construct — a family of distinct agreement-seeking behaviors that share a label but differ in form, mechanism, measurement, and required mitigation — and finds it intensifies under user pushback and across multi-turn interaction.

ye2026Academic

Ye, et al. (2026). What Counts as AI Sycophancy? A Taxonomy and Expert Survey of a Fragmented Construct [Preprint]. arXiv.

Appears in: PAN framework development

Topics: ai-safety

ConceptualClaims and behaviors spread through peer networks sideways, along informal ties — diffusion research finds wea…

Claims and behaviors spread through peer networks sideways, along informal ties — diffusion research finds weak ties and small-world clustering carry information and practices across a network far faster than formal reporting lines.

granovetter1973Academic

Granovetter, M. S. (1973). The strength of weak ties. American Journal of Sociology, 78(6), 1360–1380. https://doi.org/10.1086/210318

doi.org/10.1086/210318

Appears in: 1023AI authored research

watts1998Academic

Watts, D. J., & Strogatz, S. H. (1998). Collective dynamics of 'small-world' networks. Nature, 393(6684), 440–442. https://doi.org/10.1038/30918

doi.org/10.1038/30918

Appears in: 1023AI authored research

Topics: complexity-science

EmpiricalA single automated rule set applied uniformly and without human review produced tens of thousands of correlate…

A single automated rule set applied uniformly and without human review produced tens of thousands of correlated wrongful fraud determinations in the documented Michigan MiDAS case — one flaw repeating at caseload scale rather than averaging out.

EmpiricalModel behavior — including misaligned behavior — can propagate through model-to-model channels: research shows…

Model behavior — including misaligned behavior — can propagate through model-to-model channels: research shows narrow in-context examples and inter-model interaction can induce broadly misaligned behavior in the receiving model.

afonin2026Academic

Afonin, N., Andriianov, N., Hovhannisyan, V., Bageshpura, N., Liu, K., Zhu, K., Dev, S., Panda, A., Rogov, O., Tutubalina, E., Panchenko, A., & Seleznyov, M. (2026). Emergent misalignment via in-context learning: Narrow in-context examples can produce broadly misaligned LLMs [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2510.11288

doi.org/10.48550/arXiv.2510.11288

Appears in: 1023AI authored research

Topics: ai-alignment, complexity-science

panpatil2025Academic

Panpatil, S., Dingeto, H., & Park, H. (2025). Eliciting and analyzing emergent misalignment in state-of-the-art large language models [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2508.04196

doi.org/10.48550/arXiv.2508.04196

Appears in: 1023AI authored research

Topics: ai-alignment, complexity-science

ConceptualEmerging agentic-AI governance frameworks treat inter-agent interaction as a first-class assurance surface, re…

Emerging agentic-AI governance frameworks treat inter-agent interaction as a first-class assurance surface, requiring explicit oversight of agent-to-agent couplings rather than per-model evaluation alone.

khan2025Academic

Khan, R., Joyce, D., & Habiba, M. (2025). AGENTSAFE: A unified framework for ethical assurance and governance in agentic AI [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2512.03180

doi.org/10.48550/arXiv.2512.03180

Appears in: 1023AI authored research

Topics: ai-governance

EmpiricalDocumented benefit-automation failures replicated determinations into downstream systems with no independent r…

Documented benefit-automation failures replicated determinations into downstream systems with no independent reconciliation against the source records — Michigan MiDAS actioned replicated flags and Robodebt reversed the onus onto recipients.

EmpiricalDocumented risk-scoring deployments computed scores from multi-agency administrative records originally collec…

Documented risk-scoring deployments computed scores from multi-agency administrative records originally collected for other purposes, which is the data-protection critique recorded in independent reviews of these systems.

EmpiricalDocumented enforcement systems actioned replicated flags automatically — garnishment and penalties applied bef…

Documented enforcement systems actioned replicated flags automatically — garnishment and penalties applied before any human review step in the recorded MiDAS deployment.

EmpiricalThe Robodebt Royal Commission documented debts raised from income-averaged derived inputs with the onus placed…

The Robodebt Royal Commission documented debts raised from income-averaged derived inputs with the onus placed on recipients to disprove the automated assessments.

EmpiricalProfessional-verification cultures documented in social work practice sustain peer checking of AI output rathe…

Professional-verification cultures documented in social work practice sustain peer checking of AI output rather than unquestioned acceptance.

baez2026Academic

Báez, J. C., Ahn, E., Tamietti, A., Victor, B. G., & Goldkind, L. (2026). Clinical social workers’ perceptions of large language models in practice: Resistance to automation and prospects for integration. Journal of Evidence-Based Social Work, 23(1), 42–63. https://doi.org/10.1080/26408066.2025.2542450

doi.org/10.1080/26408066.2025.2542450

Appears in: National survey report (2026)

Topics: social-work

EmpiricalClinical assessors bound by algorithmic allocation with limited override capacity form a documented constraine…

Clinical assessors bound by algorithmic allocation with limited override capacity form a documented constrained-judgment pattern in home-care assessment.

sutton2020Academic

Sutton, R. T., Pincock, D., Baumgart, D. C., Sadowski, D. C., Fedorak, R. N., & Kroeker, K. I. (2020). An overview of clinical decision support systems: Benefits, risks, and strategies for success. NPJ Digital Medicine, 3(1), 17. https://doi.org/10.1038/s41746-020-0221-y

doi.org/10.1038/s41746-020-0221-y

Appears in: 1023AI authored research

upturnGroundingAdvocacy

Upturn, Calculated Need: automated home-care hour allocation https://www.upturn.org/work/calculated-need/

https://www.upturn.org/work/calculated-need/

Appears in: PAN framework development

Grounds: deployment audit: Arkansas ARChoices / Idaho Medicaid

EmpiricalRetrieval layers propagate rather than sanitize their inputs: studies find retrieval-augmented systems remain …

Retrieval layers propagate rather than sanitize their inputs: studies find retrieval-augmented systems remain unfaithful even when the retrieved passage is correct, so faithfulness is bounded rather than assured.

faithfulragGroundingPeer-reviewed

FaithfulRAG (arXiv:2506.08938) — RAG systems struggle in knowledge-conflict scenarios even when relevant passages are retrieved (pessimistic end).

Grounds: empirical cap: groundtruth_reliability (max)

faithfulragwithsparseautoencGroundingPeer-reviewed

Faithful RAG with Sparse Autoencoders (arXiv:2512.08892) — even with relevant passages retrieved, models contradict evidence / invent details; faithfulness is not guaranteed.

Grounds: empirical cap: groundtruth_reliability (max)

ragevaluationsurveyGroundingPeer-reviewed

RAG evaluation survey (arXiv:2405.07437) — factuality evaluation is bounded by knowledge-base coverage and retrieval accuracy; what is checkable depends on what is documented.

Grounds: empirical cap: frac_verifiable (max)

EmpiricalRestricting retrieval to a curated, vetted document set bounds what re-enters the model: retrieval-augmented s…

Restricting retrieval to a curated, vetted document set bounds what re-enters the model: retrieval-augmented systems fact-checking against a curated peer-reviewed corpus reach roughly 0.97+ accuracy and factuality evaluation is limited by knowledge-base coverage — what is checkable depends on what is documented — so a vetted corpus reduces contamination drawn back into the model relative to open retrieval, though faithfulness remains imperfect under knowledge conflict.

retrievalaugmentedcovidfactcGroundingPeer-reviewed

Retrieval-augmented COVID-19 fact-checking (PMC12079058) — CRAG/Self-RAG reach 0.972-0.978 accuracy against a curated 130k peer-reviewed corpus (optimistic ceiling).

Grounds: empirical cap: groundtruth_reliability (max)

ragevaluationsurveyGroundingPeer-reviewed

RAG evaluation survey (arXiv:2405.07437) — factuality evaluation is bounded by knowledge-base coverage and retrieval accuracy; what is checkable depends on what is documented.

Grounds: empirical cap: frac_verifiable (max)

faithfulragGroundingPeer-reviewed

FaithfulRAG (arXiv:2506.08938) — RAG systems struggle in knowledge-conflict scenarios even when relevant passages are retrieved (pessimistic end).

Grounds: empirical cap: groundtruth_reliability (max)

EmpiricalAutomated output checks are partial, not complete — measured detector-accuracy bands sit well below completene…

Automated output checks are partial, not complete — measured detector-accuracy bands sit well below completeness, especially on hard or adversarial content.

theillusionofprogressGroundingPeer-reviewed

'The Illusion of Progress' (arXiv:2508.08285) — LLM-as-Judge Precision 0.736 / Recall 0.957 / F1 0.832 vs human consensus on QA.

Grounds: empirical cap: catch_at_generation (max)

halogenGroundingPeer-reviewed

HALoGEN (arXiv:2501.08292) — best models hallucinate 4%-86% of generated facts depending on domain.

Grounds: empirical cap: model_error_base (min)

datadogllmasajudge2025GroundingIndustry

Datadog LLM-as-a-judge (2025) — detection F1 drops substantially from HaluBench to the harder RAGTruth; harder hallucinations are harder to catch.

Grounds: empirical cap: catch_at_generation (max)

mentalhealthchatbotdetectionGroundingPeer-reviewed

Mental-health chatbot detection (arXiv:2604.06216) — GPT judges 54.6% accuracy, 9.3% recall (miss 90.7% of hallucinations); traditional methods F1<0.30 on subjective content.

Grounds: empirical cap: catch_at_generation (max)

EmpiricalRanked risk lists steered which cases were investigated in documented deployments; the anchoring direction is …

Ranked risk lists steered which cases were investigated in documented deployments; the anchoring direction is documented while its magnitude is not published.

wiredlighthousereports2023GroundingInvestigative

WIRED / Lighthouse Reports, Inside the suspicion machine (2023) https://www.wired.com/story/welfare-state-algorithms/

https://www.wired.com/story/welfare-state-algorithms/

Appears in: PAN framework development

Grounds: deployment audit: Rotterdam welfare-fraud algorithm

amnestyinternational2021GroundingAdvocacy

Amnesty International, Xenophobic machines: Discrimination through unregulated use of algorithms in the Dutch childcare benefits scandal (2021) https://www.amnesty.org/en/documents/eur35/4686/2021/en/

https://www.amnesty.org/en/documents/eur35/4686/2021/en/

Appears in: PAN framework development

Grounds: deployment audit: SyRI / childcare-benefits (toeslagenaffaire)

EmpiricalModel error has a hard nonzero floor: formal impossibility results rule out zero error, and measured floors ru…

Model error has a hard nonzero floor: formal impossibility results rule out zero error, and measured floors run roughly 1.6–11.6% in frontier evaluations and 4–86% across domains.

xuetal2024GroundingPeer-reviewed

Xu et al. (2024), 'Hallucination is Inevitable: An Innate Limitation of LLMs' — formal proof that hallucination cannot be eliminated.

Grounds: empirical cap: model_error_base (min)

karpowicz2025GroundingPeer-reviewed

Karpowicz (2025) — three independent mathematical frameworks (auction theory, proper scoring, log-sum-exp) all conclude no LLM inference mechanism can be simultaneously truthful, etc.

Grounds: empirical cap: model_error_base (min)

halogenGroundingPeer-reviewed

HALoGEN (arXiv:2501.08292) — best models hallucinate 4%-86% of generated facts depending on domain.

Grounds: empirical cap: model_error_base (min)

openai2025GroundingFrontier lab

OpenAI (2025), 'Why Language Models Hallucinate' — next-token training plus IDK-penalizing benchmarks push models to bluff; explains the persistent nonzero floor.

Grounds: empirical cap: model_error_base (min)

llmstats2026GroundingIndustry evaluation

llm-stats.com failure-focused eval (2026) — FactsGrounding 89.1% accuracy => ~10.9% failure on a relatively easy grounded benchmark.

Grounds: empirical cap: model_error_base (min)

suprmindbenchmarkdigest2026GroundingIndustry evaluation

Suprmind benchmark digest (2026) — production ChatGPT ~4.8% major-incorrect with reasoning vs ~11.6% without; HealthBench 3.6%->1.6% with GPT-5 thinking.

Grounds: empirical cap: model_error_base (min)

EmpiricalAutomated catch fractions cap out below completeness — around 84% balanced accuracy in optimistic settings ver…

Automated catch fractions cap out below completeness — around 84% balanced accuracy in optimistic settings versus about 55% on hard content and 9.3% recall in worst-case measurements.

faithfulragleaderboardGroundingPeer-reviewed

Faithful RAG leaderboard (arXiv:2505.04847) — FaithJudge with o3-mini-high reaches ~84% balanced accuracy / ~82% F1 on FaithBench (optimistic ceiling).

Grounds: empirical cap: catch_at_generation (max)

theillusionofprogressGroundingPeer-reviewed

'The Illusion of Progress' (arXiv:2508.08285) — LLM-as-Judge Precision 0.736 / Recall 0.957 / F1 0.832 vs human consensus on QA.

Grounds: empirical cap: catch_at_generation (max)

mentalhealthchatbotdetectionGroundingPeer-reviewed

Mental-health chatbot detection (arXiv:2604.06216) — GPT judges 54.6% accuracy, 9.3% recall (miss 90.7% of hallucinations); traditional methods F1<0.30 on subjective content.

Grounds: empirical cap: catch_at_generation (max)

datadogllmasajudge2025GroundingIndustry

Datadog LLM-as-a-judge (2025) — detection F1 drops substantially from HaluBench to the harder RAGTruth; harder hallucinations are harder to catch.

Grounds: empirical cap: catch_at_generation (max)

samedetectionaccuracyliteratGroundingPeer-reviewed

Same detection-accuracy literature as catch_at_generation (FaithBench arXiv:2410.13210; arXiv:2508.08285); audit-time detection is bounded by the same hallucination-detection ceiling.

Grounds: empirical cap: catch_at_generation (max); empirical cap: decontaminate (max)

EmpiricalRecord audit-and-correct shares the detection-ceiling family: an optimistic anchor near 96% token accuracy fal…

Record audit-and-correct shares the detection-ceiling family: an optimistic anchor near 96% token accuracy falls away on hard content, so decontamination is bounded rather than total.

halludetectlegaldomainGroundingPeer-reviewed

HalluDetect legal-domain (arXiv:2509.11619) — best mitigation architecture reaches ~96% token accuracy in a FAVORABLE, retrieval-grounded legal setting (optimistic end).

Grounds: empirical cap: decontaminate (max)

samedetectionaccuracyliteratGroundingPeer-reviewed

Same detection-accuracy literature as catch_at_generation (FaithBench arXiv:2410.13210; arXiv:2508.08285); audit-time detection is bounded by the same hallucination-detection ceiling.

Grounds: empirical cap: catch_at_generation (max); empirical cap: decontaminate (max)

theillusionofprogressGroundingPeer-reviewed

'The Illusion of Progress' (arXiv:2508.08285) — LLM-as-Judge Precision 0.736 / Recall 0.957 / F1 0.832 vs human consensus on QA.

Grounds: empirical cap: catch_at_generation (max)

EmpiricalThe verification channel itself is bounded — curated-corpus fact-checking tops out around 0.972–0.978 reliabil…

The verification channel itself is bounded — curated-corpus fact-checking tops out around 0.972–0.978 reliability and collapses under knowledge conflict.

retrievalaugmentedcovidfactcGroundingPeer-reviewed

Retrieval-augmented COVID-19 fact-checking (PMC12079058) — CRAG/Self-RAG reach 0.972-0.978 accuracy against a curated 130k peer-reviewed corpus (optimistic ceiling).

Grounds: empirical cap: groundtruth_reliability (max)

faithfulragGroundingPeer-reviewed

FaithfulRAG (arXiv:2506.08938) — RAG systems struggle in knowledge-conflict scenarios even when relevant passages are retrieved (pessimistic end).

Grounds: empirical cap: groundtruth_reliability (max)

faithfulragwithsparseautoencGroundingPeer-reviewed

Faithful RAG with Sparse Autoencoders (arXiv:2512.08892) — even with relevant passages retrieved, models contradict evidence / invent details; faithfulness is not guaranteed.

Grounds: empirical cap: groundtruth_reliability (max)

AssumptionThe verifiable fraction of contaminated records is a planning range (0.90/0.60/0.30) that is explicitly calibr…

The verifiable fraction of contaminated records is a planning range (0.90/0.60/0.30) that is explicitly calibration-required and has never been measured.

ragevaluationsurveyGroundingPeer-reviewed

RAG evaluation survey (arXiv:2405.07437) — factuality evaluation is bounded by knowledge-base coverage and retrieval accuracy; what is checkable depends on what is documented.

Grounds: empirical cap: frac_verifiable (max)

EmpiricalIn the documented MiDAS case, error among no-review auto-adjudications ran roughly 93%, and determinations err…

In the documented MiDAS case, error among no-review auto-adjudications ran roughly 93%, and determinations erred at about 85% without human review versus 44% with it.

aiincidentdatabaseGroundingInvestigative

AI Incident Database, Incident 373 (MiDAS false fraud claims) https://incidentdatabase.ai/cite/373/

https://incidentdatabase.ai/cite/373/

Grounds: model org: michigan_midas

EmpiricalIn the documented AFST evaluation, screener overrides of the tool — roughly a third of its recommendations — c…

In the documented AFST evaluation, screener overrides of the tool — roughly a third of its recommendations — cut screen-in disparity from about 20% to 9% relative to the tool acting alone.

stapletonetal2022GroundingPeer-reviewed

Stapleton et al., Imagining new futures beyond predictive systems in child welfare (FAccT 2022) https://dl.acm.org/doi/10.1145/3531146.3533177

https://dl.acm.org/doi/10.1145/3531146.3533177

Appears in: PAN framework development

Grounds: deployment audit: Allegheny AFST

Topics: child-welfare

EmpiricalProfessional caseload standards published by the Child Welfare League of America recommend no more than about …

Professional caseload standards published by the Child Welfare League of America recommend no more than about 15 families per worker, sitting well below documented practice loads.

EmpiricalDocumentation and administrative tasks consume up to 45% of practitioner hours in documented accounts, where '…

Documentation and administrative tasks consume up to 45% of practitioner hours in documented accounts, where 'up to' marks a ceiling rather than a typical value.

communitycare2025aAcademic

Community Care. (2025). Social work and AI: Survey findings 2025. Community Care Publications.

Appears in: 1023AI authored research

Topics: social-work

jones2024aAcademic

Jones, C. (2024). The future of social work in an age of artificial intelligence. Social Work Today.

Appears in: 1023AI authored research

Topics: social-work

EmpiricalTiered HIPAA penalties run from $145 to $73,011 per violation with an annual cap near $2.19M (2025-adjusted), …

Tiered HIPAA penalties run from $145 to $73,011 per violation with an annual cap near $2.19M (2025-adjusted), and disclosure to a tool that is not a business associate is itself a violation.

hipaajournal2026bAcademic

HIPAA Journal. (2026). HIPAA violation penalties. HIPAA Journal.

Appears in: 1023AI authored research

Topics: privacy-security

EmpiricalCited per-unit intensities of roughly 0.3 Wh per inference call and about 3.14 L of water per kWh are applied …

Cited per-unit intensities of roughly 0.3 Wh per inference call and about 3.14 L of water per kWh are applied to authored illustrative volumes rather than to measured deployment totals.

jegham2025Academic

Jegham, N., et al. (2025). How Hungry is AI? Benchmarking energy, water, and carbon footprint of LLM inference [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2505.09598

doi.org/10.48550/arXiv.2505.09598

Appears in: PAN framework development

Topics: ai-governance

li2023Academic

Li, P., Yang, J., Islam, M. A., & Ren, S. (2023). Making AI Less Thirsty: Uncovering and addressing the secret water footprint of AI models [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2304.03271

doi.org/10.48550/arXiv.2304.03271

Appears in: PAN framework development

Topics: ai-governance

EmpiricalA preprint benchmark reports an in-context misalignment dose-response: 16 examples yielded about 24% and 256 e…

A preprint benchmark reports an in-context misalignment dose-response: 16 examples yielded about 24% and 256 examples about 58% misaligned behavior, with the majority rationalized.

afonin2026Academic

Afonin, N., Andriianov, N., Hovhannisyan, V., Bageshpura, N., Liu, K., Zhu, K., Dev, S., Panda, A., Rogov, O., Tutubalina, E., Panchenko, A., & Seleznyov, M. (2026). Emergent misalignment via in-context learning: Narrow in-context examples can produce broadly misaligned LLMs [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2510.11288

doi.org/10.48550/arXiv.2510.11288

Appears in: 1023AI authored research

Topics: ai-alignment, complexity-science

EmpiricalModel behavior drifts discontinuously between evaluation snapshots, and narrow finetuning can induce broad cor…

Model behavior drifts discontinuously between evaluation snapshots, and narrow finetuning can induce broad correlated failure across unrelated tasks.

betley2026Academic

Betley, J., Warncke, N., Sztyber-Betley, A., Tan, D., Bao, X., Soto, M., Srivastava, M., Labenz, N., & Evans, O. (2026). Training large language models on narrow tasks can lead to broad misalignment. Nature, 649(8097), 584-589. https://doi.org/10.1038/s41586-025-09937-5

doi.org/10.1038/s41586-025-09937-5

Appears in: 1023AI authored research

Topics: ai-alignment

li2026Academic

Li, Z., Fan, C., & Zhou, T. (2026). Grokking in LLM pretraining? Monitor memorization-to-generalization without test [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2506.21551

doi.org/10.48550/arXiv.2506.21551

Appears in: 1023AI authored research

song2026Academic

Song, P., Han, P., & Goodman, N. (2026). Large language model reasoning failures [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2602.06176

doi.org/10.48550/arXiv.2602.06176

Appears in: 1023AI authored research

anwar2024Academic

Anwar, U., Saparov, A., Rando, J., Paleka, D., Turpin, M., Hase, P., Lubana, E., Jenner, E., Casper, S., Sourbut, O., Edelman, B. L., Zhang, Z., Gunther, M., Korinek, A., Hernandez-Orallo, J., Hammond, L., Bigelow, E., Pan, A., Langosco, L., Korbak, T., Zhang, H., Zhong, R., O Heigeartaigh, S., Recchia, G., Corsi, G., Chan, A., Anderljung, M., Edwards, L., Petrov, A., de Witt, C. S., Motwani, S. R., Bengio, Y., Chen, D., Torr, P. H. S., Albanie, S., Maharaj, T., Foerster, J., Tramer, F., He, H., Kasirzadeh, A., Choi, Y., & Krueger, D. (2024). Foundational challenges in assuring alignment and safety of large language models. Transactions on Machine Learning Research. https://doi.org/10.48550/arXiv.2404.09932

doi.org/10.48550/arXiv.2404.09932

Appears in: 1023AI authored research

Topics: ai-alignment, ai-safety

nikolaou2025Academic

Nikolaou, K., Krippendorf, S., Tovey, S., & Holm, C. (2025). Beyond scaling curves: Internal dynamics of neural networks through the NTK lens [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2507.05035

doi.org/10.48550/arXiv.2507.05035

Appears in: 1023AI authored research

Topics: complexity-science

Why a governance lab at all

The Lab draws on published PAN findings — the headline one: the same AI embedded in three modeled office cultures — a modeled office, not a real one — produced error rates of roughly 75%, 20%, and 16% in the published runs[]. The governance context, not the model, drove that gap — and that is the pattern you play against here.

Environmental figures use published data current as of early 2026 to show scale — not the measured footprint of any real deployment.