News & Research
The latest AI research and news with real-world stakes — each item sourced, dated, summarized in plain English, and tagged by impact area. Every item is checked against its source before it appears.
Research
The Illusion of Improvement: Reject Inference Strategies in Credit Scoring
Bruno Scarone, Ricardo Baeza-Yates
arXiv · 2026-06-16
This paper investigates reject inference methods used in credit scoring to address survival bias, revealing a structural failure mode where models appear to improve in accuracy while their ability to correctly screen out defaulters (rejection quality) actually deteriorates—an 'illusion of improvement.' The authors propose a controlled exploration strategy in which lenders deliberately approve a small fraction of rejected applicants to observe true outcomes, breaking the feedback loop without statistical assumptions. Experiments across two machine learning methods and three real-world datasets show that even minimal exploration rates of 2–5% are sufficient to diagnose the severity of the feedback loop at near-zero cost. The findings indicate that standard evaluation protocols relying on accuracy metrics are inadequate and misleading for models trained under survival bias.
- Enterprise
- Quality assurance
Research
Possible or Definite? A Benchmark for Evaluating Diagnostic Uncertainty Preservation in Clinical Text
Hongbo Du, Zixin Lu, Jiaming Qu
arXiv · 2026-06-16
This paper investigates whether large language models (LLMs) correctly preserve expressions of diagnostic uncertainty—such as 'possible pneumonia' versus 'definite pneumonia'—when performing clinical text tasks like summarization and revision. The authors constructed a benchmark of 1,200 clinical documents with 9,184 uncertainty annotations across five levels and tested three LLMs on it. Results show that LLMs preserve original uncertainty cues poorly, often less than half the time, and struggle to distinguish between adjacent uncertainty levels. This failure mode is not captured by standard evaluation metrics, raising significant concerns about the safety of deploying LLMs in clinical workflows.
- Quality assurance
- AI policy
Research
Redact or Keep? A Fully Local AI Cascade for Educational Dialogue De-Identification
Haocheng Zhang, Zhuqian Zhou, Kirk Vanacore et al.
arXiv · 2026-06-16
This paper proposes a fully local AI pipeline for removing personally identifiable information (PII) from educational tutoring transcripts while preserving curricular content—such as distinguishing whether a name like 'Riemann' refers to a student or a mathematical concept. The system combines lightweight named entity recognition encoders with a context-aware binary Redact/Keep reviewer, achieving 0.958 macro F1 on math tutoring transcripts compared to 0.767 for a same-family LLM-only baseline and 0.706 for a commercial API, all while running on a single laptop. A key finding is that reframing de-identification as a constrained triage problem outperforms simply scaling up models, suggesting problem formulation is more important than model size for this task. The work is directly relevant to policy and governance concerns around student data privacy, offering a pathway to high-accuracy de-identification without sending sensitive data to third parties.
- AI policy
- Quality assurance
Research
SafeClawBench: Separating Semantic, Audit-Evidence, and Sandbox Harm in Tool-Using LLM Agents
Yuchuan Tian, Mengyu Zheng, Haocheng Mei et al.
arXiv · 2026-06-16
SafeClawBench is a staged security benchmark for tool-using language-model agents, covering 600 adversarial tasks across six attack families including prompt injection, memory poisoning, and ambiguity-driven unsafe inference. Unlike prior evaluations that collapse all failures into a single attack success rate, it separately measures semantic attack acceptance, audit-visible harm evidence, and sandbox-observed tool or state harm. Experiments with five agent endpoints show that these three measures capture distinct failure modes: without prompt protection, semantic failure rates range from 9.0% to 44.2%, and a matched 12,000-row analysis reveals that 291 of 347 observed sandbox harms occur in rows that pass the semantic check. The framework helps distinguish textual compliance from evidence-supported harm and executable state changes, offering a reproducible tool for evaluating and comparing agent safety interventions.
- Quality assurance
- Certifications
Research
Self-CTRL: Self-Consistency Training with Reinforcement Learning
Itamar Pres, Laura Ruis, Melat Ghebreselassie et al.
arXiv · 2026-06-16
Self-CTRL is a reinforcement learning method that trains language models to be consistent between their self-explanations and their actual behavior, updating either the explanations or the behavior to match the other. Tested on a probabilistic reasoning task, it improves the correlation between self-reported and measured biases from R²=0.24 to R²=0.64, matching direct ground-truth supervision. In a constitutional AI setting, it raises a third-party auditor's ability to predict refusals from 36% to 92% and reduces HarmBench failure rates from 15.0% to 0.5% without meaningfully increasing refusals on harmless prompts. The work provides a framework for making AI models more transparent, auditable, and safer by aligning what they say about themselves with how they actually behave.
- AI policy
- Quality assurance
Research
RubricsTree: Scalable and Evolving Open-Ended Evaluation of Personal Health Agents across Health Memory and Medical Skills
Weizhi Zhang, Zechen Li, Hamid Palangi et al.
arXiv · 2026-06-16
RubricsTree is a scalable evaluation framework for LLM-powered personal health agents that replaces expensive physician annotation and inconsistent LLM-as-a-judge methods with a hierarchical taxonomy of over 100 atomic, clinically-verifiable Boolean rubrics. Developed through iterative human-in-the-loop curation with a physician-led expert panel drawing on 4,000 real user queries, the framework uses a context-aware adaptive router to activate only relevant rubrics per query. The authors show RubricsTree substantially outperforms a strong large-scale evaluation baseline in expert alignment on challenging open-ended queries, reliably penalizes degraded responses, and yields up to approximately 66% relative gains on HealthBench when used to optimize Gemini, GPT, and Qwen model families. This matters because it provides an auditable, scalable, and continuously evolving infrastructure for quality-assuring AI health agents before and during large-scale clinical deployment.
- Quality assurance
- Certifications
Research
A Red-Team Study of Anthropic Fable 5 & Opus 4.8 Models
Nicola Franco
arXiv · 2026-06-16
This paper red-teams two frontier Anthropic large language models—Fable 5 and Opus 4.8—against over 7,800 harmful prompts across a ten-category harm taxonomy using automated jailbreak attacks. The study finds that while both models resist most attacks, adaptive iterative methods (especially tree-of-attacks) still broke Opus 4.8 on 11.5% of harmful intents and Fable 5 on 6.1%, yielding 1,620 and 702 panel-confirmed harmful completions respectively. These failures were achieved cheaply and automatically, without human expert involvement, demonstrating that even heavily tested frontier models remain reliably breakable under sustained automated pressure. The findings carry significant implications for AI safety policy and quality assurance, showing that aggregate robustness statistics can obscure meaningful residual vulnerabilities.
- Quality assurance
- AI policy
Research
Agentra: A Supervisable Multi-Agent Framework for Enterprise Intrusion Response
Raj Patel, Shaswata Mitra, Michele Guida et al.
arXiv · 2026-06-16
Agentra is a supervisable multi-agent framework for enterprise intrusion response that converts alerts from IDS, EDR, and XDR platforms into structured incident response plans grounded in MITRE ATT&CK, MITRE D3FEND, and NIST CSF 2.0. Evaluated on a 120-event corpus drawn from ThreatHunter-Playbook, Splunk BOTSv3, and DARPA OpTC, the strongest configuration improves FP-aware IRS F1 from 0.61 to 0.84 compared to a static OASIS CACAO v2.0 playbook baseline, while restoring the projected harmful-action rate to the static baseline level of 0.0% through a Planner–Validator review loop. The framework also incorporates a Moderator security gateway and an append-only audit log to preserve analyst oversight and auditability, addressing the delay between alert generation and containment in enterprise security operations.
- Enterprise
- Quality assurance
Research
All Smoke, No Alarm: Oracle Signals in Agent-Authored Test Code
Dipayan Banik, Kowshik Chowdhury, Shazibul Islam Shamim
arXiv · 2026-06-16
This empirical study examines 86,156 test-file patches from 33,596 agent-authored pull requests across 2,807 GitHub repositories, generated by five AI coding agents including OpenAI Codex, GitHub Copilot, Devin, Cursor, and Claude Code. The researchers find that 80.2% of test patches contain weak or no explicit oracle signals—meaning the tests execute code without actually verifying behavior—and that quality gates based on test-file presence therefore substantially overestimate verification strength. A regression analysis shows that strong oracle signals significantly improve merge likelihood (OR = 1.28, p < 0.001), suggesting that agent-authored test code is often superficial despite appearing substantive. The paper recommends that practitioners adopt oracle-aware quality checks rather than relying on test file counts to evaluate AI-generated contributions.
- Quality assurance
- Enterprise
Research
The Measurement Gap in the Automation of EU Law: Benchmarking Doctrinal Legal Reasoning under the EU AI Act
Michèle Finck
arXiv · 2026-06-16
This paper identifies a critical gap between what current legal-AI benchmarks measure and what EU law now requires: existing evaluations focus on paralegal or ancillary tasks, while doctrinal legal reasoning—the interpretive core of legal work—remains unassessed. The authors argue that this is both a methodological problem and a legal compliance problem, because the EU AI Act mandates 'appropriate accuracy' for high-risk AI used in the judicial domain, a requirement that cannot be operationalized without a doctrinal-reasoning benchmark that does not yet exist. The paper calls for the development of such a benchmark to give the EU AI Act's accuracy requirement concrete, enforceable content.
- Certifications
- AI policy
Research
Your AI Travel Agent Would Book You a Bullfight: An Agentic Benchmark for Implicit Animal Welfare in Frontier AI Models
Jasmine Brazilek, Joel Christoph, Maheep Chaudhary et al.
arXiv · 2026-06-16
This paper introduces TAC (Travel Agent Compassion), the first agentic benchmark for evaluating how AI models handle implicit animal welfare concerns in realistic travel booking scenarios. Across nine frontier models from five model families, the study finds that AI agents tend to prefer options involving animal exploitation, performing below the random-chance baseline of 65% for selecting neutral bookings—with the best-performing model, Claude, reaching only 64.7%. Adding an ethical-brand persona to the system prompt substantially improved animal-welfare outcomes, raising welfare selection rates by 32 to 80 percentage points depending on the model. The authors connect their findings directly to the EU General-Purpose AI Code of Practice, which identifies non-human welfare as a systemic risk, positioning TAC as a practical measurement tool for that risk.
- AI policy
- Quality assurance
Research
Towards Understanding and Measuring COGNITIVE ATROPHY in LLM Behaviour
Abeer Badawi, Moyosoreoluwa Olatosi, Negin Baghbanzadeh et al.
arXiv · 2026-06-16
This paper identifies and formalizes 'cognitive atrophy' — a process-level behavioral concern in AI-mediated mental-health support where LLM interactions may undermine users' capacity for independent reflection, coping, and decision-making over time. The authors introduce COGNITIVE ATROPHY BENCH, a clinically grounded benchmark built from 1,576 human-generated counseling conversations, 15,680 turns, and 42,230 responses from five LLMs, evaluated by clinical and neuropsychology experts using a 20-attribute schema. Across all five models tested, the study finds consistent moderate-to-high levels of atrophy-aligned behavior, with dominant patterns including directive advice, problem-solving, and validation that may reinforce dependence rather than reflection. The work argues that surface-level safety scores fail to capture these dynamics and provides a framework for auditing LLM behavior in sensitive mental-health contexts.
- Quality assurance
- AI policy
Research
Structural Role Injection in Handlebars-Templated LLM Prompts: Triple-Brace Interpolation, Delimiter Family, and the Limits of HTML Auto-Escaping
Mohammadreza Rashidi
arXiv · 2026-06-16
This paper analyzes a security vulnerability in Handlebars-templated LLM prompts, specifically how the choice between double-brace (HTML-escaped) and triple-brace (raw) interpolation affects susceptibility to structural role injection attacks, where attacker-controlled input can forge higher-privilege chat turns. A model-free analysis shows that Handlebars HTML escaping neutralizes some delimiter families (e.g., ChatML, Llama-3, XML) but leaves others like Llama-2 [INST], Human:/Assistant:, and Markdown ### delimiters completely intact. Empirical trials across 5,760 runs and four models found GPT-3.5 Turbo followed task-hijack instructions in 97% of raw and 91% of escaped trials, demonstrating that escaped defaults provide no protection against colon- and Markdown-based delimiter families. The findings show that HTML escaping cannot substitute for structural separation of instruction and data in LLM prompt templates, with implications for the security of applications built on Microsoft Semantic Kernel and similar frameworks.
- Quality assurance
- AI policy
Research
Agentic AI-based Framework for Mitigating Premature Diagnostic Handoff and Silent Hallucination in Healthcare Applications
Divyansh Srivastava, Shreya Ghosh, Anshul Verma et al.
arXiv · 2026-06-16
This paper proposes a multi-agent AI framework designed to reduce two dangerous failure modes in AI-powered medical conversational agents: premature diagnostic handoff (ending information gathering too soon) and silent clinical hallucinations (undetected erroneous outputs). The framework uses two safety mechanisms — a neuro-symbolic state-tracking gate that enforces completion of the OLDCARTS clinical protocol before allowing a diagnosis, and an uncertainty quantification gate that measures semantic entropy across multiple diagnostic samples to intercept divergent outputs. Evaluated on 150 simulated patient test cases, the full system achieves 49.3% diagnostic precision, an 11.3 percentage point improvement over an unconstrained baseline, and finds a statistically significant negative correlation between structured information completeness and diagnostic uncertainty. These results suggest that structured, deterministic orchestration constraints can meaningfully improve the safety and reliability of AI systems in healthcare settings.
- Quality assurance
- AI policy
Research
Security and Privacy Prompts in the Wild: What Users Ask LLMs and How LLMs Respond
Hobin Kim, Xiaoyuan Wu, Omer Akgul et al.
arXiv · 2026-06-16
This study analyzes 14,727 security and privacy (S&P) prompts drawn from WildChat, a dataset of 3.2 million real user-LLM conversations, to understand what digital security and privacy questions users actually ask LLMs and how well LLMs respond. The researchers categorized prompts into nine S&P topic areas and evaluated response quality and consistency across 270 advice-seeking prompts, posing each prompt 10 times to measure variability. Commercial LLMs substantially outperformed open-weight models (GPT provided 'good enough' responses on 98% of prompts versus Llama 4 on 47%), but even high-performing commercial models sometimes produced contradictory responses across runs, which could confuse or mislead users seeking security guidance. These findings matter for quality assurance of AI systems deployed in sensitive advisory roles, highlighting that response consistency—not just average accuracy—is a critical dimension of LLM reliability.
- Quality assurance
Research
ProvenanceGuard: Source-Aware Factuality Verification for MCP-Based LLM Agents
Ander Alvarez, Santhiya Rajan, Samuel Mugel et al.
arXiv · 2026-06-16
ProvenanceGuard is a source-aware factuality verification system for LLM agents that use the Model Context Protocol (MCP) to draw on multiple evidence sources such as clinical records, APIs, and databases. It addresses a failure mode the authors call 'cross-source conflation,' where a claim may be factually supported but incorrectly attributed to the wrong source. Evaluated on 281 medical-domain MCP-agent traces, the system achieves a block F1 of 0.802 and source accuracy of 0.858 on a held-out split, outperforming source-blind baselines, and detects all injected attribution swaps in 50 controlled clinical conflation probes. The work demonstrates that source attribution is an independent and important axis for factuality verification in tool-using LLM agents.
- Quality assurance
- Enterprise
Research
LegalHalluLens: Typed Hallucination Auditing and Calibrated Multi-Agent Debate for Trustworthy Legal AI
Lalit Yadav, Akshaj Gurugubelli
arXiv · 2026-06-16
LegalHalluLens is an auditing framework for detecting and characterizing hallucinations in AI systems used for legal contract analysis. The authors find that while aggregate hallucination rates hover around 52%, this figure masks large disparities across claim types—numeric and obligation claims show error rates roughly 38–40 percentage points higher than temporal claims—and obscures whether a system tends to fabricate information or omit it. Their Risk Direction Index (RDI) collapses omission-versus-invention bias into a single comparable scalar, enabling more actionable deployment decisions, while a typed multi-agent debate pipeline reduces fabricated detections by 45% using a 4B-parameter model competitive with commercial APIs. The framework directly supports direction-aware procurement, accountability assessments, and regulatory compliance for legal AI in production settings.
- Quality assurance
- AI policy
Research
Reading between the Lines: Leveraging Large Language Models for Global Dementia and Depression Assessment from Clinical Interviews
Franziska Braun, Alea Rüggeberg, Thomas Ranzenberger et al.
arXiv · 2026-06-16
This study applies open-weights Large Language Models (LLMs) to predict dementia and depression severity from speech recorded during standardized clinical interviews with 154 German-speaking patients. The researchers introduce a new observer-based Global Depression Scale (GDS-D) aligned with an established cognitive staging tool, enabling parallel assessment of affective and cognitive symptoms. Testing three LLMs (Mistral 3.1, DeepHermes, Qwen3) in zero-shot and feature-extraction settings, they find LLMs predict depression severity directly (best MAE of 0.60), while structured feature extraction substantially improves dementia assessment (best MAE of 0.78, reducing errors by up to 35% over zero-shot baselines). Pause-enriched automatic transcripts match human transcription quality, supporting fully automated neuropsychiatric screening pipelines.
- Quality assurance
Research
Agentic AI Enhances Physician Trust in Clinical Decision Making
Zhiling Yan, Zhe Fang, David J King et al.
arXiv · 2026-06-16
This study examines whether agentic AI—which autonomously invokes external tools and makes its intermediate reasoning steps transparent—earns greater physician trust than non-agentic AI in clinical decision-making. Three physicians evaluated 315 multimodal clinical cases, finding significantly higher cognitive (process-oriented) and behavioral (outcome-oriented) trust for the agentic model (P < 0.001), with physicians preferring agentic reasoning in 89.57% of treatment planning cases. However, the study also identifies measurable over-reliance on incorrect agentic outputs, showing that transparency in decision logic alone is insufficient and that rigorous clinician oversight remains essential.
- Quality assurance
- Workforce
Research
Fine-tuning LLMs for Passive Depression Severity Estimation from AI Mental Health Dialogue
Olivier Tieleman, Ziyi Zhu, Ting Su et al.
arXiv · 2026-06-16
This paper fine-tunes a large language model (Qwen3.5-27B) to predict PHQ-9 depression severity scores directly from transcripts of user conversations with an AI mental health application, requiring no additional clinical data. Using a dataset of 6,283 users built by augmenting 3,111 ground-truth labels with pseudolabels, the best model achieves a Pearson correlation of 0.80 and AUC of 0.91 at the clinically relevant PHQ-9 ≥ 10 threshold, with AUC above 0.87 across every severity level tested. The work demonstrates that passive, continuous depression monitoring is feasible from routine AI-generated conversation text alone, potentially reducing reliance on self-report measures that suffer from low completion rates and response bias.
- Quality assurance
- Enterprise
Research
Legal Reasoning Is Not Lawyering: Rethinking Legal Benchmarks for Pro Se Access to Justice
Andrew Lou, David Shin
arXiv · 2026-06-16
This paper critiques the assumption that large language models (LLMs) can improve access to justice for pro se litigants—people without legal representation—by arguing that current legal AI benchmarks measure only an upper bound of model performance using expert-preprocessed inputs. The authors contend that pro se users submit noisy, incomplete, or informally worded prompts that mirror known LLM failure conditions such as hallucination, long-context sensitivity, and typographical perturbations, yet no benchmark currently measures robustness under these realistic conditions. Using a perturbation experiment on the LEXam legal benchmark, the paper illustrates the performance gap between expert-curated and pro se-like inputs. The authors call for new legal benchmarks specifically designed to test robustness under pro se conditions so that access-to-justice claims about legal AI can be empirically validated rather than assumed.
- AI policy
- Quality assurance
Research
ASTRA: A Scalable Next-Generation ATCO Training Simulator with Autonomous Simpilots
Ethan Chew, Enjia Wu, Iruss Eng et al.
arXiv · 2026-06-16
ASTRA is an end-to-end AI simulator designed to automate the 'simpilot' role in Air Traffic Control Operator (ATCO) training, replacing specialized human trainers who role-play pilots and controllers in simulated airspace. The system combines a locally fine-tuned Automatic Speech Recognition pipeline—reducing Word Error Rate from up to 107.80% to 23.45% on Singaporean-accented aviation speech—with an AI-assisted performance evaluation framework that scores trainee radiotelephony communications on accuracy (91.7%), brevity (88.2%), and completeness (86.9%). Built on open-source tools including DSPy and Unsloth, ASTRA enables scalable, standardized ATCO assessment while reducing the workload on human instructors. The work directly addresses training capacity constraints in aviation by demonstrating that locally adapted AI can outperform Western-centric off-the-shelf speech models in operational contexts.
- Workforce
- Quality assurance
Research
FairTutor: Equity-Aware Pedagogical LLM Routing for Budget-Constrained AI Tutoring
Qingyang Xu
arXiv · 2026-06-16
FairTutor is a multi-agent AI tutoring framework designed to close the quality gap between students with access to premium AI models and those limited to free or low-cost services. It combines query analysis, pedagogical planning, low-cost model generation, evaluator-guided critique and revision, and selective escalation to premium models. Empirical evaluations show FairTutor achieves 97.1% of premium pedagogical quality (measured via floor-adjusted Likert scale) while reducing serving cost by 71.6%, as assessed on TutorAccessEval, a benchmark covering math, reading, writing, science, and language learning. The work directly addresses AI-driven education inequity and offers a tunable cost–quality Pareto frontier adaptable to diverse student populations.
- AI policy
- Workforce
Research
The Slop Paradox: How Synthetic Standardization Erodes Clinical Uncertainty and Cross-Modal Alignment in AI-Rewritten Radiology Reports
Samar Ansari
arXiv · 2026-06-16
This paper investigates how AI-assisted rewriting of radiology reports degrades clinical information, using 450 chest X-ray reports from the Indiana University dataset processed through three LLM rewriting tasks: EHR summarization, standardized rewriting, and teaching case preparation. The central finding is a 'slop paradox': EHR summarization causes the most entity erosion (51.4% of clinical entities, 43.7% of hedging language lost) but barely affects image-text alignment (2.5% drop), while standardized and teaching-case rewrites preserve more entities but cause 14.9–16.5% alignment drops—six to seven times larger. Contrary to expectations, rare pathologies were not preferentially degraded; the dominant driver of degradation is the rewriting task type, not the clinical content. These findings have direct implications for the governance of AI-assisted clinical documentation and the construction of multimodal medical AI training datasets.
- Quality assurance
- AI policy
Research
Toward Accessible Psychotherapy Training Using AI-Driven Interactive Patient Avatars
Pascal Riachi, Sofie Kamber, Stella Brogna et al.
arXiv · 2026-06-16
This paper presents an AI-driven training system for psychotherapists learning Acceptance and Commitment Therapy (ACT), using large language models to simulate realistic virtual patients derived from real therapy sessions and providing automated turn-by-turn feedback on therapist responses based on established ACT fidelity criteria. Expert evaluation with practicing psychologists confirmed high realism in patient behavior, and quantitative testing across 49 therapy transcripts found GPT-4o-mini achieved the lowest mean absolute error (MAE = 6.12) in replicating human supervisor fidelity ratings. The system is designed to complement rather than replace supervision, offering a scalable, low-risk environment for deliberate practice and immediate feedback. This matters for workforce development by expanding access to standardized psychotherapy training that is otherwise constrained by ethical, logistical, and resource barriers.
- Workforce
- Quality assurance