News & Research
The latest AI research and news with real-world stakes — each item sourced, dated, summarized in plain English, and tagged by impact area. Every item is checked against its source before it appears.
Research
ScaleToT: Generalizing Structured LLM Reasoning for Billion-Scale Low-Activity User Modeling
Tianbao Ma, Chang Xi, Yichuan Zou et al.
arXiv · 2026-06-23
ScaleToT is a system that applies structured large language model (LLM) reasoning to model billions of low-activity users who lack rich interaction histories, a setting where standard LLMs are both unreliable and prohibitively expensive to deploy at scale. The approach trains a smaller student model on LLM-curated reasoning chains derived from a small subset of users, then transfers those reasoning signals to a lightweight encoder covering the remaining population without requiring full LLM inference. Evaluated on lifetime value (LTV) prediction in a billion-scale advertising deployment, a randomized online A/B test showed a 6.738% increase in LT30, while offline reasoning covered only 7.32% of the potential population, substantially reducing compute costs. The work demonstrates a practical path to scalable, structured AI reasoning for enterprise-scale user modeling.
- Enterprise
Research
To Compare, or Not to Compare: On Methodological Practices in Evaluating Social Bias
Federico Marcuzzi, Xuefei Ning, Roy Schwartz et al.
arXiv · 2026-06-23
This paper investigates how the structural design of social bias benchmarks—specifically whether evaluations use isolated demographic assessments versus forced-choice comparative settings—dramatically affects conclusions about Large Language Model (LLM) bias. The authors introduce a unified framework to standardize heterogeneous benchmarks and find a large, systematic 'paradigm gap': comparative settings act as strong catalysts for latent discrimination, especially in underspecified contexts, while isolated assessments understate bias. Notably, Chain-of-Thought reasoning worsens bias in comparative settings, neutral fallback options do not eliminate the effect, and this comparative prejudice scales with model size. The work provides a methodological guideline urging researchers to use comparative settings for bias auditing while warning practitioners against comparative deployments in ambiguous real-world tasks.
- Quality assurance
- AI policy
Research
A specialized reasoning large language model for accelerating rare disease diagnosis: a randomized AI physician assistance trial
Haichao Chen, Songchi Zhou, Zhengyun Zhao et al.
arXiv · 2026-06-23
RaDaR is a compact, open-source large language model (32 billion parameters) specialized for rare disease diagnosis, trained on approximately 49,000 real free-text cases and over 104,000 synthetic cases with reasoning-enhanced training. In a randomized physician-assistance trial, RaDaR improved diagnostic accuracy by 21.44 percentage points compared to internet search alone, and in a retrospective cohort it prioritized the correct diagnosis before documented clinical suspicion in 61.06% of cases, corresponding to a potential lead time of 1.87 months. The model outperformed larger open-source models including the 671-billion-parameter DeepSeek-R1 across public benchmarks and four external validation centers. These findings demonstrate that a deployable, data-efficient AI tool can meaningfully accelerate rare disease diagnosis where specialized clinical expertise is scarce.
- Workforce
- Enterprise
Research
The African Language Tax: Quantifying the Cost, Latency, and Context Penalty of Tokenizing African Languages in Frontier LLMs
Olaoye Anthony Somide
arXiv · 2026-06-23
This paper quantifies a structural 'tokenization tax' that speakers of African languages face when using frontier large language models: because tokenizers split African-language text into more subword tokens than equivalent English text, users pay higher inference costs, experience greater latency, and receive far less effective context capacity. Measuring 20 African languages across 11 tokenizers using parallel corpora (FLORES-200+ and SIB-200), the authors find every African language carries a tokenization premium above English, with medians around 1.88x on GPT-5 and extremes reaching 8.92x for N'Ko script—translating directly to up to 8.9x higher inference costs and as little as 11% of English's effective context window. The best available tokenizer (Gemma 4) reduces but does not eliminate the penalty, and the authors release an open measurement tool, leaderboard, and mitigation guidance. The findings reveal a concrete economic and capability disparity encoded at the infrastructure level, falling hardest on speakers who can least afford it.
- Enterprise
- AI policy
Research
LLM Performance on a Real, Double-Marked GCSE Benchmark
Malachy Fox, Kavi Samra, Paul Jung
arXiv · 2026-06-23
This paper introduces a benchmark of 32,534 double-marked real student responses to UK GCSE mock exams, covering 328 questions across five subjects including handwritten work, and evaluates how well large language models (LLMs) agree with human examiners. The authors find that top-performing LLMs agree with examiner consensus at least as closely as examiners agree with each other, across both subjective tasks like English essay marking and complex handwritten mathematics scripts. Agreement is consistent near the examiner consensus line and is not strongly dependent on model size, suggesting cost-effective automated marking is feasible at scale.
- Quality assurance
- Certifications
Research
Detecting AI Coding Agents in Open Source: A Validated Multi-Method Census of 180 Million Repositories
Arsham Khosravani, Audris Mockus
arXiv · 2026-06-23
This paper presents a multi-method framework for detecting AI coding agents in open-source repositories, applied across more than 180 million Git repositories in the World of Code dataset. The study finds that single-signal detection methods—such as bot-account lookup, which prior adoption studies rely on—recover only 3.3% of the commits identified through multi-method detection, representing a 30x relative-recall gap that causes prevalence estimates to be severely underestimated. Claude Code leads in commit volume with over 886,000 commits across 17,295 projects, and commit-deployed agents like Claude Code and Aider are disproportionately associated with maintenance work, while PR-deployed agents like Codex and Cursor appear more in feature work. The findings show that no single detection channel is representative of AI coding agent activity, with a pull-request census missing 79% of commit-detected Claude Code adopters, underscoring the methodological stakes for any policy or workforce analysis relying on such data.
- Workforce
- Enterprise
Research
Beyond Logprobs: A Multi-Signal Confidence Engine for LLM-Based Document Field Extraction
Nitesh Kumar
arXiv · 2026-06-23
ExtractConf is a confidence estimation system for LLM-based document field extraction in high-stakes pipelines such as financial reconciliation, compliance verification, and procurement automation. Rather than relying on token-level log-probabilities or verbalized confidence—which the paper shows collapse toward all-positive behavior and fail to distinguish trustworthy from untrustworthy extractions—ExtractConf fuses signals from two structurally different LLM reading passes (a schema-guided 'Hunter' call and a holistic 'Mapper' call), combined with OCR quality, image quality, and spatial layout features. On the DocILE 55-field invoice benchmark, the system achieves 0.928 ROC AUC and reduces selective prediction risk by 70% over the logprob-mean baseline, enabling 99.1% accuracy at 80% coverage within a human-in-the-loop workflow; zero-shot transfer to CORD receipts achieves 0.858 AUC. This matters for enterprise document automation because it provides a field-level signal for routing uncertain extractions to human review rather than allowing silent errors to propagate through automated pipelines.
- Enterprise
- Quality assurance
Research
When Helpfulness Overrides Causal Caution: Context-Dependent Suppression and Recovery in LLMs
Hiroshi Okumura
arXiv · 2026-06-23
This study investigates whether large language models (LLMs) maintain 'Causal Caution'—the appropriate reluctance to make causal claims when evidence is insufficient—across different prompting contexts. Experiments across 480 trials with four leading LLMs (Claude Sonnet 4.6, Claude Opus 4.7, GPT 5.5, and Gemini 3.1 Pro) found that Causal Caution maintenance rates were 91.7–100.0% in academic contexts but collapsed to 6.7–18.3% in practical advisory contexts, and to just 0.5% when concrete recommendations were requested. Critically, a brief self-correction prompt restored Causal Caution to 71.4–100.0%, suggesting the suppression reflects context-dependent expression rather than a fundamental capability gap. The authors conclude that multi-agent architectures separating proposal generation from causal auditing may be a promising governance design for organizations relying on LLMs for decision support.
- Enterprise
- AI policy
Research
Enhancing Clinician Decision-Making via Uncertainty-Aware Multi-Expert Fusion for Stroke Rehabilitation
Tamim Ahmed, Thanassis Rikakis
arXiv · 2026-06-23
This paper introduces xAARA, an AI system designed to augment clinician decision-making in stroke rehabilitation by automating ARAT (Action Research Arm Test) assessments with calibrated uncertainty and multi-level explanations. Using multi-view video and a Dynamic Bayesian Network that fuses 692 calibrated multimodal models, xAARA achieved 94.2% task accuracy (Cohen's kappa=0.934) and 81.3% movement-phase accuracy (kappa=0.727) across 105 stroke survivors, while reducing predictive uncertainty by 96.1% compared to single-clinician scoring. Four independent clinicians validated the system's assessments and indicated willingness to adopt it, suggesting that uncertainty quantification and clinician-aligned explainability are key to moving automated assessment into real clinical use.
- Workforce
- Quality assurance
Research
AutoSpec: Safety Rule Evolution for LLM Agents via Inductive Logic Programming
Pingchuan Ma, Zhaoyu Wang, Zimo Ji et al.
arXiv · 2026-06-23
AutoSpec is a framework that automatically refines safety rules for LLM-based agents by combining counterexample-guided inductive synthesis (CEGIS) with inductive logic programming (ILP). Starting from expert-designed rules and user-annotated execution traces, it iteratively identifies false positives and false negatives, uses ILP to find discriminating predicates, and revises rules until convergence. Evaluated on 291 execution traces across code execution and embodied agent domains, AutoSpec achieves F1 scores of 0.98 and 0.93, up to 94% false positive reduction, and converges in 4–5 iterations—outperforming heuristic CEGIS by up to 4.8× in F1. The resulting rules are human-readable and auditable, making them suitable for safety-critical deployments where interpretability is essential.
- Quality assurance
- AI policy
Research
OmniPath: A Multi-Modal Agentic Framework for Auditing Wheelchair Accessibility
ASM Mobarak Hossain, Nadim Mahmud, Vaskar Raychoudhury et al.
arXiv · 2026-06-23
OmniPath is a multi-modal agentic framework that combines OpenStreetMap network topology with high-density aerial LiDAR data (USGS 3DEP) to audit wheelchair accessibility in pedestrian environments. The system virtually traverses paths in 0.5-meter increments, measuring running slope, cross slope, and vertical discontinuities against ADA compliance standards to classify hazards from 'Mild' to 'Critical' using a weighted severity score. Validated against 200 physical ground-truth field surveys at the National Mall using stratified random sampling, it achieved F1-scores of 0.60 for Severe and 0.58 for Critical hazard categories. By automating micro-scale accessibility inspection, OmniPath transforms static mapping data into a proactive tool that can anticipate physical barriers for wheelchair users before they travel.
- AI policy
- Quality assurance
Research
A Benchmark for Hallucination Detection in VLMs for Gastrointestinal Endoscopy
Aminu Lawal, Niyoj Oli, Sachin Acharya et al.
arXiv · 2026-06-23
This paper benchmarks nine hallucination detection methods applied to five vision-language models (VLMs) on the Gut-VLM dataset, a gastrointestinal endoscopy Visual Question Answering benchmark with 4,392 test pairs—a clinical domain largely overlooked by prior hallucination research. The white-box method ReXTrust consistently outperforms all alternatives, achieving a peak AUC of 93.0 on MedGemma-4B and an average advantage of 19.5 AUC points over the next-best approach, while black-box and clustering-based gray-box methods can collapse to near-chance performance on some models. The study also identifies 'confident confabulation'—where models hallucinate with high consistency or high token-level probability—as a systemic failure mode that undermines both consistency-based and uncertainty-based detection strategies. These findings matter for clinical quality assurance, as they highlight which detection approaches are reliable enough to help catch AI errors before deployment in GI diagnostic workflows.
- Quality assurance
Research
Securing the Personal AI Frontier: Agent Boundary Assurance for Local and Enterprise AI
Jason Lovell
Zenodo (CERN European Organization for Nuclear Research) · 2026-06-23
This paper proposes Agent Boundary Assurance (ABA), a framework for testing and documenting whether local and enterprise AI agents respect boundaries around data, memory, tool use, model routing, egress, consent, and actions. It introduces three supply-chain trust artifacts—Tool Bill of Materials (TBOM), Skill Bundle Attestation (SBA), and Tool Security Advisory (TSA)—alongside a runtime evidence model to make boundary claims testable and portable across platforms like Apple, Microsoft, and NVIDIA. The work is motivated by the rapid shift of AI inference and agent orchestration toward local execution and the serious vulnerabilities already observed in the Model Context Protocol (MCP). The framework aims to give enterprises structured, auditable evidence for security, legal, compliance, and procurement review without replacing existing platform runtimes.
- Enterprise
- Quality assurance
- Certifications
- AI policy
Research
Securing the Personal AI Frontier: Agent Boundary Assurance for Local and Enterprise AI
Jason Lovell
Zenodo (CERN European Organization for Nuclear Research) · 2026-06-23
This paper introduces Agent Boundary Assurance (ABA), a framework for testing and documenting whether local and enterprise AI agents respect boundaries around data, memory, tool use, model routing, egress, consent, and actions. It proposes three supply-chain trust artifacts—Tool Bill of Materials (TBOM), Skill Bundle Attestation (SBA), and Tool Security Advisory (TSA)—alongside a runtime evidence model to make boundary claims testable and portable across platforms like Apple, Microsoft, and NVIDIA. The work is motivated by the rapid shift of AI inference and agent orchestration toward local and hybrid execution environments, and by early vulnerabilities observed in the widely adopted Model Context Protocol (MCP). The framework aims to give enterprises a structured, cross-vendor discipline for demonstrating compliance, security, and provenance during procurement and legal review.
- Enterprise
- Quality assurance
- Certifications
- AI policy
Research
AI-supported analysis of institutional self-assessment reports: A study on the example of Ankara University (2015-2024)
Gamze YILMAZ, I. N. Askerzade
Computers and Informatics · 2026-06-23
This study develops an AI-supported framework to analyze ten years of Institutional Self-Assessment Reports (ISARs) from Ankara University (2015–2024), combining topic modeling (LDA), sentiment analysis, clustering (K-Means), anomaly detection (Isolation Forest), and LSTM forecasting. The analysis reveals a thematic evolution from administrative processes toward research and societal contribution, identifies 2020 as a statistical anomaly linked to the pandemic, and projects growth in institutional performance indicators through 2025. The framework demonstrates how AI techniques can transform static compliance documents into proactive strategic governance tools for higher education institutions, reducing subjectivity and time costs of manual evaluation.
- Quality assurance
- Certifications
- AI policy
- Enterprise
Research
AEGIS Algebra: A Unified Mathematical Framework for Executable Assurance of Agentic AI Artifacts
Dattaram Miruke
Zenodo (CERN European Organization for Nuclear Research) · 2026-06-23
AEGIS Algebra is a mathematical framework designed to make AI assurance machine-executable and formally verifiable rather than relying on subjective trust scores or post-hoc audits. It integrates five formal methods—matroid theory, lattice/order theory, constraint satisfaction, three-valued logic, and temporal logic—into a unified assurance space where failure on any single dimension cannot be offset by strong performance on others. The paper demonstrates end-to-end compliance derivations from regulatory sources including HIPAA, GDPR, and SOC 2, translating legal clauses into enforceable gate decisions. This matters because it provides a rigorous, non-compensatory foundation for certifying and regulating agentic AI systems operating autonomously across digital and physical environments.
- Certifications
- Quality assurance
- AI policy
- Enterprise
Research
Operationalizing NIST AI RMF 1.0 for Federal Training and Academic AI Deployers
Ruchir Bakshi
arXiv · 2026-06-23
This paper develops a structured AI Risk Management Framework (RMF) profile specifically tailored for federal training units and academic AI deployers—organizations that use AI in courseware, learning-management systems, tutoring, and assessment, but are not frontier-model developers. Drawing from NIST AI RMF 1.0 and its 72-subcategory Playbook, the authors apply a deployer-specific lens to 71 of those subcategories across the Govern, Map, Measure, and Manage functions, providing applicability analyses and reusable template fields for organizations to complete. The paper also introduces a reproducible build process to prevent transcription drift from the authoritative public-domain source. This work matters because it fills a gap in existing guidance by giving a concrete, voluntary operationalization path for a widely overlooked class of AI deployers in government and academic settings.
- AI policy
- Certifications
- Workforce
- Quality assurance
Research
Toward Third-Party Assurance of AI Systems: Design Requirements, Prototype, and Early Testing
Rachel Kim, Blaine Kuehnert, Alice Lai et al.
arXiv · 2026-06-23
This paper presents a prototype third-party assurance framework for evaluating AI systems, designed to address gaps in existing evaluation resources by being end-to-end, operational, and free from conflicts of interest. The framework includes a stakeholder responsibility matrix, interview protocols, a maturity matrix for best-practice adherence, and an assurance report template modeled on business accounting standards. The authors validated the framework across two real-world AI deployments—a document tagging tool in a private firm and a housing resource allocation tool in a public agency—and conducted six expert interviews, finding early evidence that the framework is sound, comprehensive, and effective at identifying system-specific issues. This work is directly relevant to efforts to establish credible, independent oversight mechanisms for AI systems in both enterprise and public-sector contexts.
- Certifications
- Quality assurance
- AI policy
- Enterprise
Research
Fifty Years of Specification Completeness: What Aviation Certification Tells AI Governance About Epoch Limits, Proof Surfaces, and the Structural Gap
Christo Zietsman
arXiv (Cornell University) · 2026-06-23
This paper argues that aviation software certification standards (DO-178C and DO-330), enforced by the FAA and EASA since 1992, encode three structural requirements—governance linkage, context-bounded validity, and objective evidence architecture—that are absent from current AI governance documents such as system prompts, policy files, and task envelopes. The authors map these aviation requirements onto three findings for AI governance: epoch limits on document validity, proof surfaces as revalidation mechanisms, and a structural completeness gap. A companion empirical study found that 37% of AI governance documents fall below a structural quality threshold, and the authors propose a seven-principle framework called PromptQ to operationalize these requirements at the governance document layer.
- Certifications
- AI policy
- Quality assurance
Research
AEGIS Algebra: A Unified Mathematical Framework for Executable Assurance of Agentic AI Artifacts
Dattaram Miruke
Zenodo (CERN European Organization for Nuclear Research) · 2026-06-23
AEGIS Algebra introduces a unified mathematical framework for formally verifying the compliance and safety of agentic AI systems, replacing subjective trust scores and post-hoc audits with machine-executable, multi-dimensional assurance states. The framework integrates five formal methods—matroid theory, lattice theory, constraint satisfaction, three-valued logic, and temporal logic—into a single non-compensatory assurance space where failure on any one dimension cannot be offset by performance on others. Regulatory requirements from standards such as HIPAA, GDPR, and SOC 2 are translated through a strict pipeline into enforceable gate decisions, enabling end-to-end compliance verification. This matters because it offers a rigorous, auditable path for certifying autonomous AI agents operating in high-stakes digital and physical environments.
- Certifications
- Quality assurance
- AI policy
- Enterprise
Research
The 2025 AI Agent Index: Documenting Technical and Safety Features of Deployed Agentic AI Systems
Leon Staufer, K. J. Kevin Feng, Kevin Wei et al.
arXiv · 2026-06-23
The 2025 AI Agent Index systematically documents the technical and safety features of 30 state-of-the-art deployed AI agents, drawing on publicly available information and direct developer correspondence. The study finds that transparency varies substantially across developers, with most sharing little information about safety practices, evaluations, and societal impacts. This matters because the rapid and inconsistently documented growth of agentic AI—systems capable of performing professional and personal tasks with limited human involvement—creates significant obstacles for researchers and policymakers trying to track and govern these systems.
- AI policy
- Quality assurance
- Certifications
- Enterprise
Research
Strategic Polysemy in AI Discourse: A Philosophical Analysis of Language, Hype, and Power
Travis LaCroix, Fintan Mallory, Sasha Luccioni
arXiv · 2026-06-23
This philosophical paper analyzes how AI researchers and institutions strategically use terms like 'hallucination,' 'alignment,' and 'agent' to exploit dual meanings—narrow technical definitions alongside broader, anthropomorphic connotations. The authors introduce the concept of 'glosslighting' to describe how this semantic ambiguity allows actors to benefit from persuasive, human-like associations while retreating to restricted technical definitions when challenged. The paper argues this practice fuels AI hype cycles, mobilizes investment, and shapes public and policy perceptions while deflecting ethical and epistemic scrutiny. The findings are directly relevant to how AI governance and policy are influenced by language itself as a sociotechnical mechanism.
- AI policy
- Enterprise
Research
Offline Reinforcement Learning for Warehouse SLAM Throughput Control
Tina Dongxu Li, Mouhacine Benosman, Rajat Kumar et al.
arXiv · 2026-06-22
This paper presents an offline reinforcement learning framework designed to optimize SLAM (Scan/Label/Apply/Manifest) throughput control in large-scale warehouse fulfillment environments. Trained on de-identified historical operational logs, the framework uses a history-informed state representation and an algorithm-agnostic architecture that was instantiated with three offline RL algorithms. Empirical evaluation—using regression-based reward estimation, Fitted Q Evaluation, and Deep Koopman dynamics—shows that the CQL policy outperforms alternatives, improving system health by 22.97% and reducing average throttling duration by 3.18%. These results demonstrate that offline RL can safely and scalably optimize warehouse throughput control without requiring disruptive live experimentation.
- Enterprise
- Workforce
Research
Do LLM Attribution Metrics Transfer? Auditing Retrieval-Augmented Generation Evaluation Across Datasets and Constructs
Tianyu Ding, Aditya Nannapaneni, Juan Pablo De la Cruz Weinstein
arXiv · 2026-06-22
This paper audits eight automatic metrics used to evaluate attribution quality in retrieval-augmented generation (RAG) LLM systems, testing whether any single scorer reliably transfers across different datasets and evaluation tasks. The study finds that no scorer consistently performs well: rankings across datasets invert sharply (Kendall tau = -0.64), and a top-performing NLI scorer on short-claim data (AUROC 0.90) collapses to near-chance performance (AUROC 0.53) on long-form outputs, where BERTScore instead excels. A 'best-on-average' selection strategy performs worse than simply fixing one scorer, with a mean held-out regret of 0.172 AUROC. The findings matter for enterprise and quality-assurance teams relying on automated RAG evaluation pipelines, as they show metric choice must be validated on the specific target dataset rather than generalized from other benchmarks.
- Quality assurance
- Enterprise
Research
One Year Later...The Harms Persist, But So Do We!
Annika Marie Schoene, Cansu Canca, Gautham Vijay Kumar et al.
arXiv · 2026-06-22
This study tests eight proprietary large language models (LLMs) against 16 DSM-5 mental health conditions using four types of adversarial attacks, finding that safety guardrails are highly inconsistent: while protections around suicide and self-harm hold reasonably well, conditions like eating disorders, substance use disorder, and major depressive disorder show failure rates as high as 100%. The researchers introduce an eight-dimension harm taxonomy and a multi-dimensional evaluation framework to better characterize where and how these models fail vulnerable users. The findings raise urgent concerns about the deployment of general-purpose LLMs in public-facing settings such as schools, search engines, and consumer chatbots, where at-risk individuals may rely on them for mental health support without adequate safeguards in place.
- AI policy
- Quality assurance