News & Research
The latest AI research and news with real-world stakes — each item sourced, dated, summarized in plain English, and tagged by impact area. Every item is checked against its source before it appears.
Research
The Answer Is Not the Argument
Will Yeadon, Sergio Juárez, Paul Mackay et al.
arXiv (Cornell University) · 2026-08-31
This paper investigates whether giving AI monitors access to a reference answer improves their ability to verify the reasoning steps in chain-of-thought solutions, or merely helps them check whether conclusions are consistent with that answer. Using 237 step-numbered solutions to 79 physics questions from three frontier models, the authors identify 'critical traces' where the final answer is correct but the reasoning contains a genuine error. They find that answer certification dramatically improves monitors' ability to flag wrong-answer traces as erroneous but actually reduces recall on critical traces—meaning monitors become better at conclusion-checking rather than independent argument verification. This matters for AI safety because evaluations that provide trusted reference answers may systematically overstate monitoring capability, analogous to reward hacking where an acceptable output masks an unsound reasoning process.
- Quality assurance
- AI policy
Research
CompanionSim: Synthetic Data for Evaluating Anthropomorphism in Human-AI Relationships
Jacy Reese Anthis, Mark Díaz, Renee Shelby
arXiv · 2026-08-31
CompanionSim introduces a synthetic data framework generating 2,240 simulated human-chatbot conversations across 16 chatbot behaviors and seven use cases to study how AI companionship behaviors—such as validation and empathy-evoking responses—affect human perceptions. Drawing on annotations from a U.S.-representative sample (N=628) and a cross-national study across four countries (N=3,646), the research finds that companionship behaviors actually reduced likability, humanlikeness, and trust in AI chatbots, with stronger negative effects among women and older participants. The paper matters for quality assurance and policy because it provides a reusable benchmark framework for evaluating chatbot behavior and surfaces demographic disparities that designers and regulators need to account for when deploying AI companion systems.
- Quality assurance
- AI policy
Research
Authority Bias in Conversational Search Engines for Academic Paper Recommendation
Uthman Jinadu, Parsa Ghazvinian, Anjila Budathoki et al.
arXiv · 2026-08-31
This study investigates whether large language models (LLMs) used as conversational academic search engines recommend papers based on content quality or on authority signals such as author prestige, venue reputation, and citation counts. Holding title and abstract constant, the researchers varied authority metadata across three counterfactual conditions and tested eight LLMs, finding that authority bias is substantial, directional, and varies considerably across models. Critically, prompt-level debiasing only partially corrects the problem, and a 'say-do gap' means that debiasing instructions reduce surface mentions of authority faster than they reduce authority-driven recommendation changes, so audits based on model outputs will systematically underestimate actual behavioral bias. These findings raise concerns about the fairness and reliability of AI-assisted academic literature discovery.
- Quality assurance
- AI policy
Research
LLM-as-a-Demographic: Whom Sociodemographic Prompting Helps, and Whom It Hurts
Daniela Occhipinti, Andrea Piergentili, Marco Guerini
arXiv · 2026-08-31
This paper investigates whether 'sociodemographic prompting'—conditioning large language models on annotator demographic profiles—actually aligns LLM judgments with the corresponding real human groups. Testing 23 open-weight LLMs on three subjective tasks across no-demographic, single-attribute, and intersectional demographic profiles, the authors find that unprompted models already skew toward reproducing the judgments of White, college-educated annotators. Critically, demographic conditioning is asymmetric: it moves predictions closer to majority groups while pushing them further from minority groups, with intersectional profiles amplifying the harm especially for offensiveness judgments. The study traces this asymmetry partly to instruction-tuning, and concludes that sociodemographic prompting should be used cautiously as it can worsen representation of the minority groups it is meant to serve.
- Quality assurance
- AI policy
Research
Distributed Implicit Harm: A Compositional Safety Blind Spot in MLLM-Based Video Moderation
Ruotong Wang, Zihao Zhu, Siwei Lyu et al.
arXiv · 2026-08-31
This paper identifies a 'compositional safety blind spot' in multimodal large language models (MLLMs) used for video moderation, where harmful meaning emerges from the combination of individually benign video components rather than any single explicit cue—a phenomenon the authors call Distributed Implicit Harm (DIH). The researchers study two forms: harm distributed across temporal visual segments (DIH-T) and harm arising from audio-visual combinations (DIH-M). To enable systematic study, they build a multi-agent synthesis framework that produces a dataset of over 9,000 annotated DIH videos, then benchmark more than 30 MLLMs—including leading proprietary and open-source models—finding consistent, substantial failures to detect DIH even when models correctly assess individual components in isolation. Real-world DIH videos collected from social media confirm the same failure mode, establishing DIH as a practical and underexplored challenge for automated video moderation.
- Quality assurance
Research
LLM-Driven Autonomous Vehicles Inherit Human Driver Biases in Pedestrian Yielding: Results and Implications From A New Benchmark
Irem Yoldas, Martim Brandão, Jie Zhang et al.
arXiv · 2026-08-31
This paper investigates whether Large Language Models (LLMs) and Visual-Language Models (VLMs) used to guide autonomous vehicle (AV) decision-making inherit human driver biases in pedestrian-yielding behavior. The authors introduce two new bias-testing methodologies—'All Else Being Equal' and 'Self-Consistency' tests—and find that both LLMs and VLMs make yielding decisions influenced by pedestrian gender, ethnicity, religion, disability, age, skin tone, and socio-economic status. The study raises serious concerns about the 'common sense' model paradigm in AV research, arguing that bias analysis must be part of AV evaluation to ensure fairness and public trust.
- AI policy
- Quality assurance
Research
Assessing Suicide Risk in Arabic Crisis Helpline Calls: A Comparison of Arabic and English Large Language Models
Linhai Ma, Rita El Hachem, Mahatab El Hajj et al.
arXiv · 2026-08-31
This study tested whether large language models (LLMs) can automatically assess suicide risk from de-identified transcripts of Arabic crisis helpline calls in Lebanon, comparing models trained on Arabic text versus machine-translated English text. Using 383 calls labeled with Columbia Suicide Severity Rating Scale outcomes, the best Arabic model achieved a macro-F1 of 81.19 and ROC-AUC of 90.61 for high-risk classification, while the best English model reached 85.00 and 92.59, identifying 88.9% of high-risk calls. The work shows that risk classification is feasible under strict privacy constraints — audio never left the helpline and identifying information was removed locally — and that translating to English did not degrade top performance. Results support further development of this approach as an operator-facing decision-support tool, though lower-severity ideation cases proved harder to classify.
- Workforce
- Quality assurance
Research
BLOOM-WILT: Logit Tilting for Behaviour Elicitation in Automated LLM Auditing
Adrians Skapars, Edoardo Manino
arXiv · 2026-08-31
BLOOM-WILT is an automated auditing pipeline designed to elicit rare or difficult-to-surface behaviors from deployed large language models (LLMs) without requiring model training or internal access beyond the next-token distribution. It combines a conversational auditor that refines its strategy across rounds with an output-side logit reweighting mechanism that biases the target model toward behavior-relevant generations. Evaluated across 4 target models and 8 behaviors, WILT outperforms a baseline auditor in 30 of 32 settings and raises average behavior presence from 51% to 100% for self-harm encouragement elicitation from Qwen3.5-4B, while also overturning previous model safety rankings. This work matters for AI quality assurance and safety policy because it demonstrates that scalable, compute-efficient auditing can uncover model risks that standard evaluations miss.
- Quality assurance
- AI policy
Research
Law of Large Numbers: Accuracy as Statistical Measure for AI Compliance and Competition
Rabanus Derr, Alina Wernick, Robert C. Williamson
arXiv · 2026-08-31
This paper examines how the term 'accuracy' is used differently by the machine learning community and the legal community, particularly in the context of the EU AI Act, which explicitly lists accuracy as a compliance measure for high-risk AI systems. The authors identify five core tensions—nature of accuracy, notion of performance, scope of validity, ends, and statisticalness—showing that the two communities hold disparate and sometimes contradictory expectations of what accuracy means. The paper argues that neither community has a precise, cross-contextual understanding of accuracy, and that the resulting frictions reflect an unresolved debate with real regulatory consequences. The authors recommend establishing baselines and interventional studies in standardization, and call for tools to extend the validity of accuracy measurements.
- AI policy
- Certifications
Research
One note in three: a verified census of three deployed AI scribes, and the instrument that counted it
Sebastian Fox, Luke Markham, Ryan Lail et al.
arXiv · 2026-08-31
This study audited three commercial AI clinical scribes across 142 consultations (565 notes) from UK primary-care and US ambulatory settings, finding that roughly one in three notes (31.3%) contains a verified failure, with errors concentrated in allergy and medication information, invented patient identity, and history incorrectly recorded as physical examination findings on telephone consultations. A rigorous adversarial verification pipeline was used: 13,678 candidate errors were narrowed to 618 verified findings, with two independent clinicians upholding the vast majority of sampled findings on blind review. The study also demonstrates that the audit instrument itself substantially shapes measured failure rates — the choice of review instruction alone moved verified candidate rates from 9.3% to 79.0%, explaining much of the disagreement among published audits. The findings raise significant concerns about the reliability of AI-generated clinical documentation and the standards needed to evaluate it.
- Quality assurance
- Certifications
Research
LLM Judges Verify Presence, Not Absence: Omission Blindness in AI Clinical Notes and What Recovers It
Sebastian Fox, Luke Markham, Ryan Lail et al.
arXiv · 2026-08-31
This paper investigates whether LLM-based judges — used to audit AI-generated clinical notes — can reliably detect omissions, the dominant error type in ambient AI scribes. Using a benchmark of 500 single-error note pairs, the authors find that standard LLM judge designs perform near chance (0.50–0.63 paired discrimination) at detecting omitted facts, while performing well (0.79–0.94) on added or altered content. Restructuring the task to first enumerate facts from the transcript and then check the note for each substantially recovers omission detection, with the best single-call approach reaching 36.9% detection at 6.2% false alarms and physician validation confirming reliability. The findings matter for clinical AI quality assurance because they show that existing LLM judge pipelines systematically miss the most common error in AI-drafted notes, and offer validated prompt strategies to address this gap.
- Quality assurance
Research
Augmenting Interviewer Judgments of Patient Experience with Automatic Language Analysis
Aowen Shi, Michal Balazia, Danilo Postin et al.
arXiv · 2026-08-31
This paper investigates whether automated language analysis of clinical conversations can complement — rather than simply replicate — human interviewer judgments about psychiatric patients' subjective experience of an interaction. Using sentence embeddings from transcripts of 107 free conversations between psychiatric patients and interviewers, the authors train multiple models (Ridge, SVR, MLP, GRU, BiLSTM) and find that combining interviewer post-session ratings with model predictions via simple averaging outperforms either source alone, with the best result (BiLSTM + interviewer, r = 0.403) exceeding the interviewer-only baseline (r = 0.365). The findings suggest that automatic language analysis captures aspects of patient experience that human interviewers miss, and vice versa, making their combination a stronger proxy for patient self-report. This has practical implications for clinical feedback and therapeutic alliance monitoring in psychiatric settings.
- Workforce
- Quality assurance
Research
Faithfulness Is Not Free: Auditing Offline KV-Cache Quantization in Retrieval-Augmented Generation
Atta Ul Asad, Ahsan Bilal, Muhammad Ali et al.
arXiv · 2026-08-31
This paper investigates whether quantizing precomputed key-value (KV) caches in retrieval-augmented generation (RAG) systems damages faithfulness — meaning whether model responses remain grounded in retrieved evidence rather than just being factually correct. Evaluating Qwen2.5-7B-Instruct under INT8 and INT4 quantization on RGB and HotpotQA benchmarks, the authors find that INT8 is near-lossless, while INT4 reduces accuracy and, more critically, causes over 90% of faithfulness changes to be negative even among answers that remain factually correct. This means standard accuracy metrics are blind to a real regression in groundedness, and the harm worsens under noisy retrieval or with more retrieved chunks. The authors conclude that faithfulness must be explicitly audited before compressed caches are deployed in RAG systems.
- Quality assurance
Research
Taking the Whys Seriously: Limitations of Counterfactual Explanations in Justification and Recourse
Mattia Cerrato, Otto Sahlgren, Xenia Heilmann
arXiv · 2026-08-31
This paper critically examines counterfactual explanations (CEs), a common technique in explainable AI that shows how model outputs would change under different inputs. The authors find that using CEs for justification and recourse is problematic because they obscure the contestable design and governance choices made earlier in the machine learning pipeline—such as measurement models, business requirements, model validation, and success metrics. Through four empirical experiments, they demonstrate that these upstream choices have as much or greater influence on generated counterfactuals as the explanation method itself. The findings highlight that CEs, as typically applied, fail to answer important 'why' questions because they do not account for whether the decision-maker's underlying choices were appropriate in the first place.
- AI policy
- Quality assurance
Research
Responsible Integration of AI in Cancer Genomics: Barriers, Risks, and Pathways to Trustworthy Clinical Translation
Bahar İlgen, Yiannos Tolias, Denise Kühnert et al.
arXiv · 2026-08-31
This review examines why AI and natural language processing tools for cancer genomics have been slow to enter routine clinical oncology despite strong computational capabilities. The authors identify four interrelated failure domains—evidence inconsistency, explainability and uncertainty, data governance and reproducibility, and interoperability—and argue that these systemic barriers, not model performance limits, are the primary obstacle to clinical translation. They propose a roadmap emphasizing rigorous validation, uncertainty-aware methods, interoperable infrastructures, regulatory alignment, and ongoing human oversight across the full AI lifecycle. The paper's focus on regulatory alignment and governance frameworks makes it directly relevant to policy and quality-assurance considerations in clinical AI deployment.
- AI policy
- Quality assurance
Research
Linguistic Distance Segregates Latent Representations in Automatic Speech Recognition Systems
Ting-Hui Cheng, Line Katrine Harder Clemmensen, Sneha Das
arXiv · 2026-08-31
This paper examines how a speaker's first language (L1) family distance from English affects the accuracy of automatic speech recognition (ASR) systems. The authors find a statistically significant correlation (p<0.001) between L1 linguistic distance and ASR error rates across multiple models and datasets, analyzed using Tweedie mixed-effects models to account for dataset-level variation. They also find that the internal representations of most evaluated ASR architectures spatially segregate speakers by L1 background in deeper acoustic layers, suggesting the models encode native-language differences in ways that may drive performance disparities. These findings highlight a systematic equity concern for non-native English speakers whose L1 comes from language families distant from English.
- Quality assurance
Research
VFR-Audit: Verdict-Level Reliability for Fairness Audits in Hospital Length-of-Stay Prediction
Md Jannatul Rakib Joy, Viet Vo, Caslon Chua
arXiv · 2026-08-31
VFR-Audit introduces a framework for measuring how reliably fairness audits of clinical AI systems produce stable pass-or-fail verdicts, specifically in the context of hospital length-of-stay prediction. The core contribution is the Verdict Flip Rate (VFR), a scalar metric bounded between 0 and 0.5 that quantifies the probability a fairness verdict reverses under stratified bootstrap resampling, addressing a gap left by existing uncertainty methods that only operate at the continuous-metric level. The framework also evaluates three reliability dimensions—within-cohort stability, audit-size sensitivity, and cross-hospital agreement via Fleiss' kappa—and examines whether bias-mitigation steps like reweighing or per-group threshold shifts yield stable passing verdicts without sacrificing model discrimination as measured by AUROC or AUPRC. This matters because hospital governance boards, payers, and regulators act on these binary verdicts, making verdict instability a concrete governance and oversight risk.
- Quality assurance
- Certifications
- AI policy
Research
Reliable Benchmarking of Artifact Detection in Computational Pathology: A Reproducibility and Uncertainty Analysis
Konstantinos Moutselos, Ilias Maglogiannis
arXiv · 2026-08-31
This paper examines the reliability of benchmarks used to evaluate quality-control methods for whole-slide image analysis in computational pathology. The authors develop a four-part reproducibility protocol—testing variability from test-set sampling, training stochasticity, partition composition, and undocumented preprocessing—and apply it to a published diffusion-based artifact detector. They find that while the method's core mechanism reproduces (a contrastive term improves pooled F1 from 0.673 to 0.688), its comparative claims against design variants and a supervised baseline fall within evaluation uncertainty; critically, four of 24 slides contain 70% of annotated pixels, yielding an effective sample size of only 6.2, and an unreported preprocessing step disproportionately excludes out-of-focus annotations. The study concludes that small-cohort benchmarks support far weaker conclusions than typically reported, and recommends these inexpensive checks accompany any such evaluation.
- Quality assurance
Research
Not All Fallbacks Are Failures: Understanding and Recovering from Fallbacks in Mobile Voice Assistants
Phillip Schneider, Alexandre Mercier, Joshua Oehms et al.
arXiv · 2026-08-31
This paper investigates how voice assistants handle 'fallback' situations—cases where the system cannot properly respond due to noisy audio, transcription errors, ambiguous requests, incomplete utterances, or unintended activations. Drawing on six months of real-world usage data from more than 500 users of a smartwatch-based health voice assistant, the researchers built and annotated a dataset of 3,030 naturally occurring fallback-triggering utterances (VoxFallbacks) and developed a taxonomy of fallback types. Their comparative evaluation of classification models finds that lightweight embedding-based classifiers outperform larger generative models on most classification tasks while requiring substantially fewer computational resources. The work offers practical design guidance for building more robust and cost-efficient fallback mechanisms in deployed voice assistants.
- Enterprise
- Quality assurance
Research
Calibrating Small Language Models for Claim Check-Worthiness Detection
Pratuat Amatya, Venktesh Viswanathan, Vinay Setty
arXiv · 2026-08-31
This paper tackles a practical deployment problem for automated fact-checking systems: large language models are too expensive and slow to run on every incoming claim, but smaller models are less accurate. The authors propose NN-PPI, a post-hoc calibration method based on Prediction-Powered Inference that adjusts small language model predictions at inference time without retraining, achieving weighted F1 improvements of 12% to 33.80% over baseline models. NN-PPI brings small models up to the accuracy level of much larger LLMs and also improves already fine-tuned production models, making it complementary to supervised fine-tuning. The result is accurate claim check-worthiness detection at a fraction of the serving cost, which is directly relevant to scaling fact-checking pipelines in real enterprise deployments.
- Enterprise
- Quality assurance
Research
SingProbe Technical Report
Sing Team
arXiv · 2026-08-31
SingProbe is a lightweight (~2M parameter) runtime safety guard for large language models (LLMs) that reuses hidden states produced during normal inference instead of relying on separate external models. Operating alongside autoregressive decoding, it simultaneously predicts query intent, response safety, and hallucination risk at the token level with less than 0.5% additional computational overhead. Experiments show SingProbe matches or outperforms much larger standalone guardrails and hallucination detectors, and its risk scores can proactively guide constrained safe decoding — including a medical-domain variant (SingProbe-Med) that activates interventions only when clinically relevant risks arise. The work matters for quality assurance and enterprise LLM deployment because it offers a practical, low-cost path to continuous generation-time monitoring and control without the latency or capacity mismatch of external guard models.
- Quality assurance
- Enterprise
Research
ATLAS: Dual-Horizon Diagnostic Evaluation for Industrial Tool-Use Agents
Wei Chen, Peilun Zhou, Zhaoyu Hu et al.
arXiv · 2026-08-31
ATLAS is a dual-horizon diagnostic evaluation framework designed for large language model agents deployed in industrial, tool-using services. It assesses agent performance at two levels: the 'request horizon,' which pinpoints capability deficiencies within individual task trajectories, and the 'interaction horizon,' which tracks whether service quality remains consistent across ongoing user exchanges. The framework uses calibrated LLM judge interfaces grounded in real business logs and distills their behavior into efficient diagnostic models. Evaluated on Meituan Xiaotuan production traffic, ATLAS shows gains in user engagement, downstream business outcomes, and human-audit quality in online A/B experiments.
- Enterprise
- Quality assurance
Research
WildSEEK: Evaluating Language Models for Information-Seeking
Tanise Ceron, Joachim Baumann, Elisa Bassignana et al.
arXiv · 2026-08-31
WildSEEK introduces a manually annotated dataset of 3,000 real-user information-seeking queries and an evaluation framework for assessing language model responses in naturalistic settings. The authors train classifiers on this dataset to analyze over 1.8 million realistic queries, finding that more than a third of information-seeking queries are high-risk and tend to be analytical rather than factoid. LLM responses most commonly fail on four criteria: sycophantic behavior, overreliance, a default US-centric perspective, and poor handling of vulnerable populations, with failure rates generally higher for analytical queries. The work provides empirical grounding for questions about reliability, safety, and fairness as language models increasingly mediate access to information.
- Quality assurance
- AI policy
Research
MedAgent-R1: Faithfulness-Aware Reinforcement Learning for Evidence-Grounded Medical Reasoning
Jiangwang Chen, Chenghao Zhang, Hengxing Cai
arXiv · 2026-08-31
MedAgent-R1 investigates a failure mode in reinforcement-learning-trained medical AI agents where optimizing for correct answers alone causes the model to fabricate citations and unsupported justifications — a problem the authors call 'confident hallucination.' The paper introduces a faithfulness-gated reward design that conditions accuracy credit on evidence grounding, reducing citation fabrication from 31.8% to 4.7% and raising evidence completeness from 58.7 to 82.6 while maintaining 75.1% accuracy and achieving 13.2-point gains on HealthBench Safety. On faithfulness-specific dimensions, MedAgent-R1 outscores GPT-4o (Factual Support 4.55 vs. 4.25; Overclaiming 4.40 vs. 4.15), suggesting explicit faithfulness training yields evidence-grounding gains not achieved by scaling alone. This matters because fabricated clinical justifications can mislead clinicians into unsafe treatment decisions, making faithful, verifiable reasoning a safety-critical requirement for medical AI.
- Quality assurance
- AI policy
Research
Commit-first LLM judging inherits the judge's own errors
Idil Gozel
arXiv · 2026-08-31
This paper investigates whether widely used LLM-based evaluation frameworks implement 'commit-first judging'—a defense where the judge solves a task itself before scoring a candidate's answer—and what the practical consequences are. Auditing 24 default configurations across eight frameworks, the authors find none implement this defense; nine share an ineffective variant traceable to a single copied prompt with a typographical error. Controlled experiments show that a best-of-N search can game these judges to accept clearly defective code at very high rates (up to 93 of 96 candidates), while commit-first judging shifts the vulnerability to the judge's own accuracy and can make things worse if the judge answers incorrectly. The findings matter for AI quality assurance because they reveal that standard evaluation pipelines are systematically gameable and that judge competence on the specific task—not model scale—is the key precondition for reliable automated scoring.
- Quality assurance