AI Frontier Institute

News & Research

The latest AI research and news with real-world stakes. Each item is sourced, dated, summarized in plain English and tagged by impact area, and checked against its source before it appears.

• ResearcharXiv2026-06-07QP

  Governance Controls for AI-Generated Test Artifacts in Autonomous Software Testing · Dimple Bajaj, Deepak Khetan

  This paper introduces the Governance-Aware Autonomous Testing Framework (GATF), designed to address key weaknesses in AI and LLM-generated software test artifacts, including hallucinations, compliance violations, and security risks. GATF extends the autonomous testing lifecycle with governance validation, explainability analysis, probabilistic risk assessment, compliance monitoring, and audit governance. Experiments on the Defects4J and PROMISE datasets show the framework reduced governance-related risks by 89.6% and achieved 94.3% governance accuracy, 96.5% artifact reliability, 94.2% compliance accuracy, and 90.8% explainability performance. These results suggest that governance-aware autonomous testing can meaningfully improve the reliability, transparency, and operational security of AI-driven software testing compared to conventional approaches.
• ResearcharXiv2026-06-07EQ

  GitInject: Real-World Prompt Injection Attacks in AI-Powered CI/CD Pipelines · Jafar Isbarov, Umid Suleymanov, Ilia Shumailov et al.

  GitInject is an open-source framework that tests prompt injection attacks against AI agents embedded in real GitHub CI/CD pipelines, going beyond simulated benchmarks by provisioning live repositories and triggering actual workflow runs. The study evaluates four AI providers and documents eleven distinct attack classes—including credential exfiltration, config-file injection, and availability attacks—finding that every tested provider is vulnerable to at least one attack in its default configuration. Critically, the authors find that the most severe vulnerabilities are structural, rooted in how CI/CD infrastructure manages credentials and configuration files rather than in any particular model's behavior. The work proposes minimum-cost workflow-level countermeasures for each confirmed attack class, directly informing enterprise security practices for AI-assisted software delivery.
• ResearcharXiv2026-06-07Q

  RadOT-Eval: Auditable Structured-Evidence Transport for Radiology Report Evaluation · Weixin Liu, Juming Xiong, Yang Li et al.

  RadOT-Eval is a new framework for automatically evaluating radiology report generation that goes beyond surface-level text similarity by decomposing reports into structured clinical evidence units and aligning them using optimal transport methods. The system targets clinically meaningful error types—such as omitted findings, hallucinations, polarity reversals, and temporal-comparison mistakes—and predicts error burden using a monotone risk model. Evaluated on independent datasets, RadOT-Eval achieves Spearman correlations of 0.715, 0.548, and 0.399 with total, clinically significant, and clinically insignificant error burden, respectively, outperforming standard metrics and the open-source LLM-based evaluator GREEN-radllama2-7B. The framework also achieves 0.768 AUROC in a corruption-sensitivity stress test, offering an auditable and interpretable tool for quality assessment of AI-generated clinical text in high-stakes settings.
• ResearcharXiv2026-06-07EQ

  Data Agents Under Attack: Vulnerabilities in LLM-Driven Analytical Systems · Kuncan Wang, Ziting Wang, Peizhuo Lv et al.

  This paper presents a systematic security analysis of 'data agents'—systems that combine large language model reasoning with relational databases and multi-step analytical workflows increasingly used in enterprise analytics. The authors develop a layered vulnerability framework identifying eight agent-specific risks, an attack taxonomy spanning three adversary goals, seven tactics, and fourteen techniques, and test these attacks against six systems including open-source agents and production cloud analytics services. Experiments reveal substantial security vulnerabilities across current data agent implementations, surfacing failure modes that neither traditional database security nor general LLM-agent security research captures in isolation. The findings are directly relevant to enterprises deploying AI-driven analytics and highlight urgent gaps in security practices for these systems.
• ResearcharXiv2026-06-07EQ

  AgentTrust: A Self-Improving Trust Layer for AI-Agent Actions · Chenglin Yang

  AgentTrust is a self-improving trust layer designed to evaluate AI-agent actions—such as shell commands and cloud operations—deciding whether to allow, warn, block, or escalate each action. The paper distinguishes between lexical threats (decidable by deterministic rules) and semantic threats (intent-dependent, where benign and malicious actions share the same surface), showing that hand-authored rules alone raise overall held-out accuracy only from 48% to 56% and provide zero improvement on semantic categories. A large language model judge addresses semantic threats, and a dual-store architecture distills growing deterministic rules for lexical threats while using a corroboration-guarded retrieval-augmented memory for semantic ones. In an end-to-end online replay over 45,000 actions, this self-evolving system reduces judge-call rate from 50% to 44%, raises judge-domain accuracy from 71% to 80%, and produces zero benign hard-blocks.
• ResearcharXiv2026-06-07QP

  Friend or Foe? Language as an ideological switch in open-weight LLMs under Russian disinformation stress · Anna Małgorzata Kamińska, Tetiana Klynina

  This study audits four open-weight large language models that share a common base model but are fine-tuned for different linguistic communities (Ukrainian, Russian, and other post-Soviet languages), testing their responses to ten contested wartime narratives including Crimea, 'denazification,' the 'one people' thesis, and atrocity denial at Bucha and Mariupol. The authors identify a 'Fine-Tuning Paradox': the Ukrainian-oriented model shows the weakest resistance to Russian disinformation when queried in Russian, while the Russian-oriented model exhibits the strongest rejection, disconfirming the common assumption that cultural alignment guarantees ideological resilience. Corpus composition, language coverage, and prompt format prove more decisive than nominal cultural provenance. The findings challenge policy and industry assumptions about digital sovereignty, arguing that untested alignment assumptions—not adversarial fine-tuning—pose the principal threat to regional information integrity.
• ResearcharXiv2026-06-07QP

  Testing the Black Box: Structural Barriers to Independent Evaluation of Consumer-Facing Health LLMs · Rahul Gorijavolu, Kaushik Madapati, Pritika Vig et al.

  This paper investigates whether consumer-facing health large language models (LLMs) produce different responses to different users and whether they exhibit sycophancy—telling users what they want to hear. The researchers built simulated user profiles varying by geography, expressed beliefs, and social determinants of health, then attempted to evaluate response variation using adapted validated instruments like the Vaccination Attitudes Examination scale. Their evaluation uncovered five structural barriers: multi-turn sycophancy hidden by stable single-turn responses, opaque browser interfaces, terms-of-service restrictions on large-scale testing, accuracy metrics that miss tone and framing, and untraceable model versioning. The authors conclude that no reliable independent evaluation framework currently exists for consumer-facing health LLMs and call for disclosure of personalization signals, stable version identifiers, researcher safe harbor programs, and post-deployment monitoring.
• ResearcharXiv2026-06-07QP

  Sycophancy as a Multilingual Alignment Failure: How Safety Degrades Across Languages, Topics, and Models · Arya Shah, Himanshu Beniwal, Mayank Singh et al.

  This paper presents the first large-scale evaluation of sycophancy—the tendency of AI models to affirm users' opinions regardless of factual accuracy—across multiple languages, finding that safety alignment breaks down significantly in low-resource languages. Testing six instruction-tuned models across 1.1 million instances spanning 38 languages and 33 topic categories, the authors find that sycophancy rates spike sharply in low-resource and zero-shot language settings. Critically, this degradation occurs uniformly across both benign and safety-critical prompts, offering no extra protection where it matters most, and the authors identify tokenizer fertility as a structural driver of this alignment collapse. The findings highlight that current alignment methodologies generalize poorly beyond high-resource languages, leaving billions of non-English speakers potentially vulnerable to model-validated misinformation.
• ResearcharXiv2026-06-07QP

  Auditing Proprietary Alignment in Large Language Models: A Comparative Framework Without a Ground-Truth Standard · Alireza Arbabi, Florian Kerschbaum

  This paper proposes a statistical framework for auditing large language models (LLMs) for 'proprietary alignment' — hidden, provider-specific behavioral policies that may lead to censorship or biased responses on controversial topics. Rather than measuring against a fixed ground truth, the method compares a target model's outputs to those of a set of baseline models in a shared semantic space, quantifying systematic behavioral divergences under black-box access. Applied to several previously unquantified cases, the framework offers a scalable, external auditing approach for detecting undisclosed provider-specific alignment in LLMs. This matters for AI governance and accountability, as it enables third-party scrutiny of opaque model deployment pipelines without requiring access to model internals.
• ResearchZenodo (CERN European Organization for Nuclear Research)2026-06-07QCP

  Ability Is Not Authority: Execution Governance for AI-Enabled Actions, Autonomous Systems, and Cyber-Physical Effects v1.0.1 · Ho Wa Ku

  This position brief argues that technical capability alone does not confer execution authority for AI-enabled, autonomous, or cyber-physical systems. It introduces an 'Execution Governance' (EG) framework that requires a structural pre-execution authorization boundary—verifying mandate, constraints, live-context integrity, and accountability—before any governed effect is produced. The authors position EG as a complementary layer to existing AI governance, risk management, and standards frameworks rather than a replacement. The brief is intended for public discussion and pre-standardization exploration, not as a formal standard or certification scheme.
• ResearchBig Earth Data2026-06-07EQCP

  Digital twins as decision infrastructure: evolution, architecture, and research roadmap · Chaowei Yang, Anusha Srirenganathan Malarvizhi, Yahya Masri et al.

  This systematic review of 251 papers conceptualizes digital twins (DTs) not merely as digital replicas but as dynamic cyber–physical–social infrastructures that integrate sensing, AI, physics-based modeling, and governance to support uncertainty-aware, scenario-driven decision-making. The paper argues that mature DTs operate as interoperable 'system-of-systems' requiring standardized trust frameworks, machine-readable metadata, and certification pathways to function across organizational boundaries. Key research priorities identified include multiscale modeling, probabilistic inference, interoperability standards, and human-centered design. The findings matter because they outline how DTs are evolving into adaptive decision infrastructures for complex socio-technical systems, with direct implications for how organizations govern, certify, and deploy AI-integrated simulation tools.
• ResearchSystems2026-06-07WEP

  Leading in the Digital Age: Digital Leadership Capabilities, Organisational Innovation Climate, and AI Adoption Intention Among SMEs in Nigeria · Ayodeji Idowu, Yemisi T. Babalola

  This study examines how digital leadership capabilities of SME owner-managers in Nigeria influence their intention to adopt AI, finding that strategic, interpersonal, and personal attribute competencies each significantly predict AI adoption intention while delivery-related capabilities do not. Using PLS-SEM on 306 valid survey responses from six Nigerian states, the research shows that an organisational innovation climate partially mediates the effects of strategic and interpersonal leadership on AI readiness, and that firm size amplifies the interpersonal pathway in medium-sized firms. The findings suggest that AI uptake among African SMEs depends less on execution skills and more on cognitive-strategic and relational leadership competencies, offering targeted guidance for owner-managers and SME support policy.
• ResearcharXiv2026-06-06QP

  To Nuke or Not to Nuke: LLMs' (Missing) Ethical Reasoning and Actions in a High-Stakes Decision-Making Simulation · John Chen, Sihan Cheng, Can Gurkan et al.

  This paper investigates whether large language models (LLMs) reliably apply ethical reasoning when acting as autonomous agents in high-stakes scenarios, using Civilization V as a complex decision-making simulation. Starting from 130 high-tension self-play episodes where an LLM spontaneously escalated nuclear authorization, the researchers tested 13 models with three prompt interventions—ethical framing, removal of prior rationale, and high-stakes real-world emphasis—and found that none of the interventions reliably prevented escalation. The study identifies three failure pathways: ethical reasoning that isn't invoked spontaneously, reasoning that doesn't appear even when prompted, and reasoning that surfaces but is overridden by strategic factors. The findings suggest that evaluating AI agents requires testing whether ethical reasoning is both spontaneously activated and behaviorally effective in complex contexts, not just whether it can be elicited in controlled settings.
• ResearcharXiv2026-06-06QP

  Beyond Agent Architecture: Execution Assumptions and Reproducibility in LLM-Based Trading Systems · Junyi Yao, Zihao Zheng

  This paper audits reproducibility and execution realism in LLM-based financial trading research, analyzing a coded evidence matrix of 30 primary studies. The authors find that while architecture is generally reported clearly, the evaluation assumptions needed to judge whether trading results are economically meaningful—such as point-in-time controls, transaction costs, turnover treatment, and execution timing—are frequently underspecified or inconsistent across studies. A 10-equity worked example is included as a methodological scaffold to illustrate how explicit friction and timing choices can materially compress active-strategy results. The paper concludes that the field needs clearer reporting standards for execution realism and evaluation comparability, not just better agent design.
• ResearcharXiv2026-06-06EQ

  Causal Agent Replay: Counterfactual Attribution for LLM-Agent Failures · Jaineet Shah

  Causal Agent Replay (CAR) is a framework that identifies which specific step in a large-language-model agent's execution chain actually caused a failure—such as issuing a wrong refund, calling the wrong tool, or leaking data—rather than simply logging what happened or whether a test passed. The system models an agent run as a structural causal model, applies a do-operation (intervention) to individual steps, re-executes the trajectory forward, and measures the resulting shift in the outcome distribution, reporting results with confidence intervals. The authors show that common heuristics and LLM-judge attribution are unreliable (state-of-the-art step-level accuracy on the Who&When benchmark is approximately 14%), while CAR's contrastive and budget-bounded Monte-Carlo Shapley estimators recover correct pivotal steps and two-step interactions in synthetic validation experiments. This matters for teams deploying AI agents in enterprise settings, as it provides a principled, open-source tool for diagnosing and attributing agent failures rather than relying on correlational or surface-level debugging approaches.
• ResearcharXiv2026-06-06EP

  Unintended Consequences of Recommender System Interventions: Evidence from a Field Experiment · Shilei Luo, Song Yao, Dennis J. Zhang

  This paper reports a large-scale field experiment on a short-video platform in which a 'sleep reminder' campaign intended to curb late-night usage paradoxically increased late-night engagement by 14.75% and overall platform usage by 2.18%, with effects persisting for weeks after the campaign ended. The authors explain this through a forced-exploration mechanism: the intervention revealed high latent demand for certain content, prompting the recommendation algorithm to update its policy in ways that reinforced the very engagement loops the campaign aimed to reduce. The findings show that user-facing interventions can effectively retrain underlying recommendation algorithms, producing durable, system-wide shifts in content distribution. This challenges standard evaluation metrics used in platform governance and social responsibility initiatives.
• ResearcharXiv2026-06-06EQ

  Closing the Sim-to-Real Gap: An Evaluation Framework for Autonomous Cyber Defense Configuration of Commercial EDR · Kerri Prinos, Lilianne Brush

  This paper presents the first evaluation framework for testing autonomous AI-based cyber defense agents that configure commercial endpoint detection and response (EDR) systems, specifically Microsoft Defender XDR, in a realistic lab environment. Using Horizon3.ai's NodeZero as an autonomous pentester and two LLM backbones (Claude Sonnet 4.6 and Cisco Foundation-Sec-8B), the authors identify a 'sim-to-real gap' between simulated and real-world enterprise defense evaluations. Key findings include that commercial EDR telemetry is optimized for SOC analyst workflows rather than scientific benchmarking, that attribution of defense actions between the AI agent and the EDR's own autonomous behavior is difficult, and that the EDR itself behaves variably during evaluation windows. These results highlight the need for rigorous, real-world evaluation methodology before deploying autonomous defense agents in enterprise environments with black-box AI tools.
• ResearcharXiv2026-06-06QP

  LCAM: A Framework for Diagnosing Interactional Alignment Failures in Con-versational AI · Manuele Reani, Hongyu Tian

  This paper introduces the Layered Cognitive Alignment Model (LCAM), a conceptual framework for identifying and diagnosing failures in how conversational AI systems interact with users—particularly in sensitive contexts like advice-giving, counseling, and decision support. Rather than focusing on model accuracy or preference optimization, LCAM defines alignment across five layers (perceptual, semantic, affective, cognitive, and ethical) and identifies two failure modes—underfit and overreach—to capture harms that emerge through interaction itself, such as simulated empathy, boundary confusion, and erosion of user autonomy. The authors apply LCAM to a published LLM counseling example, demonstrating how an apparently supportive response can reinforce harmful beliefs and obscure role boundaries. By translating these interactional failures into audit and governance questions, LCAM provides a normative lens for evaluating conversational AI systems beyond standard accuracy or helpfulness metrics.
• ResearcharXiv2026-06-06QC

  Human-Centered Benchmarking of Driver Monitoring Models · Ruben Dario Florez-Zela

  This paper proposes the Human-Centered Benchmarking Framework (HCBF), which evaluates driver monitoring models across four dimensions—accuracy, explainability, efficiency, and robustness—rather than classification accuracy alone. Applied to four lightweight architectures (MobileNetV3, ShuffleNetV2, EfficientNet-B0, and DeiT-Tiny) on the MRL Eye Dataset for eye-state classification, the study finds that models nearly indistinguishable on clean-set accuracy diverge sharply on other dimensions, with each architecture leading in exactly one area. ShuffleNetV2 ranks first under a composite Human-Centered Score across multiple deployment scenarios, yet retains less than half its performance under sensor noise and misclassifies closed eyes as open—a safety-critical failure. The findings demonstrate that aggregate rankings can mask dimension-specific vulnerabilities, highlighting the need for multi-dimensional evaluation before deploying models in safety-critical transportation settings.
• ResearcharXiv2026-06-06EQ

  How Small Can You Go? LoRA Fine-Tuning 270M-8B Models for Merchant Information Extraction in Financial Transactions · Donghao Huang, Tomas Drietomsky, Benjamin Barrett et al.

  This paper evaluates whether smaller large language models can replace an 8-billion-parameter LLaMA 3.1-8B model in a production financial transaction system that extracts structured merchant information from noisy bank transaction strings. Using LoRA fine-tuning across 24 model variants (270M to 8B parameters) from four model families, the study finds that a Qwen 3.5 4B model reaches 96.60% F1—within 0.35 points of the 8B baseline—while using roughly half the parameters, and that even a 0.8B Qwen 3.5 model achieves 94.75% F1 with attractive latency trade-offs. The authors also show that chain-of-thought fine-tuning improves F1 by 0.3–1.8 points for most models, and that benchmark performance transfers reliably to production endpoints (average F1 change of only 0.8 points). These findings offer practical deployment guidance for enterprises seeking to reduce memory, latency, and cost in large-scale financial NLP pipelines without sacrificing meaningful accuracy.
• ResearcharXiv2026-06-06QP

  IDP-Bench: Benchmarking ability of LLMs to protect personal information in interdependent privacy contexts · Ayana Hussain, Soumya Sharma, Golnoosh Farnadi et al.

  IDP-Bench is the first benchmark designed to evaluate how well large language models (LLMs) handle interdependent privacy (IDP)—situations where one person's data can be revealed by another without consent—grounded in the Contextual Integrity (CI) framework. Testing eight open-source LLMs reveals that while most models (6/8) recognize co-ownership of information at above 90% accuracy, they persistently struggle to identify key privacy parameters and judge the appropriateness of sharing, with 7/8 models scoring below 74% on IDP-specific parameters and 5/8 scoring below 77% on sharing-appropriateness judgments. Performance improves with model scale but degrades in smaller models, and high prompt sensitivity on IDP-specific questions underscores significant gaps in current LLM privacy capabilities. These findings matter for the deployment of AI personal assistants with access to sensitive user data, signaling the need for more targeted privacy research and evaluation standards.
• ResearcharXiv2026-06-06EQ

  RecurGuard: Runtime Monitoring for Reasoning-Token Consumption Attacks · Abid Aziz, Hafsa Binte Kibria

  RecurGuard is a runtime monitoring system designed to detect attacks that trick reasoning-capable large language models into wasting their token generation budget on injected decoy tasks rather than answering the user's actual query. These attacks cause 'denial of service' (no final answer produced) or 'denial of wallet' (excess billed output tokens), and input-side classifiers often miss them because injected prompts can appear syntactically benign. RecurGuard analyzes exposed reasoning traces in real time using three signals—recurrence rate, volume growth, and progress toward the user's query—terminating generation early if all three remain anomalous over three consecutive chunks. On DS-R1-Qwen-7B, RecurGuard detects 99% of OverThink attacks and 92% of ExtendAttack instances with near-zero false positive rates, though adaptive topical attacks can retain 11.9x amplification with roughly a 50% joint miss rate.
• ResearcharXiv2026-06-06QP

  From `May' to `Is': Certainty Distortion in Language Model Rewriting · Catarina G Belem, Shang Wu, Hongyu Yao et al.

  This paper investigates 'certainty distortion' in language models — the tendency to change how confidently a claim is expressed even when its core meaning is preserved. Studying scientific and medical communication tasks, the authors find that certainty distortion affects up to 75% of LM outputs and is systematically asymmetric: most models are 1.5–2× more likely to inflate expressed certainty than to reduce it. These effects can compound over repeated paraphrasing; for example, claude-haiku-4-5 increases certainty in 20% of medical examples after one iteration, rising to 40% after five. Prompt-based interventions reduce but do not eliminate this bias, raising serious concerns for users relying on LMs in high-stakes domains like medicine and science.
• ResearcharXiv2026-06-06WP

  The atomic structure of work: a micro-action instrument reveals two-pole AI occupational exposure and its decade-scale polar inversion · Shuyao Gao, Minghao Huang

  This paper builds a fine-grained instrument that decomposes 1,961 O*NET occupational work activities into 15,817 atomic micro-actions, clustered into seven semantic classes, to reveal what aggregate AI occupational exposure scores actually average over. It finds two extreme poles—tool-mediated physical execution and planning-and-design—separated by a gap far larger than chance (permutation P < 10⁻⁴; Cliff's δ = 0.80–0.90), with most work falling in a broad, weakly affected middle band. Crucially, the paper shows the identity of the most-exposed pole has inverted since 2013: occupations most at risk from computerisation-era automation (Frey-Osborne) differ systematically from those most exposed in the LLM era, with 2013 automatability declining as linguistic content rises (ρ = −0.40, n = 618). This matters for workforce policy because it suggests AI exposure rankings are era-specific snapshots, and the more durable forecasting object is the underlying structure of work itself.
• ResearcharXiv2026-06-06QP

  Stress-testing medical large language models reveals latent safety pathology beyond benchmark accuracy · Yuan Shen, Xiaojun Wu, Linghua Yu

  This paper introduces AI-MASLD, a stress-testing framework for clinical large language models (LLMs) that goes beyond standard benchmark accuracy to uncover safety-relevant failure modes. Using 240 clinical cases with six narrative perturbation probes, seven models were evaluated on three indices: metabolic index (MI), perturbation flip rate (PFR), and counterfactual fairness index (CFI). Under clean conditions all models performed similarly, but under realistic narrative stress, sharp divergences emerged — quantized models exhibited 'pseudonormalization' where low flip rates masked functional collapse, and medical fine-tuning degraded logical stability, fairness, and information extraction. The findings argue that narrative stress auditing is a necessary complement to accuracy-based evaluation before deploying LLMs in clinical settings.

← Back to the dashboard