AI Frontier Institute

News & Research

The latest AI research and news with real-world stakes — each item sourced, dated, summarized in plain English, and tagged by impact area. Every item is checked against its source before it appears.

• Research

  How Benchmarks Mis-Score Computer-Use Agents

  Zihan Dong, Zhiyuan Ma, Zekun Wang et al.

  arXiv · 2026-07-30

  This paper audits the reliability of benchmarks used to evaluate computer-use agents (CUAs)—AI systems that browse the web and operate desktop software. By examining 150 publicly failure-scored trajectories across five benchmarks, the authors find that 15.3% of FAIL verdicts are incorrect: 10.7% are false negatives from flawed evaluators and 4.7% stem from broken tasks. The study introduces a reliability framework covering task construction, trajectory observation, scoring, and reporting, and shows that verification/feedback and planning failures dominate over execution/grounding errors—a distinction a single scalar success rate cannot capture. The findings matter because misleading benchmark scores can distort our understanding of how capable these agents actually are, with direct implications for enterprise and quality-assurance contexts where CUAs are already being deployed.

  • Quality assurance
  • Enterprise

• Research

  Agentic Method for Deterministic Validation of Legacy Code Migration

  Andras Ferenczi, Jordan Docherty, Mariya Bessonov et al.

  arXiv (Cornell University) · 2026-07-30

  This paper introduces the 'Locksmith Loop,' an agentic test-synthesis method designed to validate the migration of legacy COBOL programs to Java. The approach instruments both the COBOL source and generated Java target in parallel runtime environments, then iteratively searches for test inputs that penetrate program branches and applies parity-preserving mutations to maximize coverage. Across three COBOL-Java case studies ranging from 430 to 4,114 source lines, the method achieved nearly complete branch coverage on two open-source programs and 91.90% branch coverage on an internal production-like program, with the generated Java matching the COBOL reference under deterministic parity checks in all accepted test cases. This work is significant as a novel approach to deterministically validating AI-generated code migrations, addressing a critical quality-assurance challenge in enterprise legacy modernization.

  • Quality assurance
  • Enterprise

• Research

  Fidelity Is Not Safety: Gently-Compressed LLMs Pass Every Data-Free Quality Guard Yet Invent Procedure Steps in Agentic Execution

  I. Kennedy, T. Kennedy

  arXiv · 2026-07-30

  This paper reveals a critical blind spot in standard compression-quality checks for large language models: models compressed with coherent low-rank (SVD) truncation can pass all conventional quality guards—perplexity thresholds, MMLU accuracy, and data-free output-fidelity tests—yet still invent procedure steps that were never in the original instructions when executing structured tasks as agents. Across three model families and architectures, the authors show that the key risk factor is not the magnitude of compression damage but the coherence of the compression error multiplied by its rate, an axis invisible to fidelity-based oracles. Crucially, magnitude pruning matched to the same perplexity level does not produce this hallucination effect, isolating the cause to low-rank compression specifically. The authors propose a data-free two-axis screening statistic (coherent-fraction and error-rate) that successfully flags dangerous builds with fixed thresholds, and conclude that perplexity, MMLU, and fidelity acceptance are insufficient to certify safety for agentic deployment.

  • Quality assurance
  • Certifications

• Research

  Old Tricks, New Models: How Simple Image Transformations Break Modern AI-based Content Moderation

  Marco Alecci, Francesco Marchiori, Iyiola Emmanuel Olatunji et al.

  arXiv · 2026-07-30

  This paper evaluates the robustness of three commercial image-moderation APIs built on large foundation models by testing seven simple, model-agnostic image transformations (such as color inversion and grayscale conversion) in a large-scale black-box setting. The authors find that all three services can be bypassed using these inexpensive transformations — no gradients, surrogate models, or knowledge of the target system required — while the transformed images remain recognizable to humans. Robustness varies significantly across datasets and harm categories, with multimodal content and self-harm showing the greatest vulnerabilities. The paper concludes that foundation-model-based moderation APIs do not by themselves constitute a reliable security boundary and should be deployed as one component of a layered moderation pipeline rather than as standalone filters.

  • Quality assurance
  • AI policy

• Research

  Rethinking LLM-Judged Helpfulness as a Pedagogy Signal: A Pre-Registered Audit Across Tutor Models

  Shuyi Fan, Boyuan Deng, Mengyu Xu et al.

  arXiv · 2026-07-30

  This pre-registered study audits whether general-purpose LLM helpfulness rubrics can distinguish direct answer-giving from pedagogical guidance in AI tutoring systems. Using three tutor models, two LLM judges (Claude Opus and GPT), and 1,179 scored tutor turns, the researchers find that helpfulness scores do not reliably separate pedagogical from conversational policies—the helpfulness ordering even reverses between judges on two of three tutor bases—while a pedagogy-specific rubric achieves perfect rank separation. The study also finds that answer-revealing turns are consistently followed by less independent student work, a judge-invariant result. The authors conclude that tutor evaluation must pair pedagogy-targeted rubrics with deterministic process measures rather than relying on general helpfulness signals.

  • Quality assurance

• Research

  ConMem: Contribution-Aware Memory for Long-Horizon Manufacturing Inspection Logs

  Bingchen Liu, Yuanyuan Fang, Lei Liu et al.

  arXiv · 2026-07-30

  ConMem is a contribution-aware memory framework designed to help large language models reason over long sequences of heterogeneous steel-equipment inspection logs. It segments logs into functional evidence units, scores each unit's diagnostic value using a Shapley-style estimation, and retains only high-value evidence within a constrained memory budget. In real-world experiments, the system achieves 76.0% question-answering accuracy, reduces input tokens by 88.2%, and cuts response time by 86.6% compared to naive 8K-context LLM baselines, while also detecting early-stage seal-wear signals across three inspection cycles. This matters because it demonstrates a practical human-in-the-loop approach to early-risk screening in industrial manufacturing inspection.

  • Quality assurance
  • Enterprise

• Research

  An Instrument to Evaluate Governance Proposals: AI Policy Analysis at Scale

  Paulo Carvao, Claudio Mayrink Verdun, Isabel Adler et al.

  arXiv (Cornell University) · 2026-07-30

  This paper presents a framework for systematically evaluating AI governance proposals by structuring policy analysis around multiple explicit policy attributes rather than binary positions. Using a mixed-methods approach that combines subject-matter expert input with computational text analysis, the framework quantifies the relative emphasis of different policy objectives and visualizes them for cross-policy comparison. The authors also benchmark commercial large language models against a domain-trained, rubric-calibrated model to assess their suitability for this kind of structured policy analysis. The result is a jurisdiction-agnostic tool intended to help policymakers, analysts, and researchers navigate complex AI governance tradeoffs while making analytical assumptions transparent and adjustable.

  • AI policy

• Research

  When AI Does the Work, What Is Learning For? Post-Instrumental Learning and the Risk of Capacity Dissolution

  Kai Yao

  arXiv (Cornell University) · 2026-07-30

  This paper argues that as AI systems grow capable of producing the artifacts—essays, code, plans, decisions—by which institutions traditionally recognize competence, the standard rationale for learning (that humans must produce those outputs themselves) becomes increasingly fragile. The authors introduce the concept of 'post-instrumental learning,' centered on five capacities—end-setting, reason-giving, contestability, refusal/revision, and participation—and warn that delegating outputs to AI risks 'capacity dissolution,' the erosion of these foundational human competencies. Using educational assessment as a central case, they contend that when a polished artifact no longer reliably evidences understanding, institutions must evaluate learners' accountable relationship to AI-mediated work rather than the artifact alone. The policy takeaway is that AI governance should assess not only system performance but whether deployment preserves people's ability to understand, challenge, revise, and take responsibility for AI-mediated practices.

  • AI policy
  • Certifications

• Research

  Scaling, Lock-In, and Proxy Compliance: A Political Economy of Responsible AI

  Florian A. D. Burnat, Brittany I. Davidson

  arXiv (Cornell University) · 2026-07-30

  This paper develops a political-economy model to explain why AI systems can appear compliant with accountability standards while still causing persistent harms. The model shows that vendors, anticipating deployers' limited ability to switch systems due to lock-in, may invest just enough in observable, auditable features to meet a procurement threshold while under-investing in substantive harm mitigation — a 'proxy-compliance equilibrium.' The authors find that policy tools such as independent audit rights, data portability, incident reporting requirements, and outcome-linked liability each address different parts of this gap by changing the incentives of vendors and deployers. The results offer a theoretical explanation for why documentation and standardized evaluations can coexist with real-world harms, and generate testable predictions about the gap between formal compliance and operational outcomes.

  • AI policy
  • Enterprise
  • Certifications

• Research

  A Cross-Architecture Audit of Direction-Based Inference-Time Defences in Vision-Language Models

  Xiangyu Yin, Tora Bodin, Rohan Menon et al.

  arXiv · 2026-07-30

  This paper audits five inference-time defence strategies that subtract directional vectors from the residual stream of vision-language models (VLMs) to prevent jailbreaks, testing them across 15 model-layer combinations from four architectural families. Results show no single defence dominates on both refusal recovery and utility preservation: the image conditioning shift performs best on LLaVA 1.5 and Pixtral 12B, while prompt instructions lead on Qwen2.5 VL and attack-specific residuals lead on Qwen2 VL 2B. The image conditioning direction is not transferable across architectures, even between dimension-compatible model pairs, and the study finds partial geometric overlap between text-only and multimodal refusal directions (mean cosine alignment 0.35, p ≈ 3e-5). The key practical takeaway is that direction-based defences must be calibrated separately for each language decoder family rather than applied universally.

  • Quality assurance

• Research

  Beyond Feeling Better: Capability-Sustaining Emotional Dialogue as a Longitudinal Research Paradigm

  Ming Wang, Jiaqi Wu Young, Wenfang Wu et al.

  arXiv (Cornell University) · 2026-07-30

  This paper proposes 'capability-sustaining emotional dialogue' (CSED) as a new longitudinal research paradigm for AI emotional support systems, arguing that current approaches focus narrowly on immediate relief rather than preserving users' long-term capacities for emotion regulation, coping, and social connection. A PRISMA-ScR-guided audit of 60 system-building papers found that 95% pursue relief-oriented goals, none evaluates capability or longitudinal outcomes, and only 1 considers dependency, autonomy, or termination risk. Analysis of 300 ESConv supporter turns revealed that capability-relevant functions appear in 43% of turns, while reappraisal and self-efficacy support remain rare (4% and 6.7% respectively). The authors outline design commitments, evaluation timescales, and governance considerations to make CSED testable across data, policy, training, and evaluation contexts.

  • AI policy
  • Quality assurance

• Research

  A Structured Knowledge Infrastructure for Domain-Specific Data Asset Discovery

  Mengdi Chen, Yuanxin Huang, Yulin Jiang et al.

  arXiv · 2026-07-30

  This paper addresses a critical failure in enterprise data analytics: generic retrieval-augmented generation (RAG) systems correctly identify relevant data assets only 19.1% of the time (Hit@10) and lack the usage context needed to prevent metric misinterpretation. The authors deploy a two-layer solution at Xiaohongshu's commercial advertising data warehouse—covering 5,300+ Hive tables across 14 domains—combining a Graph-Guided Retriever using a 2,859-node knowledge graph with a Scene-Aware Ranker that incorporates 19-class entity recognition and negative knowledge. The system raises Hit@10 from 19.1% to 96.6% (+77.5 percentage points) and knowledge coverage from 56% to 77%, with end-to-end latency of 4.84–5.33 seconds, while achieving a 71.6x token reduction. This matters for enterprises relying on AI-driven analytics agents, showing that structured, domain-specific knowledge infrastructure can dramatically improve data asset discovery and reduce the risk of analytical errors.

  • Enterprise
  • Quality assurance

• Research

  Stop Shipping AI Agents on Faith: Capability Is Not Production Readiness

  Fouad Bousetouane

  arXiv (Cornell University) · 2026-07-30

  This paper introduces the ProofAgent Index (PAI), a governance readiness framework for AI agents that combines four dimensions—Evaluation, Context, Compliance, and Governance—to determine whether an agent is ready for production deployment, not merely capable of performing a task. The authors implement PAI within an open-source infrastructure called ProofAgent Harness and validate it across healthcare and finance domains, finding that context engineering strongly influences reliability, capability alone does not determine readiness, and governance evidence must remain independently visible rather than averaged into aggregate scores. The core argument is that current release decisions based on demos or behavioral tests are insufficient, and PAI reframes agent deployment as an auditable, evidence-based decision. This matters because it provides organizations with a structured approach to reducing risk when deploying AI agents in regulated, high-stakes environments.

  • Quality assurance
  • Certifications
  • AI policy

• Research

  HALO: Heterogeneous Admission through Localized Obligations for Safe Agentic Execution

  Taewoo Park, Kyeonghyun Yoo, Kiseok Kim et al.

  arXiv · 2026-07-30

  HALO is a runtime admission protocol for agentic AI systems that handles heterogeneous responses—containing notices, requests, handoffs, and actions—by checking each component individually against its declared prerequisites rather than accepting or rejecting the entire response as a unit. The protocol rechecks each action immediately before dispatch and permits blocked actions to be replaced only by fresh candidates, preventing stale or unsupported components from reaching execution. In evaluation, HALO matched all 96 admission expectations, retained 248/248 supported components in structured-response replay (compared to 0/248 for a whole-response policy), and blocked every tested stale route across ten PX4/Gazebo sessions. This matters for quality assurance and safety in agentic AI deployments, where conditions can change between response generation and action execution.

  • Quality assurance
  • Enterprise

• Research

  AWARE-FX: An Auditable Knowledge-Guided AI System for Measuring Corporate Foreign-Exchange Hedging Disclosure

  Qi Wang

  arXiv · 2026-07-30

  AWARE-FX is an auditable AI/NLP system that extracts and scores corporate foreign-exchange hedging disclosures from annual reports. Built on a professional-source lexicon, negation and accounting-status logic, domain-specific financial encoders, and an audit ledger, it processes 543,527 snippets across 24,909 Hong Kong firm-years from 2008–2025. FinBERT achieves higher mean F1 in seven of eight encoder comparisons (temporal F1 ranging from 0.702 to 0.872), and abstaining on the 20% least-confident observations raises F1 by 0.050–0.077; a general-purpose LLM (Qwen3-8B) performs unevenly across label types, showing domain constraints remain necessary. The system's strict FX score is negatively associated with FX exposure in baseline and stress periods, providing external construct validation, while its modular architecture keeps retrieval, classification, uncertainty handling, and aggregation separately auditable.

  • Quality assurance
  • Enterprise

• Research

  Is Solving Better Than Evaluating GenAI Solutions?

  Ethan Dickey, Marios Mertzanidis, Alexandros Psomas

  arXiv (Cornell University) · 2026-07-30

  This randomized A/B crossover study (N=220) in a junior-level algorithms course compared having students evaluate flawed GenAI-generated solutions against traditional problem solving across six assignments. The study found no statistically significant differences in midterm scores, final exam scores, or overall course grades between the two approaches, though students earned higher homework scores when evaluating GenAI solutions. The advantage from evaluation tasks did not transfer to summative assessments, and most students reported no change in study habits—though those who did adapt their strategies found the GenAI-evaluation assignments more helpful. The authors conclude that GenAI-evaluation activities can be introduced without broad performance losses, but meaningful learning gains likely require deliberate scaffolding beyond simple error diagnosis.

  • Quality assurance

• Research

  Evaluating Agentic Bioinformatics through Function, Evidence, and Validation

  Phuc Pham, Truong-Son Hy

  arXiv (Cornell University) · 2026-07-30

  This paper introduces the Function–Evidence–Validation (FEV) framework for evaluating AI agents that plan and execute biological data analyses. Rather than judging agents solely on final outputs or benchmark scores, FEV examines the full inspectable workflow trajectory for demonstrated operations, traceable evidence, and use-case-specific validation. Applying FEV to 109 agentic systems and 28 benchmark resources across genomics, proteomics, drug discovery, and other bioinformatics domains, the authors find that planning and tool execution have advanced faster than replayability, provenance tracking, and prospective empirical testing. The work argues that agentic bioinformatics must be assessed on workflow correctness rather than final-answer correctness to achieve scientific accountability and auditability.

  • Quality assurance

• Research

  Using Large Language Models for Idea Generation in Innovation

  Lennart Meincke, Karan Girotra, Gideon Nave et al.

  arXiv (Cornell University) · 2026-07-30

  This study compares new product ideas generated by GPT-4 (using zero-shot and few-shot prompting) against ideas produced by university students in a product design course. AI-generated ideas outperformed human ideas on average purchase intent, and were seven times more likely to rank in the top 10% of all ideas — a figure the authors call a conservative estimate given AI's higher productivity. However, AI ideas were rated as less novel and showed greater pairwise similarity, especially with few-shot prompting, indicating a less diverse solution space. The findings suggest LLMs offer a substantial advantage for high-quality idea generation in new product development, with some trade-offs around diversity and novelty.

  • Enterprise

• Research

  WitCert: Sound Runtime Risk Observability and Gating for KV-Cache Quantization

  Fanzhe Wei, Li Liu

  arXiv (Cornell University) · 2026-07-30

  WitCert introduces a provably sound runtime monitor for KV-cache quantization in large language model serving, providing per-layer, per-head, per-step upper bounds on the quality degradation caused by cache compression. The system operates at two tiers—a deterministic worst-case bound valid for any quantizer and a tighter probabilistic certificate for a specific INT8 scheme with core theorems machine-checked in Lean 4. Integrated into the SGLang serving framework, meter-driven gating empirically restores output quality floors; for example, raw-cast FP8 recovers from 22.8 to 79.7 on hard RULER tasks, and the certified INT8 cache achieves 1.88× more KV tokens at the same memory. This matters because it shifts KV quantization validation from offline benchmark averages to live, per-request risk observability and automated repair.

  • Quality assurance
  • Certifications

• Research

  Cloned Voices, Real Consequences: Evaluating Bias in Political Deepfake Detection for Electoral Integrity in Brazil

  Lucas Rafael Stefanel Gris, Daniel Casanova, Frederico Santos de Oliveira et al.

  arXiv (Cornell University) · 2026-07-30

  This paper introduces ParlaSpoof-BR, an audio deepfake dataset built from Brazilian Chamber of Deputies recordings and augmented with synthetic speech from text-to-speech and voice conversion models. The authors benchmark state-of-the-art deepfake detectors on this dataset, finding that current systems struggle to generalize consistently to Brazilian Portuguese political speech, with methodological factors like synthesis model choice dominating over demographic disparities. The work highlights the risks AI-generated disinformation poses to electoral integrity and provides a domain-specific benchmark for developing more robust detection tools in an underrepresented political context.

  • AI policy
  • Quality assurance

• Research

  To Add Is Machine, To Delete Is Human: Measuring and Mitigating Deletion Avoidance in LLM Code Editing

  Amir M. Ebrahimi, Mohammed Mehedi Hasan, Aaditya Bhatia et al.

  arXiv (Cornell University) · 2026-07-30

  This paper identifies 'deletion avoidance' in large language models—a systematic tendency to retain code that should be removed during editing—and quantifies it across leading models on SWE-bench Verified. Across five top models, deletion recall against developer patches reaches at most 71.7%, and models locate the right file over 92% of the time but cut the exact required line in under 52% of cases. The authors introduce a new benchmark (CanItDelete) of 200 real-commit tasks requiring only deletions, and show that retrofitting existing tests to check for removal drops frontier model pass rates from 63.2% to 41.9%. Their findings suggest deletion avoidance is an undertrained behavior rather than a fundamental limitation, with post-training on deletion tasks improving both deletion and broader code-editing performance.

  • Quality assurance
  • Enterprise

• Research

  From Process to Evidence: How Computing Can Ground Appropriate Reliance on Legal AI

  James Williams

  arXiv (Cornell University) · 2026-07-30

  This paper examines how courts—specifically the New York court system—are responding to over 1,500 cases involving AI hallucinations in legal filings, finding that official guidance repeatedly calls for evidence (such as error rates and do-not-use lists) that does not yet exist. Instead, courts substitute procedural requirements like training mandates and checklists, placing the greatest burden on those least equipped, such as legal aid programs and self-represented litigants. The authors map legal duties onto human-computer interaction concepts of 'appropriate reliance' and argue the computing community must develop task taxonomies, shared error metrics, benchmarks, and test harnesses to supply the empirical grounding the justice system currently lacks. Without such evidence, oversight of legal AI risks widening rather than closing the justice gap.

  • AI policy
  • Quality assurance

• Research

  Effect of evaluation prompt strategies on LLM-as-a-judge reliability in critical care

  Jia-Yu Yan, Wing-Sum Chan, Ching‐Tang Chiu et al.

  Anaesthesiology Intensive Therapy · 2026-07-30

  This study tested whether the way evaluation prompts are structured affects how reliably large language models can score AI-generated critical care reports compared to human clinicians. Using 90 ICU clinical reports evaluated under three prompt strategies with GPT-4o and o3-mini, the researchers found that a bottom-up incremental prompting approach produced the closest alignment with clinician ratings (ICC = 0.94, mean deviation 0.1), while a top-down decremental strategy showed significantly lower agreement (ICC = 0.82, mean deviation 4.7). The findings demonstrate that prompt design meaningfully influences both scoring patterns and human-AI concordance, highlighting the need for standardized prompt architectures before LLM-as-a-judge systems can be reliably used in clinical AI evaluation.

  • Quality assurance

• Research

  Three futures for the diagnostic radiologist: A structured disagreement about what AI actually changes

  Jan Beger, Amine Korchi, Christoph A. Agten

  European Journal of Radiology Artificial Intelligence · 2026-07-30

  This paper presents three independently authored 2035 job descriptions for diagnostic radiologists, written by two radiologists and a health IT professional to capture optimistic, trade-off, and stratification perspectives on how imaging AI will reshape the profession. All three scenarios agree that routine workloads will be AI-managed and radiologists will bear accountability for AI output, but they diverge sharply on headcount, career security, and whether the profession expands, concentrates, or stratifies into differentiated tiers. The paper concludes that AI will not eliminate diagnostic radiologists, but that workforce outcomes depend on health system decisions not yet made. It is directly relevant to workforce planning in radiology, highlighting that both optimism and economic caution are simultaneously defensible.

  • Workforce
  • AI policy

• Research

  FORENSIC ACCOUNTING TECHNOLOGIES AND OCCUPATIONAL FRAUD MITIGATION IN THE NIGERIAN MARITIME SECTOR

  Malik Sidique Muhammad, Ajape Mohammed Kayode

  Journal of Business Management and Accounting · 2026-07-30

  This study finds that forensic accounting technologies—including AI-driven analytics, blockchain traceability, cybersecurity resilience, regulatory enforcement, and organizational technology readiness—each have statistically significant positive effects on occupational fraud detection in Nigeria's maritime sector. AI-driven forensic analytics was the strongest predictor (β = 0.391, p < 0.001), and together the five factors explained 63% of the variance in fraud detection effectiveness across 100 surveyed maritime organizations. The findings support integrated adoption of these technologies to strengthen transparency, accountability, and governance in the sector.

  • Enterprise
  • Quality assurance

← Back to the dashboard