Every week seems to bring another story about an organization investing heavily in artificial intelligence and then struggling to answer a basic question:
What did we actually get from it?
The problem is not always the AI system itself. It’s often the evidence available to the people deciding whether, where, and how to use it.
Many organizations rely on information about how an AI model performs on a benchmark or in a controlled test. Some organizations conduct local pilots, audits, or evaluations that closely reflect one specific setting. Both forms of evidence provide key insights, but neither, on its own, gives organizational decision-makers a comprehensive view of what happens when AI enters everyday work.
Our recent article in the “Consequences of AI” special issue of Social Science Computer Review addresses this challenge. In Wild, Thick, and Wicked: Situated Evidence on AI-In-Use for Decisions About Deploying AI Systems, we propose a framework for generating a different type of evidence, one that can better support real-world decisions about deploying, adopting, and managing AI.
At the center of the framework is a simple idea: AI-in-use.
AI-in-use means looking at what happens when people encounter AI systems in their actual environments, how they use them, adapt to them, rely on them, question them, and sometimes work around them. It also means examining the outcomes and consequences that emerge over time.
This perspective helps connect a system’s technical capabilities with its real-world outcomes. It gives leaders, practitioners, communities, and policymakers a way to ask not only, “What can this system do?” but also:
• What does it actually do in practice?
• What happens when the system meets real users, real tasks, and real constraints?
• What value does it provide in setting?
• Who takes on new work, risks, or responsibilities?
• How does it change decisions, workflows, and accountability?
• What are the longer term outcomes of the deployment?
Three Layers of Evidence
Today’s AI evaluation ecosystem produces two common forms of evidence. Each useful but also limited for organizational decision making about AI deployments.
• Abstract evidence comes from benchmarks, model evaluations, and lab-style tests. It helps developers compare systems and improve performance. Because these test outputs are standardized, it is easy to compare one system against another, but they generate little to no information about what materializes when people use these tools in real workflows and tasks, or their longer-term outcomes.
• Contextual evidence comes from local pilots, audits, and site-specific evaluations. It can be rich in detail and closely connected to the questions of interest within particular organization or community. The localized nature of this evidence can make it difficult to compare across settings or tie outcomes to specific model behavior.
Our paper argues for a third layer: Systematic Evidence.
Systematic evidence is generated through repeated observation of AI systems as they are used in real-world workflows and tasks at scale. It does not replace benchmarks or local evaluation. It gives stakeholders outside of the AI development stack the details to connect system behavior to the deployment context, and downstream consequences.
To build this evidence base, we need to take three realities of deployed AI seriously.
Wild: Real deployment settings are not controlled test environments. People work under time pressure, with incomplete information, competing priorities, and existing tools and practices. Evaluation must extend beyond ideal conditions to account for these factors.
Thick: Understanding AI requires close attention to context. We need to observe how people actually use a system, how they modify their practices around it, and how it fits, or fails to fit, within existing workflows.
Wicked: The consequences that matter most are often complex and evolving. AI may redistribute work, change reliance on professional judgment, create new forms of rework, or shift accountability in ways that cannot be captured by one performance score or a single moment of evaluation.
Four Shifts for Better Evaluation
Our paper lays out four shifts that we think are needed to build stronger evidence for making decisions about AI deployments.
1. Design evaluation for decision-makers.
Evaluation should serve the organizations, practitioners, and communities deciding whether and how to adopt AI, not only the developers optimizing a model.
2. Measure the full range of effects.
Accuracy and safety remain important, but evaluation should also examine value creation, workload changes, rework, reliance, access, and the distribution of benefits and burdens.
3. Treat users as knowledgeable observers.
People who work with AI systems every day are not merely test participants. They are first-hand reporters of how systems function in practice and where risks, workarounds, or unexpected effects emerge.
4. Study variation rather than treating it as noise.
AI systems may work differently across users, tasks, teams, and settings. That variation is often the evidence organizations need most when deciding whether a system is appropriate for broader use.
From Framework to Practice
We also propose a shared evaluation architecture for producing this kind of evidence in this paper. At Virginia State University’s Center for Responsible AI, we are developing this approach through the Forum for Real-World AI Measurement and Evaluation (FRAME).
FRAME is intended to help make real-world AI evaluation more rigorous, comparable, and useful for decision-making. Its emerging architecture includes:
A structured observation environment where diverse panels of users engage with deployed or near-deployed AI systems on meaningful tasks that resemble real settings of use.
A metrics hub that translates how people use AI systems, and the outcomes that follow, into practical indicators for organizations, sectors, and decision-makers.
Reusable consortium models that help identify how AI system behavior varies across contexts, to allow lessons learned in one setting inform decisions in others without assuming every context is the same.
An Evidence Challenge
AI deployment is not only a technology or benchmarking challenge. It is an evidence challenge. Organizations need credible ways to understand how AI performs once it enters the complexity of everyday work and public life. They need evidence that reflects not just model capability, but adoption, adaptation, burden, value, accountability, and consequences over time.
Our paper describes the kind of evidence that is missing. FRAME is an effort to build the methods, infrastructure, and partnerships needed to produce it.
If your organization is deploying AI and wrestling with the evidence gap this paper describes, we would like to hear from you.
Paper link