Friends with Benefits: How to Leverage Interdisciplinarity and Level Up Your AI Team
Contextual Awareness is All You Need Post #10: Reva Schwartz
Recent reporting [1] about very low uptake of AI products across industry lays bare a number of challenges. From a real-world AI evaluation perspective, there is a major bottleneck between the tech industry’s almost singular focus on producing raw AI capabilities and the market’s demand for compelling applications that integrate seamlessly into the existing tech ecosystem.
There are some key levers that organizations can leverage to overcome this bottleneck, and they are built on a foundational need to integrate a diverse set of real-world inputs into AI evaluation practice. Consider an organization that wants to assess the impacts of hallucinations in the legal domain. Instead of relying solely on existing legal benchmarks, a real-world AI approach would define the measurement challenge, specify key contextual factors in collaboration with stakeholders, and collect data from user interactions with AI systems under quasi-realistic conditions. To support this type of work, teams must have the necessary expertise to effectively elicit, capture, and translate a diverse stream of real-world inputs at scale for integration within computational processes. Real-world AI evaluation teams may include disciplines such as:
Interdisciplinarity plays a vital role in producing safe AI applications that meet public expectations, yet it is notoriously difficult to set in motion and maintain. The broad range of disciplines displayed above is typically only found in highly resourced and specialized settings. Non-Machine Learning (ML) experts on the AI evaluation team must also be familiar with ML models, their inherent capabilities, and the fields of trustworthy AI, responsible AI, and AI ethics. For these reasons, and more, interdisciplinarity rarely materializes naturally.
Interdisciplinarity’s superpower lies in its ability to encourage open dialogue and discussion among team members. When individuals from diverse fields collaborate, their lack of familiarity with each other's domains can highlight gaps in understanding and encourage a "fail-fast" environment. Done well, this kind of environment can allow teams and organizations to efficiently abandon non-viable processes, mitigate losses, and course-correct. Without organizational commitment and empowerment, team members may struggle for recognition, leading to power imbalances. In tech settings, this often means unequal decision making, with power concentrated in the ML-related disciplines.
The Role of Language in Interdisciplinary Collaboration
A team of four to six individuals from vastly different fields requires mutual understanding to succeed. One common communication roadblock is "polysemy”, which refers to the ability of words to have multiple meanings. This phenomenon, often experienced as “talking past each other” occurs when team members bring their own interpretation of words into a discussion without clarifying their meaning, and can easily lead to misunderstandings and miscommunications.
Polysemy can also exacerbate the power dynamics that exist in tech settings. The computational disciplines in AI have a highly specialized lexicon. Terms such as “FLoPS” (floating point operations per second), “chain-of-thought reasoning”, “backpropagation”, and “feed-forward neural networks” are unlikely to have multiple meanings–no matter where they are used. In stark contrast, more common words like "bias," "responsible," "ethical," "fair," "risk," and "safety" can have many meanings in a given workplace setting. These differences in language use can cleave the culture of interdisciplinary teams. The disciplines using the more specialized lexicon have higher prestige and get to (quite literally) set the terms of the debate. Team members whose vocabulary consists of more common terms have to pay a higher cost to access the discussion, and clarify their own language in the process.
This disciplinary prestige has other effects. Policy makers and journalists adopt ML terms like “one-hot encoding” and “zero-shot learning” in an effort to gain access and legitimacy within AI. While these technical terms retain their specific meanings across boundaries, more common polysemous words can be easily co-opted and redefined–leading to increased ambiguity. For example, compound terms like “socio-technical” and “human-in-the-loop" have lost their definitional boundaries, contributing to ongoing conceptual confusion in the AI community.
Team members from socio-technical disciplines often serve as translators in multiple ways. First, they conduct translation as part of their evaluation tasks, converting the diverse feedback collected from real-world testing into usable inputs for downstream processing on the AI lifecycle. And, since their lexicon is more flexible, they may be called on to connect parts of an interdisciplinary team and reduce misunderstandings.
The Example of “AI Alignment”
The interaction between disciplinary prestige and the adoption of technical vocabulary is currently playing out for another term whose meaning has shifted within AI discourse. The term “alignment” isn’t new to AI but has been recently repurposed by the AI safety community to describe the process of ensuring that system outputs align with desired objectives and human values. This re-framing of “alignment” has accumulated multiple interpretations across the various disciplinary communities in AI, including computational, ethical, and socio-technical perspectives.
From a computational perspective, alignment tends to refer to how well AI system outputs meet prespecified quantitative performance metrics related to human values and preferences. Ethical perspectives on alignment seek to move beyond mere technical correctness to assure AI system fairness, accountability, and transparency. Finally, a socio-technical lens on alignment is contextually focused and considers whether human, organizational, and societal factors are accounted for in AI system design and development.
Without a consideration of the power dynamics underlying AI system development, the polysemy of alignment discourse risks being dominated by the single discipline with the most prestige; machine learning. This dominance can perpetuate a system-centric framing of AI’s effects and an incomplete understanding of the factors at the center of the alignment debate.
Balancing Complex Factors
Interdisciplinary teams must consistently navigate these complex factors. Without strong organizational commitment and empowerment, the benefits of interdisciplinary teams—and the innovation they can generate—can be impeded. In the longer term, a lack of support for interdisciplinarity can more deeply embed a single perspective on AI’s potential. This can further stagnate AI adoption by making it even more challenging to transform AI model capabilities into applications that deliver real value. Organizations can integrate the following practices and create a gentle slope for interdisciplinary culture to thrive:
Recognize Gatekeeping: Be mindful of team culture that tends to favor a single disciplinary perspective or limits access to information and resources.
Foster Connection: Use informal non-work-related discussions and social events to build up team rapport and strengthen relationships.
Facilitate Collaborative Learning: Create opportunities for team members to share their expertise, exchange methods, and acknowledge each other's contributions.
Make the Implicit Explicit: Surface communicative misunderstandings by empowering team members to ask clarifying questions and paraphrase what others have said.
Highlight Differences: Integrate terminology from all disciplines into a shared vocabulary to highlight lexical differences, minimize confusion, and strengthen cross-disciplinary insights.
Notes and Additional reading
[1] https://fortune.com/2025/08/18/mit-report-95-percent-generative-ai-pilots-at-companies-failing-cfo/
[2] Fazelpour, S., & De-Arteaga, M. (2022). Diversity in sociotechnical machine learning systems. Big Data & Society, 9(1). https://doi.org/10.1177/20539517221082027
[3] Slota, S. C. ; Fleischmann, K. R. ; Greenberg, S. ; Verma, N. ; Cummings, B. ; Li, L. & Shenefiel, C. (2023). Many hands make many fingers to point: challenges in creating accountable AI. AI and Society 38 (4):1287-1299.
[4] Eckert P (2018) Meaning and Linguistic Variation: The Third Wave in Sociolinguistics. Cambridge, UK: Cambridge University Press.
[5] Liao, Q.V., & Xiao, Z. (2023). Rethinking Model Evaluation as Narrowing the Socio-Technical Gap. ArXiv, abs/2306.03100.