Certification and Trustworthy AI

Conversational Agents in Multi-User Environments

---

Abstract

Passing as human in a room full of people requires more than fluent speech, it demands reading the room. While Large Language Models (LLMs) have transformed human-AI interaction in a one-to-one setting, they still fall short in multi-user conversational settings where social dynamics define the interaction. In such environments, a conversational agent that merely generates coherent text will struggle to maintain consistent socially plausible behavior. We propose a structured Theory of Mind (ToM) framework that equips conversational agents with the cognitive machinery to reason over participant beliefs, intentions, and evolving group dynamics in real time. Rather than relying on a single LLM prompt, our architecture decomposes social reasoning into explicit modules---a knowledge base, belief system, goal generator, and intention planner---coupled with a dual-process response architecture that balances immediacy with strategic depth. To evaluate this approach, we deployed the framework within the Turing Game and Reverse Turing Game environments, further enhancing the agent's plausibility with a simulated human-like response timing algorithm. Preliminary evaluations demonstrate that our ToM-equipped agent exhibits stronger conversational coherence, sustains longer exchanges, and is less frequently identified as a bot compared to its predecessor without structured social reasoning.

How to Cite:

Tanriverdi, U., Halmdienst, T., Schmid, S., Nessler, B. & Lewandowski, M., (2026) “Conversational Agents in Multi-User Environments”, Proceedings of the Austrian Symposium on AI, Robotics, and Vision 3(1), 282-295.