Publications
My publications in reverse chronological order.
2026
- UIST
CoNarrate: Computationally Modeling Information Prematureness and Scaffolding the Co-Construction of Patient Narratives in Asynchronous TelehealthDingdong Liu, Yun Wang, Hei Ching Iu, Li Zhang, Chuhan Shi, Yan Lu, and Xiaojuan MaIn Proceedings of the ACM Symposium on User Interface Software and Technology. Conditionally accepted , 2026Work done during an internship at Microsoft Research Asia.An independent project to improve telehealth communication, analyzing 5,000+ encounters and resulting in a taxonomy of information-delivery failures. Developed a conversational agent that significantly accelerated communication and improved information accuracy and relevance in patient narratives.
- Under Review
Dissecting Interaction Gulfs: Towards Automating Neurocognitive Disorder Screening Dialog TasksDingdong Liu, Lueyang Zhang, Junze Li, Hei Ching Iu, Bolin Zhao, Ka Ho Wong, Helen Meng, Xiaojuan Ma, and Jiaxiong HuManuscript under review , 2026Deploys a conversational agent combining coordination-level and content-level communicative abilities in a neurocognitive disorder screening scenario, where older adults must make sense of an unfamiliar and complex system. Using the gulfs of execution and evaluation to analyze interactions with 38 older adults, we identify the interaction gulfs that can undermine test validity and derive design considerations for automating screening dialog tasks.
- From Human Pragmatic Language Skills to Conversational Agent Design: A Systematic Review of Transfer StrategiesJiaxiong Hu, Xiwen Yao, Zeyu Huang, Danxuan Liang, Dongjie Yang, Dingdong Liu, Junze Li, Yuanhao Zhang, and Xiaojuan MaIn Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems, 2026
While conversational agents’ (CAs) semantic and syntactic capabilities have advanced, their pragmatic skills , using language appropriately in context , have emerged as a critical focus in practical applications. Hence, scholars integrate conversational skills derived from human-human interaction into CA designs. However, existing research mainly adopts an empirical approach and focuses on specific CA deployment, making it challenging to identify overarching patterns or develop a comprehensive methodology for transferring human pragmatic skills to CA design. Thus, we conducted a systematic review of 85 studies from primary databases (e.g., ACM, IEEE, etc.), focusing on designing CAs with human-derived conversational skills. We identified skill categories (verbal, paralinguistic, nonverbal), transfer strategies (from dialog data, theories, and via co-design), implementations, and evaluation metrics. We consolidated these insights into a four-stage design process: human skill exploration, definition, transfer, and iterative evaluation. Future research can leverage this to design CAs that achieve conversational goals through contextually appropriate language use.
@inproceedings{huFromHumanPragmaticLanguage2026, author = {Hu, Jiaxiong and Yao, Xiwen and Huang, Zeyu and Liang, Danxuan and Yang, Dongjie and Liu, Dingdong and Li, Junze and Zhang, Yuanhao and Ma, Xiaojuan}, title = {From Human Pragmatic Language Skills to Conversational Agent Design: A Systematic Review of Transfer Strategies}, year = {2026}, isbn = {9798400722783}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3772318.3792787}, doi = {10.1145/3772318.3792787}, booktitle = {Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems}, articleno = {911}, numpages = {21}, keywords = {Conversational Agent, Design Process, Conversational Skill, Pragmatic Skill, Systematic Literature Review}, series = {CHI '26} }
2025
- InsightBridge: Enhancing Empathizing with Users through Real-Time Information Synthesis and Visual CommunicationJunze Li, Yue Zhang, Chengbo Zheng, Dingdong Liu, Zeyu Huang, and Xiaojuan MaIn Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, 2025
User-centered design necessitates researchers deeply understanding target users throughout the design process. However, during early-stage user interviews, researchers may misinterpret users due to time constraints, incorrect assumptions, and communication barriers. To address this challenge, we introduce InsightBridge , a tool that supports real-time, AI-assisted information synthesis and visual-based verification. InsightBridge automatically organizes relevant information from ongoing interview conversations into an empathy map. It further allows researchers to specify elements to generate visual abstracts depicting the selected information, and then review these visuals with users to refine the visuals as needed. We evaluated the effectiveness of InsightBridge through a within-subject study (N=32) from both the researchers’ and users’ perspectives. Our findings indicate that InsightBridge can assist researchers in note-taking and organization, as well as in-time visual checking, thereby enhancing mutual understanding with users. Additionally, users’ discussions of visuals prompt them to recall overlooked details and scenarios, leading to more insightful ideas.
@inproceedings{liInsightbridgeEnhancingEmpathizingUsers2025, author = {Li, Junze and Zhang, Yue and Zheng, Chengbo and Liu, Dingdong and Huang, Zeyu and Ma, Xiaojuan}, title = {InsightBridge: Enhancing Empathizing with Users through Real-Time Information Synthesis and Visual Communication}, year = {2025}, isbn = {9798400713941}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3706598.3713640}, doi = {10.1145/3706598.3713640}, booktitle = {Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems}, articleno = {167}, numpages = {15}, keywords = {User-centered Design, Human-AI Collaboration, Empathy Map, Visual Communication}, series = {CHI '25} } - Scaffolded Turns and Logical Conversations: Designing Humanized LLM-Powered Conversational Agents for Hospital Admission InterviewsDingdong Liu, Yujing Zhang, Bolin Zhao, Shuai Ma, Chuhan Shi, and Xiaojuan MaIn Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, 2025
Hospital admission interviews are critical for patient care but strain nurses’ capacity due to time constraints and staffing shortages. While LLM-powered conversational agents (CAs) offer automation potential, their rigid sequencing and lack of humanized communication skills risk misunderstandings and incomplete data capture. Through participatory design with clinicians and volunteers, we identified essential communication strategies and developed a novel CA that implements these strategies through: (1) dynamic topic management using graph-based conversation flows, and (2) context-aware scaffolding with few-shot prompt tuning. Technical evaluation on an admission interview dataset showed our system achieving performance comparable to or surpassing human-written ground truth, while outperforming prompt-engineered baselines. A between-subject study (N=44) demonstrated significantly improved user experience and data collection accuracy compared to existing solutions. We contribute a framework for humanizing medical CAs by translating clinician expertise into algorithmic strategies, alongside empirical insights for balancing efficiency and empathy in healthcare interactions, and considerations for generalizability.
@inproceedings{liuScaffoldedTurnsAndLogicalConversationsDesigning2025, author = {Liu, Dingdong and Zhang, Yujing and Zhao, Bolin and Ma, Shuai and Shi, Chuhan and Ma, Xiaojuan}, title = {Scaffolded Turns and Logical Conversations: Designing Humanized LLM-Powered Conversational Agents for Hospital Admission Interviews}, year = {2025}, isbn = {9798400713941}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3706598.3714196}, doi = {10.1145/3706598.3714196}, booktitle = {Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems}, articleno = {643}, numpages = {23}, keywords = {Conversational Agents, Clinical Communication, Hospital Admission Interview, Healthcare Automation, Scaffolded Dialogue, Participatory Design, Large Language Models}, series = {CHI '25} } - StuGPTViz: A Visual Analytics Approach to Understand Student-ChatGPT InteractionsZixin Chen, Jiachen Wang, Meng Xia, Kento Shigyo, Dingdong Liu, Rong Zhang, and Huamin QuIEEE Transactions on Visualization and Computer Graphics, 2025
The integration of Large Language Models (LLMs), especially ChatGPT, into education is poised to revolutionize students’ learning experiences by introducing innovative conversational learning methodologies. To empower students to fully leverage the capabilities of ChatGPT in educational scenarios, understanding students’ interaction patterns with ChatGPT is crucial for instructors. However, this endeavor is challenging due to the absence of datasets focused on student-ChatGPT conversations and the complexities in identifying and analyzing the evolutional interaction patterns within conversations. To address these challenges, we collected conversational data from 48 students interacting with ChatGPT in a master’s level data visualization course over one semester. We then developed a coding scheme, grounded in the literature on cognitive levels and thematic analysis, to categorize students’ interaction patterns with ChatGPT. Furthermore, we present a visual analytics system, StuGPTViz, that tracks and compares temporal patterns in student prompts and the quality of ChatGPT’s responses at multiple scales, revealing significant pedagogical insights for instructors. We validated the system’s effectiveness through expert interviews with six data visualization instructors and three case studies. The results confirmed StuGPTViz’s capacity to enhance educators’ insights into the pedagogical value of ChatGPT. We also discussed the potential research opportunities of applying visual analytics in education and developing AI-driven personalized learning solutions.
@article{chenStuGPTVizA2025, author = {Chen, Zixin and Wang, Jiachen and Xia, Meng and Shigyo, Kento and Liu, Dingdong and Zhang, Rong and Qu, Huamin}, journal = {IEEE Transactions on Visualization and Computer Graphics}, title = {StuGPTViz: A Visual Analytics Approach to Understand Student-ChatGPT Interactions}, year = {2025}, volume = {31}, number = {1}, pages = {908-918}, keywords = {Data visualization;Chatbots;Oral communication;Education;Visual analytics;Artificial intelligence;Data collection;Visual analytics for education;ChatGPT for education;student-ChatGPT interaction}, doi = {10.1109/TVCG.2024.3456363} } - Dynamic Prompting Improves Turn-taking in Embodied Spoken Dialogue SystemsYifan Shen, Dingdong Liu, Xiaoyu Mo, Fugee Tsung, Xiaojuan Ma, and Bertram E. ShiIn 2025 34th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN). Co-first author, equal contribution , 2025
The ability to coordinate turn taking during spoken dialogue is crucial for an embodied spoken dialogue system (SDS), e.g., in a humanoid robot. The SDS needs to model transitions in the conversational floor, which describes each party’s stance (either speaking or listening). Further, the SDS needs to signal its perception of the floor to the human, so that they can coordinate floor transitions and resolve conflicts. Conventional SDS employ standalone modules to control floor transitions but do not produce timely and appropriate responses. Recent end-to-end audio LLMs generate responses quickly, but do not coordinate floor transitions as accurately. In this work, we propose an SDS architecture that dynamically adjusts its prompts to an end-to-end audio LLM based upon its perception of the conversational floor state. The LLM output determines not only the audio output, but also the perceived floor state. This enables the system to signal its stance to the human, both when listening and when speaking. We conducted an experiment where a humanoid robot administered a semi-structured interview with human subjects. Results show that, compared with baseline systems using static prompts, dynamic prompting enables the LLM to model floor transitions more accurately, to generate more appropriate signalling, and to interrupt less, leading to smoother turn-taking in dialogue.
@inproceedings{shenDynamicPromptingImproves2025, author = {Shen, Yifan and Liu, Dingdong and Mo, Xiaoyu and Tsung, Fugee and Ma, Xiaojuan and Shi, Bertram E.}, booktitle = {2025 34th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN)}, title = {Dynamic Prompting Improves Turn-taking in Embodied Spoken Dialogue Systems}, year = {2025}, pages = {692-699}, keywords = {Navigation;Robot kinematics;Humanoid robots;Systems architecture;Oral communication;Manuals;History;Interviews;Floors;Signal resolution}, doi = {10.1109/RO-MAN63969.2025.11217726} }
2024
- 🏆 A Humanoid Robot Dialogue System Architecture Targeting Patient Interview TasksYifan Shen, Dingdong Liu, Yejin Bang, Ho Shu Chan, Rita Frieske, Hoo Choun Chung, Jay Nieles, Tianjia Zhang, Kien T. Pham, Wai Yi Rosita Cheng, Yini Fang, Qifeng Chen, Pascale Fung, Xiaojuan Ma, and Bertram E. ShiIn 2024 33rd IEEE International Conference on Robot and Human Interactive Communication (ROMAN). Co-first author, equal contribution , 2024
UBTECH Best Industry Application Award (Bronze), IEEE RO-MAN 2024.
Humanoid robots are promising approach to automating patient interviews routinely conducted by medical staff. Their human-like appearance enables them to use the full gamut of verbal and behavioral cues that are critical to a successful interview. On the other hand, anthropomorphism can induce expectations of human-level performance by the robot. Not meeting such expectations degrades the quality of interaction. Specifically, humans expect rich real-time interactions during speech exchange, such as backchanneling and barge-ins. The nature of the patient interview task differs from most other scenarios where task oriented dialogue systems have been used, as there is increased potential of engagement breakdown during interaction. We describe a dialogue system architecture that improves the performance of humanoid robots on the patient interview task. Our architecture adds a nested inner real-time control loop to improve the timeliness of the robot’s responses based on the notion of "stance", an elaboration of the concept of a "turn", common in most existing dialogue systems. It also expands the dialogue state to monitor not only task progress, but also human engagement. Experiments using a humanoid robot running our proposed architecture reveal improved performance on interview tasks in terms of the perceived timeliness of responses and users’ impressions of the system.
@inproceedings{shenHumanoidRobotDialogueSystemArchitecture2024, award_photo = {roman2024-award-photo.jpg}, author = {Shen, Yifan and Liu, Dingdong and Bang, Yejin and Chan, Ho Shu and Frieske, Rita and Chung, Hoo Choun and Nieles, Jay and Zhang, Tianjia and Pham, Kien T. and Cheng, Wai Yi Rosita and Fang, Yini and Chen, Qifeng and Fung, Pascale and Ma, Xiaojuan and Shi, Bertram E.}, booktitle = {2024 33rd IEEE International Conference on Robot and Human Interactive Communication (ROMAN)}, title = {A Humanoid Robot Dialogue System Architecture Targeting Patient Interview Tasks}, year = {2024}, pages = {1394-1401}, keywords = {Target tracking;Navigation;Robot kinematics;Humanoid robots;Systems architecture;Real-time systems;History;Interviews;Standards;Monitoring}, doi = {10.1109/RO-MAN60168.2024.10731285} } - Exploring Scaffolding Techniques for Agent-Administered Brief Cognitive Screening in Hospital SettingsDingdong Liu, Sensen Gao, Zixin Chen, Yifan Shen, Chuhan Shi, Bertram E. Shi, and Xiaojuan MaIn Companion Publication of the 2024 ACM Designing Interactive Systems Conference, IT University of Copenhagen, Denmark, 2024
Cognitive screening in hospitalized older patients is a critical, yet time-consuming process. While conversational agents present a promising solution to aid clinicians, current models fall short in their ability to scaffold questions to accommodate patients with potential cognitive decline effectively. To bridge this gap, we conducted a study with 13 clinicians to identify effective scaffolding strategies empirically. Our findings revealed six key strategies that clinicians use to scaffold the Abbreviated Mental Test (AMT) in practice, together with the underlying rationale and potential challenges. We discuss the implications of these findings for the design of conversational agents to assist in cognitive screening and propose design considerations for future research.
@inproceedings{liuExploringScaffoldingTechniques2024, author = {Liu, Dingdong and Gao, Sensen and Chen, Zixin and Shen, Yifan and Shi, Chuhan and Shi, Bertram E. and Ma, Xiaojuan}, title = {Exploring Scaffolding Techniques for Agent-Administered Brief Cognitive Screening in Hospital Settings}, year = {2024}, isbn = {9798400706325}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3656156.3663697}, doi = {10.1145/3656156.3663697}, booktitle = {Companion Publication of the 2024 ACM Designing Interactive Systems Conference}, pages = {185–189}, numpages = {5}, keywords = {cognitive screening, conversational agent, scaffolding}, location = {IT University of Copenhagen, Denmark}, series = {DIS '24 Companion} }
2023
- CoArgue : Fostering Lurkers’ Contribution to Collective Arguments in Community-based QA PlatformsChengzhong Liu, Shixu Zhou, Dingdong Liu, Junze Li, Zeyu Huang, and Xiaojuan MaIn Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, Hamburg, Germany, 2023
In Community-Based Question Answering (CQA) platforms, people can participate in discussions about non-factoid topics by marking their stances, providing premises, or arguing for the opinions they support, which forms “collective arguments”. The sustainable development of collective arguments relies on a big contributor base, yet most of the frequent CQA users are lurkers who seldom speak out. With a formative study, we identified detailed obstacles preventing lurkers from contributing to collective arguments. We consequently designed a processing pipeline for extracting and summarizing augmentative elements from question threads. Based on this we built CoArgue, a tool with navigation and chatbot features to support CQA lurkers’ motivation and ability in making contributions. Through a within-subject study (N=24), we found that, compared to a Quora-like baseline, participants perceived CoArgue as significantly more useful in enhancing their motivation and ability to join collective arguments and found the experience to be more engaging and productive.
@inproceedings{liuCoArgueFostering2023, author = {Liu, Chengzhong and Zhou, Shixu and Liu, Dingdong and Li, Junze and Huang, Zeyu and Ma, Xiaojuan}, title = {CoArgue : Fostering Lurkers’ Contribution to Collective Arguments in Community-based QA Platforms}, year = {2023}, isbn = {9781450394215}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3544548.3580932}, doi = {10.1145/3544548.3580932}, booktitle = {Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems}, articleno = {271}, numpages = {17}, keywords = {CQA Platforms, Collective Arguments, Lurker Support}, location = {Hamburg, Germany}, series = {CHI '23} }
2022
- Exploring the Effects of Self-Mockery to Improve Task-Oriented Chatbot’s Social IntelligenceChengzhong Liu, Shixu Zhou, Yuanhao Zhang, Dingdong Liu, Zhenhui Peng, and Xiaojuan MaIn Proceedings of the 2022 ACM Designing Interactive Systems Conference, Virtual Event, Australia, 2022
An effective task-oriented chatbot should be able to exert a certain level of Social Intelligence (SI), the ability to emulate human social behaviors to reduce user frustration and dissatisfaction. However, few studies explored using humor, a common rhetorical device in human-human interactions, to improve chatbots’ overall SI. To fill this gap, we proposed to apply self-mockery humor to a customer service chatbot in different interaction stages with users. We proposed a pipeline to create situated self-mockery for the chatbot and conducted a within-subject experiment (N=28) to compare it with a chatbot without self-mockery utterance. Results showed that the self-mockery chatbot was perceived as significantly funnier, more satisfactory, and delivering higher performance in two out of the five measured characteristics of SI with comparable performance in the rest. We further discussed how participants’ individual factors might affect the perceived helpfulness of self-mockery on SI and concluded with design considerations.
@inproceedings{liuExploringTheEffects2022, author = {Liu, Chengzhong and Zhou, Shixu and Zhang, Yuanhao and Liu, Dingdong and Peng, Zhenhui and Ma, Xiaojuan}, title = {Exploring the Effects of Self-Mockery to Improve Task-Oriented Chatbot’s Social Intelligence}, year = {2022}, isbn = {9781450393584}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3532106.3533461}, doi = {10.1145/3532106.3533461}, booktitle = {Proceedings of the 2022 ACM Designing Interactive Systems Conference}, pages = {1315–1329}, numpages = {15}, keywords = {Chatbots, Conversational Agents, Self-Mockery, Social Intelligence}, location = {Virtual Event, Australia}, series = {DIS '22} } - PlanHelper: Supporting Activity Plan Construction with Answer Posts in Community-based QA PlatformsChengzhong Liu, Zeyu Huang, Dingdong Liu, Shixu Zhou, Zhenhui Peng, and Xiaojuan MaProc. ACM Hum.-Comput. Interact., Nov 2022
Community-based Question Answering (CQA) platforms can provide rich experience and suggestions for people who seek to construct Activity Plans (AP), such as bodybuilding or sightseeing. However, answer posts in CQA platforms could be too unstructured and overwhelming to be easily applied to AP construction, as validated by our formative study for understanding relevant user challenges. We therefore proposed an answer-post processing pipeline, based on which we built PlanHelper, a tool assisting users in processing the CQA information and constructing AP interactively. We conducted a within-subject study (N=24) with a Quora-like interface as the baseline. Results suggested that when creating AP with PlanHelper, users were significantly more satisfied with the informational support and more engaged during the interaction. Moreover, we performed an in-depth analysis on the user behaviors with PlanHelper and summarized the design considerations for such supporting tools.
@article{liuPlanHelperSupporting2022, author = {Liu, Chengzhong and Huang, Zeyu and Liu, Dingdong and Zhou, Shixu and Peng, Zhenhui and Ma, Xiaojuan}, title = {PlanHelper: Supporting Activity Plan Construction with Answer Posts in Community-based QA Platforms}, year = {2022}, issue_date = {November 2022}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, volume = {6}, number = {CSCW2}, url = {https://doi.org/10.1145/3555555}, doi = {10.1145/3555555}, journal = {Proc. ACM Hum.-Comput. Interact.}, month = nov, articleno = {454}, numpages = {26}, keywords = {information digest support, cqa platforms, activity plan construction} }
