The Collaborative Qualitative Analysis (CQA) process can be time-consuming and resource-intensive, requiring multiple discussions among team members to refine codes and ideas before reaching a consensus. To address these challenges, we introduce CollabCoder, a system leveraging Large Language Models (LLMs) to support three CQA stages: independent open coding, iterative discussions, and the development of a final codebook. In the independent open coding phase, CollabCoder provides AI-generated code suggestions on demand, and allows users to record coding decision-making information (e.g. keywords and certainty) as support for the process. During the discussion phase, CollabCoder helps to build mutual understanding and productive discussion by sharing coding decision-making information with the team. It also helps to quickly identify agreements and disagreements through quantitative metrics, in order to build a final consensus. During the code grouping phase, CollabCoder employs a top-down approach for primary code group recommendations, reducing the cognitive burden of generating the final codebook. An evaluation involving 16 users confirmed the usability and effectiveness of CollabCoder and offered empirical insights into the LLMs' roles in CQA.
翻译：Collaborative Qualitative Analysis（CQA）过程可能耗时且资源密集，需要团队成员进行多次讨论以细化代码和想法，以达成共识。为了解决这些挑战，我们引入了 CollabCoder，这是一种利用大型语言模型（LLMs）支持三个 CQA 阶段的系统：独立开放编码、迭代讨论和开发最终代码簿。在独立开放编码阶段，CollabCoder 提供按需的 AI 生成代码建议，并允许用户记录编码决策信息（例如关键字和确定性）以支持此过程。在讨论阶段，CollabCoder 通过与团队共享编码决策信息帮助构建相互理解和富有成效的讨论。它还通过定量指标快速确定同意和不同意意见，以建立最终共识。在代码分组阶段，CollabCoder 使用自上而下的方法进行主要代码组建议，从而减轻了生成最终代码簿的认知负担。一个涉及 16 名用户的评估证实了 CollabCoder 的可用性和有效性，并提供了 LLM 在 CQA 中的角色的实证洞察。
相关内容
微信扫码咨询专知VIP会员