Empirical technology and society research is in a methodological crisis. Problems increasingly involve closed platforms, targeted content, and context-specific behavior. Prevailing research methods, such as surveys, tasks, and web crawls, pose design and ecological validity limitations. Deploying studies in participant browsers and devices is a promising direction. These vantage points can observe individualized experiences and implement UI interventions in real settings. We survey scholarship that uses these methods, annotating 284 sampled papers. Our analysis demonstrates their potential, but also recurring implementation barriers and shortcomings. We then present Rally and sdkName, a platform and toolkit for browser-based research. These systems lower implementation barriers and advance the science of measuring online behavior. Finally, we evaluate Rally and sdkName against our design goals. We report results from a one-month pilot study on news engagement, analyzing 4,466,200 webpage visits from 1,817 participants. We also present observations from interviews with researchers using these systems.
翻译：经验技术和社会研究正处在一种方法危机中。问题越来越多地涉及封闭平台、目标内容和特定背景行为。调查、任务和网络爬行等常用研究方法提出了设计和生态有效性限制。在参与者浏览器和装置中开展研究是一个很有希望的方向。这些优势点可以观察个人经验,在真实环境中实施UI干预。我们调查使用这些方法的奖学金,注明284份抽样文件。我们的分析表明其潜力,但也反复出现执行障碍和缺陷。我们然后提出Rally和SdkName,一个浏览器研究的平台和工具包。这些系统降低了实施障碍,并推进了在线行为计量科学。最后,我们对照我们的设计目标评估Rally和dkName。我们报告了为期一个月的关于新闻参与的试点研究的结果,分析了1 817名参与者的4 466 200次网页访问。我们还介绍了与研究人员访谈使用这些系统的情况。
相关内容
微信扫码咨询专知VIP会员