Online controlled experimentation is widely adopted for evaluating new features in the rapid development cycle for web products and mobile applications. Measurement of the overall experiment sample is a common practice to quantify the overall treatment effect. In order to understand why the treatment effect occurs in a certain way, segmentation becomes a valuable approach to a finer analysis of experiment results. This paper introduces a framework for creating and utilizing user behavioral segments in online experimentation. By using the data of user engagement with individual product components as input, this method defines segments that are closely related to the features being evaluated in the product development cycle. With a real-world example, we demonstrate that the analysis with such behavioral segments offered deep, actionable insights that successfully informed product decision-making.
翻译:为评价网络产品和移动应用快速开发周期的新特征,广泛采用在线控制实验。测量总体实验样本是量化总体治疗效果的常见做法。为了了解为什么治疗效果会以某种方式发生。为了了解为什么治疗效果会以某种方式发生。为了了解为什么治疗效果会以某种方式发生,分解会成为更精细的实验结果分析的一种宝贵方法。本文介绍了在网上实验中创建和利用用户行为部分的框架。通过使用用户参与单个产品组成部分的数据作为投入,这种方法界定了与产品开发周期中评估的特征密切相关的部分。我们以真实世界为例,表明对此类行为部分的分析提供了深入、可操作的洞察力,为产品决策提供了成功知情的信息。