This article is the end of the experimental trilogy, mainly for students who have some experimental experience or are interested in experiments. In the growth experiment, in addition to the "comparability" mentioned above, there are also analysis traps of bulk sms service one kind or another. This paper mainly discusses two representative issues: orthogonal stratification and its limitations, SRM problems and their coping methods. 01 How Orthogonal Layering Realizes "Parallel Universes" In the previous article on experimental design, we briefly mentioned orthogonal layering, which is used to use a limited number of users or traffic in multiple experiments at the same time without bulk sms service interfering with each other. Ideally, each layer in an orthogonal layered system would be like a "parallel universe" that could conduct independent experiments on its own.
In an orthogonal layered system, a user is likely to be hit by multiple experiments at the same time. In this case, how can the experiments not interfere with each other? The random grouping in the AB experiment uses a hash algorithm with better performance bulk sms service to perform special conversion processing on the user ID to ensure that the grouping is as random as possible. It can be understood that some complex processing is performed on each of our mobile phone numbers to avoid deviations between the samples of the tail number 8 user group and the tail number 4 user group when grouping directly according to the tail number. Random grouping occurs in each layer, bulk sms service and orthogonal layering refers to the need to ensure orthogonality between layers. There are ready-made inspection methods. Interested students can find them by themselves. I will not go into details here. .
With the help of a series of orthogonal hash algorithms (the orthogonal table algorithm is more commonly used at present), we can guarantee the experimental independence between any two layers. As shown in the figure above, assuming that we select the same bulk sms service user group, we arbitrarily select two layers in several orthogonal layers: the Nth layer and the N+1th layer, respectively. We decided to conduct the AB experiment on the Nth layer, that is, the users of this layer are randomly divided into two groups, A and B; at the same time, we will conduct the AB experiment on the N+1th layer, which is recorded as A1 and B1. The populations covered by the two groups of experiments bulk sms service are the same, and we issue different strategies. Orthogonal layering enables users of group A in layer N to be randomly distributed to groups A1 and B1 in layer N+1. When we analyze the experimental effect of the N+1 layer,