When integrating Statsig with Vercel's Edge Config, it is important to understand the behavior of user allocation during updates to experiment configurations.
According to our expert, increasing the allocation percentage in an experiment will not affect users who have already been assigned to the test. However, it is the decrease in allocation that could potentially remove users from the test upon their revisit.
For users who were not allocated during their first visit at a certain percentage, they can indeed be allocated if they revisit after the exposure allocation has increased. This is applicable when using a user ID as the identifier without considering sessions. It is also noted that the Statsig Client SDKs offer a keepDeviceValue
argument with getExperiment
that enforces stickiness via localStorage, although this does not apply to Server SDKs.
During an update delay, there might be a temporary mismatch in user allocation results when polling from the edge versus polling from the client SDK. The edge configuration might not reflect the new allocation immediately, leading to a period where the user could be allocated on the client side but not yet on the edge.
This mismatch will persist until the edge data is updated to match the new allocation configuration.