Sometimes activities don’t go as expected. Here are some potential challenges that you might face while using Automated Personalization (AP), and some suggested solutions.
There are several activity setup changes that can decrease the expected time to build models, including the number of experiences in your Automated Personalization activity, the traffic to your site, and your selected success metric.
Solution: Review your activity setup and see if there are any changes you are willing to make to improve the speed at which models build.
There are several factors required for an Automated Personalization activity to generate lift:
Solution: The best course of action is to first make sure the content and locations that make up the activity experiences truly make a difference to the overall response rates using a simple, non-personalized A/B test. Be sure to compute the sample sizes ahead of time. Computing sample sizes ahead of time helps to ensure that there is enough power to see a reasonable lift. You can then run the A/B test for a fixed duration without stopping it or making any changes. If an A/B test result shows statistically significant lift on one or more of the experiences, it is likely that a personalized activity is successful. Personalization can work even if there are no differences in the overall response rates of the experiences. Typically, the issue stems from the offers or locations not having a large enough impact on the optimization goal to be detected with statistical significance.
In Automated Personalization, the URL and template testing rules are added to the Target request entry constraint (for example, target-global-mbox), where they are evaluated only once. Once a user qualifies for an activity, the Target-request-level targeting rules are not reevaluated. However, the targeting audience is added to location targeting rules.
Solution: Add the necessary template rules as the input-audience of the activity. Audience evaluation happens upon each request/call.
This is expected.
In an Automated Personalization activity, once a conversion metric (whether optimization goal or post goal) is converted, the visitor is released from the experience, and the activity is restarted.
For example, there is an activity with a conversion metric (C1) and an additional metric (A1). A1 depends on C1. When a visitor enters the activity for the first time, and the criteria for converting A1 and C1 are not converted, metric A1 is not converted due to the success metric dependency. If the visitor converts C1 and then converts A1, A1 is still not converted because when C1 is converted, the visitor is released.