MiaProva Blog

Optimization Management is so much more than managing the intake and output of tests. A true Optimization Management Platform enables the organization to monitor tests or groups of tests and answers the big questions about their testing program as a whole and their investments in resources and technologies.

MiaProva automatically stitches the organizational meta-data related to Activities to data from Adobe Target and Adobe Analytics. This approach allows organizations to use this data to analyze and manage their tests and what offline variables are impacting the program – similar to how organizations use eVars and list variables in Adobe Analytics.

One of the many ways organizations use this is to understand how Adobe Target Premium is adding value to their Optimization Programs. MiaProva automatically classifies Activities that are specific to Adobe Target Premium as a dimension for Program Analysis. Doing so exposes a potential problem that needs addressing if limited or no value is being realized or creates an opportunity to expand on any value that is being realized.

MiaProva just onboarded two organizations that have made significant investments with Adobe and are heavy users of Adobe Target. Both organizations have launched more than 100 Activities launched since January of 2021. While mapping test tickets or test design documents to Adobe Activities to put all that data to life, MiaProva was able to immediately highlight for both of these organizations that they were not receiving the amount of value they should be with Adobe Target Premium.

In this blog post, we are going to highlight how we were able to identify this opportunity (and others) for these two organizations, what is being done to change things, and a great example of another MiaProva client who sees considerable value in Premium and MiaProva has used the ROI models to expand on their use of the Automated Personalization.

Optimization Program Macro-Analysis

MiaProva automatically creates dimensions related to the Adobe data such as “Adobe Target Premium vs. Standard,” A4T vs. non-A4T, Activity Types, Activity Sources, etc… Organizations use this data to manage live Activities and for all historical tests. While this is powerful (and very helpful for these two new MiaProva clients), the strategic value lies in tying the organization’s internal data to their optimizations efforts.

With Adobe Target, organizations have the tools they need to analyze individual tests. Organizations can go into Adobe Target and understand how the test impacts any metrics configured in Adobe Target. Lift, Confidence, segments, Success Metrics, etc…..all the great things you need to analyze an individual test.

And with, Analytics for Target (A4T), the analysis capabilities are exponentially more powerful by automating the inclusion of any metric, full use of organizationally accepted segments, metrics, calculated metrics, etc… Organizations can also look at all Activities in aggregate and use Activities and Activity Experiences as Segments which is a massive capability in and of itself that can’t be understated.

What is missing, though, is all the great data that is driving the Activity or test, to begin with. That is where MiaProva comes in by automating the process of mapping data from test tickets or test design documents to the rich data collected and managed by Adobe. MiaProva does this by way of MiaProva Programs – customize templates for their test design documents.

MiaProva Programs allow organizations to ‘dimensionalize’ any internal data related to tests just like MiaProva is ‘dimensionalizing’ the Adobe data like how we are distinguishing Activities that are part of ‘Adobe Target Premium.’ The use cases are limitless, but here are some that are being used by our clients today:

• What idea source is leading to the greatest number of tests? the greatest lift against metric x, y, or z?
• How are the tests distributed across organizational locations?
• What audiences are being used across test locations?
• How does our feature flaging solution (that we capture via an Adobe Analytics eVar) compare in terms of impact to revenue compared to tests that were executed via Adobe Target?
• What value is Adobe Target Premium giving us? What is our win rate with Premium Activities vs. Standard Activites?
• What is our ratio of Premium Activites vs. Standard Activities? What business units are seeing value from Premium and which ones are not using Premium?
• What Product Managers are runing the most tests?
• What is the distribution of Activities to Analysts?
• Are HIPPO driven tests as effective as data-driven tests?
• What is the ratio of Adobe Target ‘personalization’ efforts (pushing the winner or just targeting content) to actual tests

and we can go on and on. Any data that an organization would like to use is available for management for any Live Activities and all historical ones. This data can come by way of our APIs, JIRA integration, or as users supply tickets as part of their test management process.

Our approach to macro-analysis of optimization programs has enabled organizations to learn fascinating things and exponentially increase the realized value of investments in optimization. It is this approach that these two new organizations to MiaProva started to ask, “Why are we paying for Adobe Target Premium”?

Automated Personalization Playbook

Both of these new MiaProva organizations reached out to us to better understand what they saw in terms of our Program Analysis regarding their investments in Adobe Target Premium. It turns out the value they were realizing was non-existent or low was because they were not using the Activity features that come with Adobe Target Premium (Automated Personalization, Recommendations, and Auto-Target) correctly or in a minimal capacity despite licensing them for a considerable period of time.

Diving deeper into their use cases and their understanding of the Premium Activity types, it became clear that their understanding of Automated Personalization, how it worked, and how best to apply it was incorrect or less than ideal. We dove into the other Premium Activity types, but the investments made for Adobe Target Premium were tied to Automated Personalization.

We tried to point them towards documentation that could help, including a co-authored book on Adobe Target. Still, there was nothing out there that provided a soup-to-nuts approach to how best to use Automated Personalization. So, for these two organizations, and anyone else that may want to learn about Automated Personalization, we are committing ourselves to author what will be referred to as the Automated Personalization Playbook.

Give us a follow below to follow this series over the next month or two automatically:

This Automated Personalization Playbook will go very deep into what Automated Personalization is, how to give you the best chance to ensure success, and outline the best-in-class:

• Framework
• Execution
• Strategy
• Analysis

Stay tuned, as this will be a pretty powerful tool for those organizations that license Adobe Target Premium. It will also be exciting to possibly report on the change in ROI of these two organizations if they can adapt the strategy as outlined in this playbook!

Automated Personalization for the win

Another organization that leverages MiaProva is seeing significant ROI with investments with Adobe Target Premium. This company had applied a macro-view of optimization way before adopting MiaProva, which was helpful, but now MiaProva automates much of this effort.

This organization invested more than $3MM on an internally built engine that sits on top of its content management system. This system enabled business teams to establish rules for sections of their home page – very similar to how Adobe’s Experience Manager works. The team made heavy use of this solution, leading to significant increases in engagement and impact on key organizational metrics.

In an ongoing effort to continue optimizing, they tested this solution against Automated Personalization (using strategies outlined in the above-mentioned playbook). This effort lasted more than six months and spanned dozens of iterations. Without going into the details, here were the high-level results:

This effort clearly showed that humans are clearly smarter when it comes to applying rules to digital experiences based on data. This effort also showed the value of removing the cognitive bias that humans bring to decision-making and the value that AI could bring.

This ROI is managed by MiaProva and clearly visible in the MiaProva dashboard as a target that they continue to try to hit. This visibility has enabled this organization to demonstrate the value of automation, which has led to its operationalization to several other key areas for its digital consumers.