During my time as Director of Data Science for one of the leading innovations and marketing consultancies in Germany, I was the responsible team lead for Data Science & Analytics projects. Although I’ve completed numerous innovations projects with significant business impact, I had a steep learning curve to get there. I want to write about a specific project, which led me to the breakthrough developing a framework that was further on beneficial for the outcome of a lot of other projects.
The business problem our customer was facing was that the market for advertising has become more complex and expensive and forced the company to steadily increase their media investment. As the cost for advertising has been increasing over recent years and the customer has been forced to invest more in advertising every year, the return in media investment (ROI) has become more important. The customer was faced with the problem of how to improve media ROI through better decision-making.
My team and I invented a near to “real-time” approach to track and predict media ROI, comparing media actions of the company’s most successful brands across several product categories and numerous media channels. Furthermore, my team developed a change management approach, which led to a sustainable integration of the solution in the stakeholders’ working processes and a strong business impact.
The Problem in Detail
Historically, econometric Marketing Mix Models (MMM) have been used to evaluate media ROI to support investment decisions allocating budget across media channels by estimating the impact of media spend on sales volume. While those models provide insights on an average ROI per media channel over a time period of 24 months and support a long-term budget allocation strategy, they are barely adequate to provide information on an operational level. Additionally, the results usually come with a delay of up to 6 months on an aggregated channel level.
For the customer this was not helpful for several reasons: a) some products are only sold for one season and feedback from those product campaigns have limited value for new products in the next season, b) the delayed results of the models do not allow to correct a media plan to improve the ROI of campaigns of the current season and c) the process does not allow for running experiments to evaluate different actions during campaigns and to constantly improve ROI over time.
Additionally, monetary incentives for media planners are usually not connected to media ROI. They are mostly based on media coverage for a specific target group. Furthermore, previous campaign reporting had been provided by media agencies with interpretation of the results and action recommendations, but lacked a unified and data-driven approach. Our concern was that a new analytical approach could be rejected, as it would force stakeholders to change how they operate and would require connecting analytics to action.
How my team built a sustainable Business Solution
In order to tackle the complexity of the project, my team created a project plan, which included an analytical/data science and a business/change management project stream.
The analytical/data science side of the project included (1) stakeholder interviews, (2) creation of hypotheses, (3) analytical concept, (4) data audit, (5) hypotheses verification & analytics (performance driver verification and ROI modeling), (6) pilot the solution, (7) the definition of ETL processes and the deployment of the final solution.
We created a stakeholder interview concept (1) and the consultants on my team collected information about the stakeholders’ current working processes as well as hypotheses on relevant performance measures (2), which are currently or could be used to evaluate the performance of media campaigns. In order to create a solution that supports actionable decision-making, my team collected actions, such as channel selection, budget and target group definition, and content, creative or influencers used in campaigns. Those decisions were made on a daily, weekly, monthly and quarterly basis. Therefore, we decided that in order for the customer to optimize actions influencing media ROI it is necessary to provide performance insights on the most actionable level: per media campaign and on a daily basis.
Based on the collected information, my team developed an analytical concept (3) to evaluate ROI for all media channels (offline & online) on a daily campaign level. The concept included variables that a) would explain the performance of campaigns and b) could be measured on a daily basis. Those campaign performance indicator variables were closely aligned with the performance KPIs collected during the interview phase. Additionally, extended literature research on campaign and media efficiency led us to either confirm those performance indicators or to add/create further variables that could be used as such. The data audit (4) confirmed if all the necessary data was available on a daily/weekly basis and if it can be collected through APIs, web-scraping or required manual input. My team developed an analytical approach (5) to identify performance indicators that bridge the gap between media spending in online and offline channels and sales volume. Afterwards, different types of models were used to predict daily ROI estimates for campaigns for a brand/product category/media channel breakdown. We created a pilot solution (6), which included daily ROI calculations and campaign visualizations for one of the product categories and a subset of the media channels to showcase the results. The solution took into account the current working processes of the different stakeholders and the ability to compare campaigns and gain insights on actions taken. After getting the buy-in from the stakeholders, two data scientists/engineers and I wrote the code to build a robust ETL pipeline for all relevant data, provided daily calculations of ROI values for all brands and product categories (7), deployed it on a cloud platform, and created a business dashboard solution with the customer’s corporate design elements.
From a business/change management perspective, we had the experience that even if we had a great solution that could theoretically lead to a tremendous business impact, the most important component to get there is the acceptance and implementation from business operations. We conducted a literature research on the topic and brainstormed to collect motivational drivers and actions that could facilitate the acceptance of the new analytics approach at the stakeholders’ workplace.
My team focused on supporting intrinsic motivation and created stakeholder ownership for the future solution, reinforced project involvement through partaking in decision-making at crucial project steps, and created a project community with the help of customer interviews and frequent stakeholder check-ins. In order to fulfill performance expectations, it was necessary to create trust via transparency and knowledge exchange. Therefore, my team showed how our results were aligned with or different than previous MMM results and how the new approach provides all relevant features to answer the business question. To create relevance and passion for the solution, it was necessary to explain how the results are connected to the stakeholder targets, provide insights on potential opportunities, and focus on actions taken by stakeholders that led to successful campaigns.
During the implementation phase of the project, my consulting team provided weekly consulting sessions with the different stakeholder groups, translating relevant insights from the dashboard to a format the stakeholders are familiar with (e.g. PowerPoint), and helped them to transition to a new working approach by using the insights from the new dashboard solution.
Results
With the new approach it was possible to compare, track, and influence actions for running campaigns, and to provide an experimental environment to optimize future campaigns. The stakeholders were able to take actions such as shifting budgets between media channels, selecting the best content for a specific media channel or selecting weekdays with the highest ROI to run a social media campaign. The approach allowed a unique scaling potential across brands, product categories and media channels and reduced the need for manual campaign reporting. Most results were in alignment with the stakeholders’ assumptions but some negated existing beliefs. For example, the idea that the highest level of campaign personalization would create the highest ROI is not always true. Over time, the stakeholders changed their working pattern and implemented a learning process and frequent optimization loops in their day-to-day job. The first minimal viable solution was built within only five months within clear budget constraints.
Conclusion
Our new approach extended and overcame limitations of traditional MMMs and provided new highly actionable insights that led to a strong business impact. Implementing a change management approach overcame the problem that the analytics solution would not be sustainably integrated in the stakeholders’ business processes. Reflecting back, I learned that a structured framework with the two project streams (analytical/data science and a business/ change management) is crucial. Given the success of the implementation of the project, I now always try to involve my clients as much as possible in all phases of my projects.