A cry in the dark
Consider a person who has just walked into a Macy’s in a mall. So, why is she in the store and what is she looking for? Has she been to other stores or other Macy’s stores looking for the same item(s) she now wishes to purchase?
In the traditional world, Macy’s can never know any of the above and that precisely has been and always shall be, the Achilles’ heel of traditional marketing. It constitutes a 2-player game (Player 1: Buyer, Player 2: Seller), where a player 1 has a distinct advantage due to the incomplete information at Player 2’s disposal. The Buyer here is looking to maximize his Utility from the purchase and the Seller here is looking to make a sale and maximize his margins from it. In the traditional setup, the Buyer generally knows what products are being offered, their price, the potential cost of those products to the Seller and similar stats even on the seller’s competitors. The Seller on the other hand has little to no information on what the Buyer has on her mind about her tastes and preferences, purchase behavior, prior purchase attempts, the urgency of her need, or the trigger for her purchase decision. The best guess the Seller can make on the Buyer is her purchasing ability and her intent to buy. This is what we would call an incomplete information set and on top of that an asymmetric one (given that the Buyer knows more). Any system that improves on this information set for the Seller, improves on his ability to maximize his objective function.
“I know what you did last summer…and even 5 minutes ago”
Now if you take our current e-commerce environment – chances are all the activities of the buyer are recorded in what we call cookies. This includes how many times she has viewed the product, in how many sites, for what length of time, how many times she has shown the intent to purchase by adding it to her shopping cart, what related products she has viewed or purchased, and how related searches she had conducted. It is these cookies that hold the key to unlocking the utility function of the consumer, by revealing her tastes and preferences, purchase behavior, and the whole nine yards. The question is – How do you wield that key? There is terabytes of data to trudge upon before you get to something meaningful and actionable. Every Buyer has her own length of history and search pattern for one single purchase. Multiply that by the few dozen purchases she makes in a year and a few million customers the single Seller is dealing with.
In my opinion, the most amazing gift endowed upon us marketers by digital media is this ability to deconstruct the drivers, needs, and aspirations of buyers down to atomic levels. Thanks to marketing organizations religiously farming SERP keywords, cookies, and site navigation data, we are in the lucrative position of unlocking the buyer’s utility function provided we are able to eliminate the noise in the signals by applying advanced quantitative methods on big data. Before we get into the ‘geek talk’ overdrive let us define the fundamental questions we are seeking to answer in order to arrive at individual buyer specific targeting strategies.
a) WHO IS THE BUYER?
It all begins with a search: Every search is nothing but an ‘expression’ of intent which offers the key to unraveling the buyers need of the hour.
b) WHAT HAS THE BUYER BEEN UP TO?
Understanding the long and short of the buyers recent web-trail creates an opportunity for the digital marketer to define customized strategies empowered by contextual and behavioral targeting.
c) WHAT DOES THE BUYER DO ONCE SHE COMES TO MY SITE?
Combining search intent and cookie trail with site navigation (i.e. Pathing) helps understand which acquisition journeys lead to conversion and which paths are, well, roads to nowhere.
Search (Organic+ Paid) traffic coming into an established e-commerce website often comprises hundreds of thousands of unique keywords. However these seemingly distinct searches can be assigned to a finite set of ‘intent groups’ through logical classification of ‘semantic’ and ‘thematic’ similarity… Consider these two keywords: “best credit card for small business” vs. “top small business credit cards”. Clearly, these two searches are semantically different, but evidently they express very similar, if not the same, intent on the part of the searchers. Essentially, the intent here can be categorized as falling in the segment COMPARATIVE. Needless to say, by applying various text mining techniques, we can capture the massive number of searches in distinct intent groups such as INFORMATIONAL (“what is…”, “How to…”), CALL TO ACTION (“apply for…”, “buy online…”), and so on.
The objective of the entire exercise is to reduce the dimensionality of the massive keywords data into actionable, logical, accurate intent groups. This, when achieved, enables the digital marketer to rank site visitors from search channels in terms of ‘purchase propensity’. For instance, a visitor coming into the site having searched “apply today or “instant approval” is way lower in the sales funnel (i.e. closer to conversion) than one who arrived searching “low interest cards”. You can therefore understand how robust search intent segmentation can create a definitive early advantage for the e-marketer as far as addressing ‘WHO IS THE BUYER?’ is concerned.
All this is great, but we also know that all the people in the ‘CALL TO ACTION’ group do not convert, and that few in the weaker intent segments actually do. This is often a function of how the visitors interact with the site (‘Pathing’). A smart e-commerce site can actually manipulate the visitor’s site navigation based on the knowledge of their search intent groups as well as cookie trails thus maximizing the likelihood of ‘site navigation’ culminating into ‘acquisition journeys.’
Once we are able to define concrete intent groups and recent purchase priorities or needs (i.e., search history) of the visitor, it enables customized page/content displays that keeps the visitor on the ‘conversion path’.
The analytical techniques here get way more complex than standard regression models. This is because one cannot make the oversimplified assumption of identifying the triggers of conversion based on site navigation on the day the conversion happens. Why so? Let’s try and illustrate with an example: A visitor comes to a luxury watch retailer site, landing on the Homepage and takes the following path:
Homepage–>Products–>Add to Cart–>Checkout
If our dependent variable was ‘conversion'(Y/N) and the independent variables (predictors) were visit/no visit flags to the website pages then the traditional logistic regression model would tend to imply that the pages ‘Products’, and ‘Add to Cart’ and ‘Checkout’ are the strongest influencers of conversion. But as logical minds we know these are ‘self-selected’ pages for people who convert in as much that they cannot select the product of choice without clicking on the ‘Products’ page, and cannot complete the purchase without going through the ritual of visiting the subsequent two pages. This constitutes a unique situation where ‘correlation’ does not imply ‘causality’.
So where did the model fail?
It failed because it ignored the buyer’s entire acquisition journey through the sales window. On the day of the purchase, the decision to buy has most likely already been made in the buyer’s mind. The ‘pathing’ on that day is a mere execution of a foregone decision.
The real journey of awareness-to-interest-to-decision, hidden away in the buyer’s prior visits to the site or related sites when she was mulling over the idea of whether or not to commit to the sale, holds the key to unlocking which pages/features on the site actually influenced her decision. These, mis amigos, are the real ‘foot soldiers’, the ‘movers & shakers’ that cradled the visitor to conversion.
Mathematically one therefore has to estimate a panel data based mixed-effects models, where the pathing of each visitor, whether converted or not, on each visit is accounted for. One needs to understand the critical importance of integrating search intent and pathing based insights into e-commerce strategies. The digital marketing world is a two-edged sword – while on the one hand it offers tremendous opportunity to decode the buyer’s utility function, it also creates a perilous situation where the substitution of the seller by one of its competitors is a mere click of the mouse with the buyer not having to move an inch, and having the opportunity to compare offerings across multiple competing sellers in real time. The marketing campaigns that often fail are those where the seller puts his ‘brand’ above the buyer’s ‘needs’. Digital marketing should not be afflicted by seller’s ego which makes him self-assured of the footprint of his brand because the buyer is simply interested in her own best interest. If by leveraging intelligent big data analytics you can weave yourself into her scheme of things whereby she resonates with your brand as “THIS IS WHAT I WANT!” you would convert a site visitor into a customer or otherwise your competitor surely will.