Unearthing the intelligence hidden in free form data
Text Mining – What does it add to transaction data
Text mining refers to extraction and putting together of textual information into quantitative forms in order to derive information and garner insights from it.
There are many industry applications of text mining-
- Market research surveys use text mining to make sense of open ended questions in surveys.
- CRM data analysis uses text mining for adding value to customer churn modeling using customer feedback data with transaction data.
- The entertainment business uses text mining as ‘sentiment analysis;’ to gauge if new movie releases garner favorable or unfavorable word of mouth reviews.
- Publishers use text mining to get access to information in large databases via indexing and retrieval.
In retail, text mining or text analytics in conjunction with transaction data analytics helps retailers-
- Look deeper at real customer, product and service issues
- Enhance value from market research and may even help to cut costs of doing large scale market research studies
- Improve customer service by cutting lead times to address common issues
- Create better products
The process by which retailers can extract value from text data is:
1. Identifying where text data is collected
The three sources where text mining data is available and can be leveraged are:
- Surveys – These are usually customer satisfaction surveys that a retailer initiates with a customer. A lot of open ended information provided in these surveys contains valuable text information that should be mined for a deeper look at customer issues.
- Contact centre data – This data consists of e-mails, phone in transcripts and web chat or submissions by customers who are communicating an issue. Analysis of this data can yield a lot of very valuable information.
- Internet data – Data on the internet in blogs, product review sites and expert groups contains a wealth of information that is not gleaned by satisfaction surveys or customer feedback via phone.
2. Changing text data to structured form
The next step in the process is to change unstructured data into a more manageable form of structured data. This involves several small steps:
- Identification of the sources from where text data needs to be extracted
- Decision on which unstructured data to analyze i.e. product related, sentiment related, time period related, particular promotion related etc.
- Use of software that can extract the relevant information from various places
- Creation of theme or concept buckets to be able to take a closer look at extracted information and link it to transaction data
3. Analyzing text data
Once the unstructured data has been made manageable; reports can then be generated from it. These help the retailer focus on addressing key metrics as they come up and resolve the relevant issues. Thus keeping cleaned text data as a separate entity allows retailers to focus on data which would otherwise not be looked at.
4. Integrating text data with transaction data
A lot of actionable insights can be generated if text data that is cleaned up is then integrated to the larger transaction data warehouse. The linking of these two complementary data sets generates added value for retail organizations. It helps answer questions like:
- What is the reason for higher returns in a particular town/city/region?
- Why are customers calling in regarding a particular SKU?
- Which offer will a customer be most likely to accept?
- Why did a particular promotion not do well?
- What are the real reasons why customers have lapsed?
- Which competitor is doing better in terms of product and quality and price?
- Is a certain customer group adopting a new product more than others?
While most retailers have the text information they need to improve their knowledge of their customers, products and service, very few presently mine this information. Retailers thus need to unlock the value lying in unstructured data with a clear vision on how they will clean and integrate this data to larger quantitative data sets. They can then start to and use the insights generated from this data to improve customer experience through better service, products, quality and process.
Using text data to capture and add value to voice of customer
Mining the web to add semantics to retail data mining-Rayid Ghani
A method for generating plans for retail store improvements using text mining and conjoint analysis-T Kaneko in Proceedings of the 2007 conference on Human interface: Part II
Mining Text in a Retail Enterprise Assessing Customer Sentiment and Satisfaction by Sara Charen, Dan Ross
Text Analytics 2009-Users perspectives on solutions and providers-Seth grimes Alta Plana
Calling for Customer Experience Insight Social media may be hot, but don’t leave contact centers out in the cold. By Sid Banerjee Posted Mar 22, 2010 CRM.com