Utilizing Association Rules to Improve Personalization of Offers to Customers

What Association Rules are, how to build them and most importantly, how to leverage them in your marketing strategy.

The concept of Association Rules has been around since the early 1990’s and has become a core tool utilized by data miners.  The process is based on the concept of discovering regularities between products in large scale transaction data recorded by systems in retail and product fulfillment. For example, the rule  “(Potatoes, Onions)--> Hamburger”  found in the sales data of a supermarket would indicate that if a customer buys onions and potatoes together, he or she is likely to also buy hamburger meat. Such information can be used as the basis for decisions about marketing activities such as, e.g., promotional pricing or product placements.   This type of analysis is useful in developing recommendations similar to what you see when suggesting “Customers who purchased product X also purchased product Y”.

The first step it to clearly define what you what to associate.  Association rules differ from Classification rules in that there are no preset outputs.  So Association rules allow more freedom for unique combinations to be discovered.  The most typical use is to associate baskets of product purchases, but this technique has also be used to help set pricing strategy or retail product placement analysis.  Sometime the Association rules can discover some “odd” relationships.  

A famous story about association rule mining is the "beer and diaper" story. A purported survey of behavior of supermarket shoppers discovered that customers (presumably young men) who buy diapers tend also to buy beer. This anecdote became popular as an example of how unexpected association rules might be found from everyday data. There are varying opinions as to how much of the story is true.  While I don’t find diapers around many beer sections in stores, I’m sure some enterprising retailer has tested this at some point.

So what tools are able to perform this type of analysis?  The usual power house applications from SAS and SPSS are of course quite capable of crunching through volumes of data to develop these rules.  Depending upon your budget and expertise either of these tools offer solutions.  If you want to keep your tool costs down you can consider a tool I recently discovered (see this link http://www.cs.waikato.ac.nz/ml/weka/)  You will need to do a bit of reading to get up to speed, but as this is freeware, I have found the time to get up to speed more than offset by this tool’s very effective performance.

Once you discover the Association rules in your database what do you do with them?

I have found these rules provide an excellent foundation to build a cross media promotional offer engine.  I have developed some great campaigns via email where a product that the customer had recently purchased is used to re-establish the relationship and then a couple of items that showed the highest support and confidence rankings via the association rule set were offered with minimal purchase incentives.  These campaigns were set-up with test and control groups and the groups receiving the communication based on the Association rules showed a 40% lift in response as compared to the control group.

You can also create tables of product associations to feed recommendations on your website.  These can be very powerful as compared to making recommendations based on less robust analytics.

This technique can also be useful in determining how to sequence product presentations in your catalogs and brochures.  Just as retailers utilize this analysis to help layout merchandise in stores to be more consistent with customer shopping patterns, this analysis helps do the same for printed materials.  Typically the improved sequencing of products in printed material improves the response by 5 to 10 percent.

All these actions result in improved sales, retention rates and ultimately profits. 

How much of your Website Business is incremental versus channel shift from your offline efforts: A simple analytical answer along with a couple of methods to measure the quality of web buyers.

Almost every multi-channel marketer is struggling with the question, “how much of my web business is incremental”?  This is a key question, for marketers who have observed that as the web channel grows, typically the other traditional offline channels fall.  So how can a company determine if the web sales are purely a channel shift, or might it be incremental?

My experience is to look at what portion of your web business is coming from customer who are new to your file.  Obviously, if your web business is dominated by customers who have been on your file prior to the web activity, then the web channel activity is a pure shifting of business.  But that is most likely what you won’t find.  There will be a proportion that is new to file and the key is to compare that portion of new with your other traditional offline channels.

The table below illustrates this concept well.

New to File Proportion of Business by Channel
		Existing Buyer Business	New to File Business	Proportion existing Business	Proportion New to File Business
Offline Channels		$300	$100	75%	25%
Web Channels (Ver 1)		$25	$10	71%	29%
Web Channels (Ver 2)		$25	$20	56%	44%
Web Channels (Ver 3)		$25	$100	20%	80%

If your proportions looks like those in Ver. 1, your web business is clearly a channel shift only and should not be viewed as incremental at all.

If your proportions looks like Ver. 2, your web business is partially incremental, but your offline channels are still your primary growth source. 

If your proportions look like Ver. 3, you should view your web business as incremental, and just like any marketer, you need to understand what do these new customers cost to acquire, and is that an appropriate investment.  This brings up the next topic, how to measure quality of the web buyers.

Two of my previous articles will help with this task.  First is to run the “Buyer Matrix” on your web buyers to understand at what level do they return to make subsequent purchases, and how do those compare to other similar aged customers.  These behavior metrics will quickly help you determine if the investment made to acquire these customers was appropriate or not.  It will also provide you a relative measure of quality between your customers acquired from offline efforts versus online.  You should also review my previous article on Acquisition Cost to utilize the appropriate financial perspective on the appropriate investment returns.

Another useful tool is the Generic Scoring model.  In a similar fashion, you should compare the scores of your new to file buyers across channels.  There are many myths that web buyers are of lower quality of the offline buyer, and thus do not warrant the same level of investment.  My experience is that this is not always the case.  Be sure you understand how your customers behave using their actual purchase data. 

Customer File Health Check: Use this simple analytical approach to gage the health of your business

The health of your customer file is the key to future growth and profits

A few weeks ago I wrote about utilizing the Generic RFM equation in finding the break-even point for more efficient promotion execution.  One suggestion I made in the article, for which I have received many comments and questions, has to do with “freezing” the score ranges for the deciles.  In doing this you have a view of how your customer file is changing (in terms of the percent change in each decile) across time.  This dynamic view can tell you a great deal about the health of your file.  In this article I will show some of the most common file dynamic patterns along with how to interpret what the changes mean for future sales and profit growth.  

Keep in mind that the transactions included in calculating the RFM score should include all channel activity.  If you are not including all channel activity when you calculate the score the observations made in this article will probably not be applicable.  As the score can be calculated back to any point in time this analytical technique is easy to employ.  Just be sure there is sufficient time between the freeze date and the end date (my experience has been at least one year) such that file migrations are occurring in sufficient quantity to be projectable to the whole file.

When you run the scoring for your customer file and get the decile counts you will have a report that looks something like the table below:

Decile	Freeze	Later	% Change
1	5,000	7,000	40.0%
2	5,000	6,500	30.0%
3	5,000	6,000	20.0%
4	5,000	5,750	15.0%
5	5,000	5,500	10.0%
6	5,000	5,500	10.0%
7	5,000	5,600	12.0%
8	5,000	5,700	14.0%
9	5,000	5,900	18.0%
10	5,000	6,500	30.0%
Total	50,000	59,950	19.9%

From this you can see that the file grew by about 20%.  Also the deciles at the top grew faster than the deciles in the middle, and the bottom decile grew at a rate lower from the top decile.  So what would this mean for your business?

First is the good news, you are building value in your customers as the top decile is growing faster than your bottom decile.  In other words, your best quality customers are growing faster than your poorest quality and typically this means your customers are purchasing more than defecting.   Some of the middle deciles are not growing as fast as the top or bottom deciles.  This can mean that customers are migrating quickly either to the top or bottom of the file.  This suggests that the life cycle of customer may be short and you should know quickly if new customers are connecting with your brand or not.  In this case you may want to focus in on a sample of new buyers who are sinking to the bottom and contrast with customer floating to the top.  What are the differences?  Does a product category or price point differentiate the two groups?  Can anything be done to prevent the customers from sinking to the bottom?

If you see that the top three deciles are larger and growing faster than the bottom three deciles then you can conclude that the file is fairly healthy.  If on the other hand you observe that the bottom three are larger and growing faster, then you have a problem.  You need to understand what your brand is missing or has the competition begun to offer some product or service that you might have missed?

Understanding how the customer is migrating through deciles tells a lot regarding how the customers are reacting to your brand and offers.  This type of analytics is critical when answering the questions regarding the health of your customer file.

Improve your Brand’s Perception and Position now; don’t wait for the “storm’ to pass.

Some simple steps to improve and enhance your brand using some straight forward Direct Marketing Metrics.

While speaking with many business owners over the past months I have heard too many stories of the business “hunkering down” and waiting for the storm to pass.  This is the best time, in fact, to spend a little time and resources to understand why your customers do business with you as opposed to your competitor and what could you do now to take more share.  Such activity is not expensive, yet it has potential to pay off handsomely if executed and followed up in a methodical manner.  So what am I talking about?

 What could you do now to ensure your business retains and expands its leadership position in the marketplace?   

My recommendation would be to first profile your customer base.   This requires answers to a few key questions including:

• ·         Why do your customers buy from you? 

• ·         What do customers think your brand stands for? 

• ·         Why did the customer purchase from you as opposed to one your competitors? 

• ·         How many of your customers intend to purchase from you in the future?

These profiling questions would best be answered at your customer segmentation level (i.e. Best Customers versus Marginal Customers).   To create such segmentation please refer to my article on the Generic RFM Scoring methodology.  

If you already have this kind of information from customer primary research studies you are well on the way to securing your position in the marketplace.  If not, I have the expertise to get this process moving for you if you wish.

Having years of experience analyzing customer transactions in marketing databases I would then want to analyze the following marketing metrics for the buyers profiled above:

1. How many customers purchased from you in the last 12 months?
2. How many times did they order and what was the average order size?
3. How many new customers did you acquire last year? 
4. How many of last year’s new customers are you expecting to buy this year?
5. How have these metrics changed in the past few years?
6. What marketing programs have the most impact on these metrics?

Answers to these metrics questions coupled with the customer profiling above would give your brand a clear road map of how to grow the customer base at rates much greater than your competitors.  While at the same time you would have improved information on key metrics driving your business.  The metrics are critical in setting appropriate benchmarks for ROI goals on your marketing programs.  In that way you will have a better and timelier understanding of what is working and what is not.  Then you can expand the programs that work and fix or stop programs that are not working.

Doing these things now is critical, especially if your competitors are “hunkering down” and waiting for the storm to pass.  For when the competitors raise their heads again you will have secured a more defensible position in the marketplace from which they will be required to play catch up, which almost always is a more difficult and costly exercise.

Customer Retention: How to improve it utilizing Customer Behavior Analytics

Understanding how often your customers repurchase items can yield large paybacks in terms of sales and profits 

Many companies are not aware they are sitting on a gold mine of sales and profits and all they have to do is look for it.  Over the years I have analyzed customer purchase behaviors and in many instances found opportunity to capture more sales, improve retention rate all while increasing profits.  Does this sound too good to be true?  The key to identifying purchase behavior that could be cultivated into one of the most profitable marketing programs is to look at item sales across time to identify if the same customer purchases the same item multiple times.  The easiest method of identifying such behavior is to list item by customer number who purchased.  If it is observed that over twenty percent of your items are repeat purchases from the same customer, you’ve struck gold.

Many companies have this behavior hidden in their transaction data and are not aware of its value.  I have found such behavior at both B2B and B2C companies.  The majority of this buying behavior can be associated with products that are “consumed” or regularly replaced.  Some good examples of consumable products include paper supplies, business forms, annual replacement items, nutritional supplements, filters, equipment that fails after use and numerous other examples.  If in doubt, the basic analysis is straight forward and should be done, it only takes a small percentage of your items to generate impressive ROI on the resulting program.

So let’s say you found your product portfolio contains consumable products, now what?  The next step is to find what the minimum time interval is between purchases.  For illustration purposes, you find that the average time between purchases is nine months.   The key is to contact the customer prior to the average replacement window and secure the reorder as opposed to allowing the customer an opportunity to explore alternative sources for the product.  

In this example, I will use some actual data from a B2B marketer I worked with who sold supplies which have a clear replacement cycle as the products ultimately fail with repeated use.  In this study there are 4,100 customers who purchased one (or more) of seven similar items used to deliver services to their customer.  These customers had purchased one of the items within a window of a minimum of 6 months and a maximum of 24 months ago.  For this test 2,950 will receive a special communication regarding the item replacement while another 1,150 are held out as a control group to measure the incremental lift.  

The communication’s content should be personalized to the recipient such that the purpose and action is clear.  For this test the communication contains a picture of the product purchased, date of purchase and a brief explanation that the product’s useful life may soon expire, and photos of products with various component failures are shown to help the customer evaluate the remaining life of the item.  The customer is urged to examine the product and if any signs of potential failure are found, a replacement should be ordered immediately.   The communication was sent via snail mail, but the technology has advanced quickly and this customized communication can be sent via the customers preferred channel of communication.  The control group continued to receive all the regular communication except for this replacement notification.

The performance of the test and control groups is documented below.

	Control - Total Qty 1,150			Test - Total Qty 2,950
Product	Data	Total		Data	Total
Category	Buyers	103		Buyers	330
Performance	Sales	$19,359		Sales	$54,460
	Sales per Member	$16.85		Sales per Member	$18.45
	Response	8.96%		Response	11.18%
				Sales Increase	9.5%
				Response Increase	24.7%
SKU	Data	Total		Data	Total
Performance	Buyers	48		Buyers	207
	Sales	$5,109		Sales	$20,185
	Sales per Member	$4.45		Sales per Member	$6.84
	Response	4.18%		Response	7.01%
				Sales Increase	53.8%
				Response Increase	67.9%
Overall	Data	Total		Data	Total
Performance	Buyers	243		Buyers	762
	Sales	$90,607		Sales	$338,106
	Sales per Member	$78.86		Sales per Member	$114.53
	Response	21.15%		Response	25.81%
				Sales Increase	45.2%
				Response Increase	22.1%

Not only did the effort result in elevated levels of customers reordering the items included in the communication, but additional items in the product category and products outside of the category were purchased too.  Overall retention was improved by 22% and with an increase of average order value sales increased by an average of 45%.  These are actual results achieved, not all programs will have such strong results, but I have always seen this program deliver acceptable results when implemented as outlined.  For some businesses that have a highly consumable product portfolio this type of program typically becomes the mainstay of top line sales and bottom line profits.