Managing to Expectations: A Primer

By Dipti Parmar

The best advice I received during my career in corporate America can be summed up in these four words; inspect what you expect. These four words that can provide focus for managing a business, a staff, a team, and even your children.

When it comes to business, the only metrics you should concern yourself with gathering are those that will help you make the right decisions. Most analytical software tends to emphasize metrics that might make you feel good about your business but do not really provide any useful guidance for making decisions.

For example, a report that reveals you have a total of 20,000 “hits” to your website may make you feel good, but the report tells you absolutely nothing about how you achieved those hits. In this sense, such statistics aren’t terribly useful.

You may have seen this in your business. You launch a new feature or product and a few days later sales and revenue are up. Everyone pats themselves on the back. The product guys think it is the result of the feature, the sales guy thinks it’s the new promotion and the customer service people think it’s the customer-friendly policies. The fact is, you don’t really know what caused the up-tick, but when sales and revenues drop back to baseline … no one wants to accept the blame!

Compare this to what I would describe as an actionable metric. For example, by adding a new feature to your website but allowing only every other customer to see it, you would be able to examine both sets of revenue streams a week later and make some meaningful conclusions. This metric is designed to allow you to ascertain the effectiveness of the new feature based on revenue differences. If the new feature increased sales, then you obviously want to implement that feature for all your customers. If you see that it didn’t move the needle for either group, you could scrap it. The important take-away here is that these types of metrics are actionable. It is data from which a conclusion can be readily made and acted upon.

How to Achieve Actionable Metrics:

Split tests—such as the one I described above, will allow you to take the right course of action on anything from minor copy tweaks to major product changes. These tests are widely known as A/B tests and you can get more information and background from this whitepaper titled “Controlled Experiments on the Web: Survey and Practical Guide” (PDF).

Per Customer Metrics—because people are metrics! Ordinary metrics can fog our focus on reality by diverting attention to unreal groups and pseudo concepts. It is significantly advantageous to examine data from a per customer or per segment perspective. Try focusing, for example, on the number of page views per new or repeat customer rather than just the total number of page views. Per customer data can indicate that you are increasing the level of engagement with your customer. Looking at aggregate data will not reveal this trend. There are several analytical packages that offer a business the ability to reduce aggregate data to per customer and/or per segment analyses. One is Google Analytics, which in combination with Google’s goal tracking feature will allow you to see which web referrers are driving the most conversions. Armed with this information, you can make decisions on which referrers are worth your time and money. This allows your business to maximize its return on investment.

Group analysis and funnel metrics—can be among the most useful metrics for forward decision making. For purposes of illustration, let’s say you have an e-commerce product with a few life-cycle events. These may include registering for the product, signing up for a free trial, using the product and, ultimately, buying the product. A simple report can be created to show these metrics for groups in a defined time period. For example, you might create a weekly report which shows what percentage of customers registering in that week went on to take each life-cycle step. If these numbers reflect no changes from group to group, then we have learned that nothing significant is happening. If one spikes up or tumbles down, then we have an unmistakable reason to investigate. Using funnel metrics to consolidate this data into a few useful numbers is easy to do manually, even if you have a large number of registrants. Simply break out the old fashioned index cards and record the number of customers registering each day. Then for each conversion (sale), make a tally mark on the index card corresponding to the date that customer registered (not the date they bought). Then on a weekly or monthly basis, you can compute conversion rates for the customers registering in that time period. Obviously, it is this number you want to focus on driving up!

What I have shared here today has been focused on the e-commerce business but the theme of managing to expectations is equally applicable to brick and mortar businesses. The idea of inspecting what you expect is applicable to all business enterprises, from invoice financing companies like CBAC Funding to the mom and pop dry cleaning store in your neighborhood.

If you expect to achieve a goal, measuring your progress is essential; otherwise, how will you know you reached it?