I Love “not-so-big” Data

I’m fascinated by data. Perhaps you should be, too?

One of the things I love about working online is that you can so quickly do two things: understand – through data and conversation – what your customers want and do not want; and find out – through structured experimentation – what really matters to people.

Online, with good use of data, you can learn in days what would take months or years of patient analysis to learn offline. And you can quickly apply this learning to enrich people’s experience of your work, build ever-stronger products and services, and develop ever-better organizations as a result.

Because of this, in sector after sector over the last 20 years, “doing business” has come to mean “doing business digitally.” The insights and improvements that come from data analysis have become ever-more fundamental to the success of very many businesses, and this, in part, has led to the buzz around the idea of Big Data.

There are many different definitions of Big Data. The one I like most talks about it in the context of 3Vs: data that comes in very high Volumes, has a lot of Variety to it, and flows with very high Velocity.

Big Data focuses on things like mashups from the “Internet of things,” mass analysis of healthcare data, and interpretation of vast quantities of social media posts – all of these make sense of enormous amounts of fast-flowing, loosely structured information.

I have to admit, though, that, while this type of data is hugely important, it doesn’t particularly interest me. I’m fascinated by the smaller, more structured data sets that come from other people’s packaging of social media information, from analytics systems, and from the results of structured experiments.

The problem with even these, though, is that it’s easy to get overwhelmed by the sheer volume of information available.

This is why I’ve particularly enjoyed reading “Behind Every Good Decision: How Anyone Can Use Business Analytics to Turn Data into Profitable Insight,” by Piyanka Jain and Puneet Sharma. (You can listen to our Book Insight on this here. The definition of Big Data I gave above came from this book.)

Jain and Sharma do a fantastic job of explaining how any reasonably intelligent person can make sense of data by starting with the problems they want to solve, asking questions about them, developing hypotheses that can be tested robustly, and then gathering and analyzing information to create revenue-generating insights.

Pleasingly, they say that many of the techniques learned in statistics class are too convoluted to be practical in real-world situations, and that simple techniques are often the most effective. They say that 70 to 80 percent of decisions can be addressed with these by any professional who can use an Excel spreadsheet.

These simple techniques start with aggregate analysis, where you just describe, for instance, the people who buy your products. If you have a clear picture of who likes your business (for example, by knowing your customers’ genders, ages and locations), you can work out how to find more of them, and you can think about how you can serve them better.

Another approach is correlation, which looks at the relationship between factors. Perhaps, when you were conducting your aggregate analysis, you started building up a picture of what encouraged your visitors to become customers. You can structure your observations into hypotheses, and then test these hypotheses with appropriate analysis. This allows you to explore, for example, which marketing ideas and approaches are most effective, so that you can direct your budget towards the campaigns that are giving you the best return.

Jain and Sharma offer many more approaches, and then go into detail about how you can use them effectively to drive serious improvements in your business. With my interest in business tools, I was particularly fascinated by their BADIR™ process (BADIR stands for Business question, Analysis plan, Data collection, Insights and Recommendations) – perhaps we’ll look at BADIR in detail in a few months’ time. [Editor's note – see our article on BADIR here.]

Anyway, if you have even the slightest interest in data analysis as a topic, I strongly recommend “Behind Every Good Decision”. It beautifully describes a key element of the “growth hacking” approach that, I suspect, will become one of the most-important and best-rewarded skills in 21st Century business.