Skip to main content

The problem with Big Data


I am NOT a BIG Proponent of BIG data for business use especially in identifying consumer behavior. It has a lot flaws that are being neglected in the present big data narrative.

The interesting part that most of us neglect is "How is it useful?" It tells us what people are doing. Inferring why they are doing is up to the skill of the researcher and that is where most people fail.

To give a short example it was only a few economists including the now famous Nouriel Roubini who predicted the 2008 recession. While all economists had access to the same data, most of them failed. There were cases where the predictions were almost divergent.

Another important part of big data is "what is it that we are collecting?" When we have truck loads of data to collect, it is important to collect relevant information. What that relevant information is depends again on the researcher's skill. When you have a lot of dots, your inference depends on which dots you choose to connect.

Finally where big data fails miserably is that it only tells us what people disclose. In an age where most of us are curating an online identity (and concealing the real self) do you think you can figure out accurately what is going on in their mind. I don't think so!

Agree with this? Share your thoughts below.

Popular posts from this blog

Your Customer - Another line in a database

Email marketing can be the trickiest thing in the online marketing jigsaw. Most promotional emails end up as being marked as read or spam. No one cares! Well because most of the time, the ones sending it through a database also don't care.

Marketer's Job: Showing the money

A marketer's biggest dilemma is a client who misunderstands his service. " A writer who is in a hurry to be understood today or tomorrow runs the danger of being misunderstood the day after tomorrow ." ~ Johann G. Hamann Most of the times marketers face the challenge of being misunderstood. During the days when I was part of a startup, I felt that even if I'd break my head, the client will not really understand what my firm was offering and the benefits it gave.

Giving the finger to a scripted life!

It's crazy to take jobs just because they look good on your resume. That's like saving sex for your old age. ~ Warren Buffet. You have plans for your life, career etc. You've built an entire script but what is this based on? Scott at Ted talks about how almost 80% of the people around the world don't like the job they do. In his career, he found one day that most of the job he did was automated by his competitors. Building the resume was clearly not way forward. It was during that time he he heard the quote by warren buffet, " It's crazy to take jobs just because they look good on your resume. That's like saving sex for your old age ". This was when he left his job. While there are know universities, which talk about finding your passion, Scott talks about a lens that can help in finding the job we love.