CEO | 16 May 2014
The current interest in Big Data has been primarily fuelled by two key trends: massive volumes of publicly available data on the internet, and growth of technological solutions. Opinions differ widely as to what Big Data is, what it can do and how it can deliver real business value. However, there are many things that Big Data does poorly.
Anshul Jain is the Founder & CEO of ThoughtBuzz, the analytics arm of TO THE NEW, specialising in social analytics, business intelligence and data mining. Jain leads the product strategy of the company and has worked with clients across FMCG, BFSI, telecommunications, retail and media. A true geek, Jain is an expert in the field of social media analytics and helps organisations develop dashboards focused on identifying ROI. More recently, he has been heavily involved in the Social CRM domain. Prior to founding ThoughtBuzz, Jain was with Oracle.
In conversation with exchange4media,
Q. What Big Data can’t do – and how to fix it?
Big Data is all the rage these days. According to Nasscom, Big Data is expected to be a market worth $1 billion in India by 2015. The number of Big Data related job postings in September 2013 was 88,000, up 13 per cent from September 2012, as per a report from Venturesity, a Bangalore-based university. Political parties are not far behind and are using sophisticated data analysis technologies to identify voting patterns, swing booths and a host of other information.
The current interest in Big Data has been primarily fuelled by two key trends: massive volumes of publicly available data on the internet, and growth of technological solutions. Opinions differ widely as to what Big Data is, what it can do and how it can deliver real business value. However, there are many things that Big Data does poorly. Let’s talk about a few of them quickly.
Q. Big data can’t…
Firstly, data struggles with quality. Computer algorithms are great at math, but not at social cognition. In other words, algorithms struggle with quality. They can measure the quantity, but not the quality of interactions.
You’re also out of luck if you want to derive context. This is a point made by Nassim Taleb, the author of ‘Antifragile’. In his words, “As we acquire more data, we have the ability to find many, many more statistically significant correlations. Most of these correlations are spurious and deceive us when we’re trying to understand a situation. Falsity grows exponentially the more data we collect. The haystack gets bigger, but the needle we are looking for is still buried deep inside.”
Finally, Big Data favours the big over the small. Big Data analysis can predict when large numbers of people take a liking to a product, but it can’t predict that a product will become a hit even if it’s initially hated.
Q. But it can…
None of the above means, however, that we should ignore it completely. If harnessed correctly, the benefits to an organisation are immense. But how should an organisation move Big Data from the domain of data scientists to sales or marketing? I suggest the following:
1. Start concentrating on small data first. Big Data is all about processing, volume and machines. Small data is all about end users, context and individual requirements. Organisations should look at breaking down big data into smaller and more meaningful chunks. Doing this brings the focus on the end user and not on data analysis, leading to better results.
2. Marketers should play a far more active role in big data initiatives. They should demand applications and dashboards, which are tailored to their needs and then share insights with others in the organisation. When marketers drive initiatives, the results are completely different. Marketers demand tangible results that lead to actionable insight, something that data scientists aren’t always focused.
3. Start with a clear objective. Too often, organisations believe hiring data scientists and setting up machines on Amazon Cloud will give them great insights. Unfortunately, it isn’t that easy. Assess the core question, find out what kind of data is needed and then decide on what sources are required.
4. Don’t write-off intuition completely. In complex scenarios, it’s wise to have a balance of data and intuition, especially in cases where decisions influence other human interactions.
As put by R Madhavan, Chief Data Scientist at JP Morgan Chase, “We’ve seen a huge amount of interest across lines of business, but not a lot of discipline. Big Data is a big problem if you don’t start with a clear objective. Have a clear goal in mind and you will then see the massive benefits that data can bring to the table.”