Data analytics and "Big Data" have given people high hopes for completely transforming the field of marketing.
Most companies collect vast amounts of data from many sources, including financial data, behavioral data, transactional data, customer research data, mobile data, social media data, and more. Fueled by new analytical technologies, exponentially growing computing power, and the instant availability of online resources, a host of powerful tools have emerged, bringing about permanent change.
The power of these analytical tools and technologies has greatly expanded the scope of analytics applications, not only helping to unlock new opportunities and strategies, but also creating possibilities that were once unimaginable.
However, the idea of analyzing "Big Data" has been made to sound far too simple. In reality, working with big data is still a messy, extremely labor-intensive undertaking.
To borrow the words of two people who actually do this work: the overhyped claims about big data have caused us trouble, because they have created a false expectation — as if big data were somehow easy.
It is therefore time to take a more realistic view of big data, and to put an end to the myths we so often hear.
Big Data Myth 1: Big Data Is "Big"
Big data is not "big" at all — it is diverse. The word "big" is quite misleading.
What we are actually talking about is a massive number of "data points" (an identifiable element within a dataset, broadly referring to a piece of information or a fact) from a wide variety of sources, updating in real time at high frequency. Its nature is highly granular — it might be an individual transaction, such as a specific credit card being used at a particular gas station to pay for a specific amount of fuel.
In other words, big data is actually an enormous, massive volume of very tiny pieces of data. It is not a torrent of data crashing down like a mudslide — it is a sandstorm. In the desert, countless grains of sand whipped up by gusts of wind can blind you and leave you disoriented.
With that in mind, what other myths do we need to dispel in order to see clearly through the sandstorm?
Big Data Myth 2: You Must Adopt It Immediately
Most things in life that are important and valuable are difficult, and big data analysis is no exception. The only way to tackle this challenge is to take it one small step at a time, starting with very clear and specific objectives.
Before you start hoarding data, think carefully about what you actually want to do with it.
Big Data Myth 3: The More Granular the Data, the Better
Is more real-time, more granular data always better? Not at all. Watching only the first quarter of a soccer match does not allow you to predict the outcome of the entire game. Information obtained in real time may simply be too early to act upon. Sometimes you need to extend your time horizon and pull the camera back further to see the full picture and grasp the complete story.
Big data is filled with a great deal of white noise. As resolution increases — say, data by the minute rather than by the week, or at the town level rather than the state level — the proportion of noise relative to the overall signal increases.
Do not confuse precision with accuracy; the distinction between the two needs to be clearly understood. Big data in its raw, scattered form can be misleading — it needs to be aggregated to an appropriate level to filter out the noise.
Big Data Myth 4: Big Data Is Good Data
There is a difference between "a large amount of data" and "a large amount of good data."
Poor-quality data contains many errors and many missing values, and is prone to causing misinterpretation. Photos and videos may be mislabeled, and do the rambling messages written by teenagers truly reflect positive or negative sentiment? Sometimes we need a sophisticated model to sort all of this out.
In order to correctly interpret the meaning of data, you must discard some of it. One of the first priorities in properly analyzing big data is to clarify what data you intend to include in your analysis and what data you must leave out.
Big Data Myth 5: Big Data Means "Analysts" Will Become Exceptionally Important
It is often said that big data will give rise to the profession of analyst, who will become "the darlings of the information age." However, this claim about the impending rise of "analytics teams" is greatly overstated, because the dramatically accelerating pace of data generation means that people simply do not have time to "brief the analytics team" anymore.
What we need are faster tools capable of keeping up with the speed, volume, and granularity of incoming data.
Ideally, a small group of specialized analysts, skilled at harnessing the power of technology, will enable businesses and marketers to perform more data analysis, scenario modeling, and decision support on their own. We anticipate that dedicated Analytics Departments will disappear, and self-service will become increasingly widespread. The era in which Data Scientists hold a dominant position will not last forever — because there is simply too much data!
Big Data Myth 6: Big Data Can Give You Definitive Answers
Ambiguity is a defining characteristic of big data. Multiple data sources — such as transactions, lead generation, and media — may pull you away from where the evidence actually points. Different data sets, if not analyzed correctly, can indeed produce conflicting evidence. Which data do you choose to trust?
Big data requires human judgment to mediate and resolve seemingly conflicting evidence — and that is precisely where skilled analysts can demonstrate their value.
The more data you have, the more likely you are to encounter contradictions and ambiguities that need to be resolved. Big data is not all-powerful or omnipotent — in fact, quite the opposite. More data can give you more evidence, but it will not bring you closer to the truth unless you can skillfully apply experienced human judgment to reconcile the conflicting evidence.
The future of data analytics lies in combining, weighing, and judging multiple sources of information and different types of analysis.
Big Data Myth 7: Big Data Is a "Magic 8-Ball"
(Editor's note: The "Magic 8-Ball" refers to the Magic 8-Ball toy, shaped like the 8-ball on a billiard table. Users ask a question, shake the ball, and an answer floats to the surface. It is used for fortune-telling and seeking advice.)
Well, that is not entirely wrong — but you have to ask questions in a very precise way. It is a bit like being granted 3 wishes by a genie: you need to choose your words very carefully.
When applying data analytics, a lack of precision or failure to form well-developed hypotheses in advance can actually lead you astray and yield an incorrect answer. When you hold "big data" as your crystal ball, you must ask your questions with great care.
Big Data Myth 8: Big Data Can Build Self-Learning Algorithms
From a marketing perspective, erroneous inferences drawn from "rogue data" (unstructured, unmanaged data) highlight the limitations of automated models — for example, using direct response TV ads to predict call center volume will inevitably produce inaccuracies.
By the same token, "rogue data" gathered during a Super Bowl weekend could easily distort a self-updating algorithm.
Algorithms, when configured correctly, can indeed be extremely powerful — but human intervention is always required. Mobile carriers have already set a strong example of using non-marketing data for marketing purposes: they know who your friends are; they can estimate your age; they know which parts of town you frequent; they know which websites you have visited; and they know which apps you are using and where you are using them.
Insurance companies can also use telemetrics to collect data for marketing purposes, rather than solely for underwriting.
Fundamentally, the intention behind debunking these myths is to dispel the blind belief that simply possessing data guarantees business success.
The truth is that big data is essentially a tool — no different from a computer or a smartphone. It is a remarkable, game-changing tool, but only in the hands of someone who knows the right commands and coordinates can it truly fulfill that potential.