Big Data technologies and Analytics promise to unleash an unprecedented amount and depth of insights from huge unstructured datasets of infinite data. But it is not all about technology. Big data need big brains, and here are 5 keys to structure your analytics designs and experiments:
1. Questions. Without the right question you won’t get the right answer. Knowing what to ask is the #1 key success factor for any analysis.
2. Listening. It is important to acquire and capture all data that might be relevant to your questions. You need to identify all sources and make use of technology to cope with real-time data in any amount that is required to reach your goal.
3. Structure the data, so that you can model it. The model should enable you to pose diverse questions to your dataset, so that you can test different hypothesis.
4. Categorize. Find categories, discover attributes, similarities or abstractions that put a higher layer of structure.
5. Find patterns that will hold true in new situations not yet encountered -> predictive analytics
Analytics is a circular process where the analyst must constantly seek for patterns to set hypothesis and the drive experiments to validate them.
Computer scientists are interested in finding the needle in the haystack.
Social scientist are interested in characterizing the haystack.
Both approaches have a big role in data science and analytics. Data scientist must understand which one of the two they are pursuing when they design the strategy and questions to reach the goal for your analysis.
With so much data to mine for, you need the right tools and methodology to find your gems. Still, big data needs big brains because “a fool with a tool is still a fool.”