Big Data technologies have changed the way we collect data, enabling us to handle infinite amounts of data.
In the old model, with relational databases, you first define the schema for what you collect and then put data into that schema, before analyzing it with BI tools. This is how data warehouse and data mining have been used during decades. You first needed to figure out the question, then you collected the data.
Big Data enables you to change that order. Now, you collect unstructured data first, and you ask the question later.
Modern analytics start by collecting everything, and then formulating your question.
This is what Alistair Croll explains in his book and in the slides and charts above.
The new paradigm allows you to search for the “unknown unknowns.” In analytics, most good answers will lead to another question. Data driven decisions depend on the ability to ask better questions and then ask again.
Implications for Business
In times of rapid market changes, what differentiates your business is how fast you experiment, how fast you can ask iterative questions and measure your progress, and how fast you readjust your business based on your learnings.
These are the principles behind the lean startup, and behind innovation. The most important metric for modern companies becomes “how fast does your organization learn?”
Big Data changes the cost of making data-driven decisions. It is an enabler for a more disciplined and empirical thinking about innovation and strategy. In that sense, Big Data analytics becomes one more enabler for disruption.
“I once debated Michael Porter at Davos on the overall value creation of the internet. I argued that the internet created valued and he argued that the internet destroyed value.”
Many examples come to mind, where Internet seem to be killing multi-billion industries:
– Skype is dropping the cost of voice to virtually zero. What before was a hefty business of international telephone calls, has been reduced by several orders of magnitude with VoIP.
– Instant Messaging killing SMS. We all know that SMS is terribly expensive in terms of price/byte. At a rate of 5 cents per SMS, considering 140 bytes per message, we are paying at $375 per Megabyte!! As you can imagine, with IM and unlimited data plans becoming popular, in the mid-term SMS will die.
– Music Industry. Used to charge $20 for a CD with 8-12 songs, the labels are struggling as they see the revenues from music downloads far too low to offset the reduction in revenues from CD sales. Music distribution in discs is likely to disappear in a few years, replaced by online distribution.
– Online Newspapers cannibalize sales from print editions, with online ads revenues not always compensating.
Are these examples really showing that Internet is destroying value? or is value simply shifting?
Looking back in history, it is not the first time there is a perception that technology progress destroys value. The Industrial Revolution saw workers protest as they lost their jobs to machines. Demand for unqualified workers doing repetitive activities dropped, forcing workers to get trained and qualified to build, operate and maintain machines. Those machines produced goods (and new machines) that were cheaper and in bigger volumes, deriving in more people benefiting from those goods. What initially seemed like job destruction, in fact it was wealth generation for the society by making goods affordable for more people.
History demonstrates that innovation simply removes inefficiencies, and in its way destroys industries if needed, shifting value to other parts of the economy, not necessarily within the same industry. And Internet is a fine example of that. Internet (and mobile) makes communications affordable to billions of people, and it is one of the engines behind Globalization, making knowledge and information available widespread.
So where is all this value (business) shifting to?
Let’s take the Music Industry. By removing inefficiencies (like physical distribution of information in discs or in print) the Digital Revolution is shifting the value from selling expensive disc copies to improving fan’s experience, by forcing artists to do more live concerts and spectacular shows as Madonna’s latest World Tour. Value is also shifting from the monopoly of recording labels to select the next star, to the most democratic system ever, with Internet empowering all sorts of musicians to publish their work. Labels and intermediaries are clearly losing business, but isn’t the Digital Revolution bringing more value to both fans and thousands of creators?
There are still many things to do to make this World a better place for our children and grandchildren. Let’s put the focus on improving it harnessing the power of innovation that Internet brings.
Internet clearly creates huge value for the society. A different question is, are Internet start-ups with unclear business cases (other than getting funds from VC) sustainable? We got an answer to that question when the late 90s bubble burst. Now we are getting the same answer after Bubble 2.0 is bursting too.
Any ambitious innovation must target to fix the problems of tomorrow considering the technology environment of tomorrow. Fail to consider the future environment and you might be pouring R&D dollars into the bin.
Innovations that did bet on tomorrow:
Remember when Gmail launched in 2004? It offered 1GB of storage when Hotmail or Yahoo only gave a few MBs. Google was anticipating the increasing volumes of email people would have to deal with in the future, while betting on the exponential reduction of storage costs. Gmail now give more than 7 GB of storage.
GPS navigators vendors should also be careful with Google Maps on new handsets such as Android or iPhone. With Unlimited Data Plans, why have a GPS Navigator when the GPS mobile phone can provide online up-to-date maps with extra real-time information? There are some advantages to run maps from the cloud rather than from a device memory, and Google will leverage on them.
Many anticipated that people would not carry a mobile phone and an mp3 player, when only a single device would do it. For Apple the iPhone was a matter of survival with iPod having the biggest market share by far, and Nokia adding Mp3 players to their phones. Not only did it created a great convergent phone-mp3 player, but also added the multi-touch and the mobile web revolutions.
Innovations that were shortsighted and are soon to die:
ATM (Asynchronous Transfer Mode) aimed to manage the bandwidth of data connections so that different QoS could be handled over the same network. With increasing amounts of bandwidth, driven among others by Gigabit Ethernet, the Internet can perfectly cope today with Voice (and even Video) communications, as Skype demonstrates, without any complex bandwidth management in the network. Why managing the bandwidth when there is plenty, and more is to come?
WAP was designed as a scaled down version of HTML to render web pages in a small screen. With faster microprocessors and larger memory powering-up mobile phones, WAP window is over before it ever delivered on its promise. MMS is a similar case. Why MMS, when you can send an email with a photo from new phones with web/email capabilities, using unlimited data plans. Mobile Internet is become more and more Mobile Internet the Internet way.
Fon and other Wifi Hotspots networks might suffer from the limitations of Wifi (100m-300m of range) as a technology to provide wide area coverage. With WiMAX, HSDPA, EVDO (and LTE coming), wireless broadband is crossing the chasm. Once Wireless broadband is available and affordable, how can a Wifi Hotspots Network spotty coverage compete?
Technology trends can easily be predictable, at least regarding capacity growth, be it Moore’s Law on computing elements or its equivalent on bandwidth growth. Before embarking on an adventure, make sure it is consistent with future trends and its corresponding problems.