The state of Hype vs reality in Big Data
Undoubtedly, there is a lot of hype around Big Data. But overall, the substance outweighs the hype. Having said that, there is still a fair amount of confusion, for instance on how to get value from it, and how to create a Big Data strategy.
On the technology level, there is a lot of fragmentation, and a lot of very niche technologies, that may offer specific functionality, but do not excel in integration. Big Data technologies, such as Hadoop, have become accepted pretty fast.
My general advice is to go ahead, and invest in Big Data in your information infrastructure, but accept there is still some risk in technologies that are not mature enough.
Factors playing a critical role in the success of Big Data projects Consumer–Personal Analytics: Analytical mobile apps, wearable technology, personal virtual assistants & smart advisors to offer suggestions, make recommendations, improve how tasks are handled, etc.
Market–Sentiment Analysis: Who are the influencers in the market on social media, how are the brands being discussed, what business trends need to be followed that affect business performance?
Partners/Suppliers/Distributors: Through sensor-technology, creating an (even) more efficient supply and demand chain, for flow performance, energy consumption optimization and predictive asset maintenance. Partners/Suppliers/Distributors: More real-time, behaviour-based data collection and prescriptive analytics for benchmarking purposes.
Shareholder-Infonomics: Showing how the information base, sharing information with all stakeholders and its analytical activities can be economically valued, and contribute to shareholder value.
Selection criteria to evaluate Big Data outsourcing vendors
I would approach the question not from the infrastructure and technology point of view, but from a skills and capability point of view.
Doing Big Data in the cloud is a commonly accepted approach; you would not be doing something “weird”. Increasingly, it is also the option that vendors of software and services offer. The one reason to do it on-premises would be the sensitivity of the data being in the cloud.
Moving from cloud to outsourcing is primarily a question of skills. If you feel you don’t have both the technology skills and analytical skills, it makes sense to outsource.
Technology skills mostly in terms of data integration, analytical skills mostly in terms of data science and data discovery.
The biggest challenges towards successful deployment of Big Data projects
If you are making Big Data an integral part of the business strategy, it requires an overall information management strategy. You cannot buy Big Data off the shelf. It requires strategic thinking and executive buy-in.
The three pillars to start a Big Data program are:
Vision & Strategy: This should supplement your Information Management strategy and requires the definition of Vision and Values
Organization: This is not just another capability in your arsenal, it requires its own organization from Strategy to Operation Support.
Architecture: As described above the Data and Technology (Infrastructure) architecture needs to be carefully planned given the intrinsic elasticity of the information being managed.
Verticals to benefit the most from Big Data and Analytics
In essence, every vertical yet Media/Communication, Banking Services, Education and Healthcare seem to have the most amount of Big Data programs while retail, insurance, transportation, utilities and government are still testing the waters.
However, it is very interesting to see what use cases there are per industry. My general advise which is applicable beyond Big Data strategies is to compare yourself to your peers, and even better to compare yourself to other industries and see what ideas you can adapt to your value streams.
The next “Big Thing” in Big Data
Simple. Big Data is moving away as a theme of itself.
First, because volume and velocity are temporary issues. The moment we become more comfortable with the speed and the data size, it becomes information management as usual again.
I didn’t mention variety here, which will remain to be difficult for the foreseeable future. Second, it becomes part of everything else. Big Data techniques will be used in mobile, social, security, analytics, business applications/BPM and every other imaginable IT field.
The state of Hype vs reality in Big Data