Hype Vs Reality in Big Data
“We ourselves sometimes speak about ‘Big Data solutions,’ but in reality what our Stratecast Big Data & Analytics practice is about are data management solutions, some of which are beginning to effectively manage Big Data. And, contrary to what is fast becoming conventional wisdom, some organizations are in fact deploying data management solutions addressing Big Data. However, especially noteworthy given the benefits that harnessing Big Data has to offer, the rate of adoption is low. One reason is a lack of clarity in the value proposition: too many providers and hopeful solution buyers are not making the case effectively for why those who sign the checks should invest. The benefits include the opportunity to create an agile, data-driven organization; leveraging data to support KPIs; and transforming fear of Big Data into employee empowerment, offering the insights people need, when they need them, to make smarter decisions. All of that translates directly into revenue generation and retention, and while vendors obviously hope to capitalize on this by selling more products, we see examples all the time where companies actually save money by deploying a Big Data solution. For example, a bricks-and-mortar mega retailer in Europe just markedly enhanced its competitive position with Big Datadriven real-time analytics. Deploying ‘the right stuff’ and not just ‘more stuff’ also enabled the retailer to slash its IT footprint, and as a result, it is now saving more than $1.4 million annually—while understanding its business better than ever before, and responding faster to the needs of its customers.
“Stratecast estimates the market for Big Data, analytics, and Business Intelligence (BI) solutions at $25 billion for 2013, increasing at a Compound Annual Growth Rate (CAGR) of 12.7 percent to $40 billion by 2017.”
"Deploying ‘the right stuff’ and not just ‘more stuff’ also enabled the retailer to slash its IT footprint, and as a result, it is now saving more than $1.4 million annually"
Developing Big Data Capabilities In-House Vs. Outsourcing
“While some organizations are certainly crafting their own Big Data systems, they are often better served by engaging a professional team to either provide a Big Data solution outright, or, at minimum, work with the organization on the initiative. This can easily cost the organization less than doing the job in-house. We urge buyers to consider the following direct, indirect, and opportunity costs in constructing a business case:
• “Failure to capture mission-critical data–with a poorly planned Big Data implementation, the business will not capture all relevant data to compete effectively in the
marketplace and retain existing customers. This can render the business vulnerable to competitors, leading to missed opportunities, lost revenue and higher churn. For the business case analysis, calculate the likely and potential top line and opportunity costs of continuing to operate in the absence of complete data.
• “Taking multiple ‘shots’ at Big Data can delay implementation–implementing a Big Data system is a massive undertaking that touches every area of the business;
and the downstream impact of any delay is magnified. From decades of experience optimizing IT infrastructures, and more recent experience implementing Big Data solutions, expert consultants understand where the vulnerabilities and risks are, and plan accordingly. With proper upfront planning comes lower exposure to risk, decreased chance of unexpected disruptions, on-time project completion, and an agile business that seizes new opportunities instead of missing market windows.
• “Resource drain on internal team–most IT and Data Science teams are under pressure to maintain daily operations and incorporate new business-enhancing technologies, without increasing staff. By hiring a team of experts, they can keep internal staff doing the things they need to do and avoid the costs of delaying or deferring essential tasks.
• “Implementing Big Data architecture confines costs to hardware–while merely adding servers will not manage Big Data, it is sometimes necessary to add servers when implementing a Big Data infrastructure. Even so, if the organization has worked with an experienced professional service team to incorporate the Big Data Architecture elements in Stratecast’s Data Management Model, the cost of the servers themselves should be the only incremental cost.
“At Stratecast, we have developed a 12-part data management model designed to transcend what one vendor or another may be best at, or industry buzzwords seemingly everyone wants to gravitate toward. We advise buyers to turn to providers with a good mix of broad-based data management capabilities mapping to our model—or partnerships in place to deliver pre-integrated solutions that cover bases the provider does not cover itself. A provider needs to have proven experience managing all types of data (structured, unstructured, and semi-structured), and both online and so-called offline (physical world) data. It also must have a proven approach for integrating Big Data into the existing IT infrastructure, and into the business.”
Making Big Data an Integral Part of Business Strategy
SMBs and below should have their heads in the clouds. Where ‘Big Data for the rich’ may leave midsized companies is in cloud-based Big Data solutions. These solutions can turn the embedded/legacy IT model on its head, and help companies do more with less. Reducing the IT footprint of legacy technology reduces both Capital Expenditures (CAPEX: upfront investment) and ongoing Operating Expenses (OPEX). Some providers who are providing more affordable solutions, some entirely cloud-based, some premises-based, and some a hybrid of technologies, include Birst, GoodData, Starcounter, GridGain, Platfora, and GigaSpaces. Do they have huge market share at present? Compared to the market leaders— No. Yet Birst has more than 1,000 customers including Citrix, YMCA, and CBS Interactive; Starcounter has 80 large customers including Hyundai, Telia, Canal+, European megaretailer Gekas Ullared, and Verizon Wireless; and the others
in this grouping also have robust customer bases.
Another challenge to the growth of this market is the ‘IT glass walls’ phenomenon: at present, it still requires too much specialized expertise, in the form of the data scientist or CIO and team, for business users to quickly get the data they need. The cloud-based data management providers and other innovators in this space are making headway by providing solutions that are not only affordable but also geared to enabling business users to get the insights they need without having to beg IT for information.
Seriously consider Hadoop—but get all the Hadoop you need. We hear too many companies in effect saying, ‘I’d like to order one Hadoop, please,’ as if that is going to solve the Big Data riddle for them. It is certainly not an exact analogy, but I liken Hadoop to my Firefox browser: I love it, but only with my favorite add-ons like Tab Mix Plus, FireShot, Google Translate, QuickTime, and Adobe Shockwave. Similarly with Hadoop, to get the most out of your data management solution you need modules that have been developed by other Apache Software Foundation working groups: modules such as Pig to accommodate semi-structured data and simplify various tasks. Hive to use Hadoop as an Enterprise Data Warehouse (EDW). Sqoop and Flume to import Web log data into a Hadoop Distributed File System (HDFS), and to integrate structured data into the mix. Ambarri, Whirr, Zookeeper, and Ozzie to accelerate deployment, simplify ongoing management, manage workflow, and maintain data synchronization.
Industry having Potential for Big Data
The verticals we see doing the best in terms of deploying Big Data solutions are retail, financial services, and telecommunications.
“I’ll focus on retail because that is where some of the most advanced work is taking place—and also where privacy concerns are on the front burner right now. This may be the most challenging time in history to be a retailer. Smaller retailers are under pressure by so-called Big Box stores, such as Target and Wal-Mart; and retailers of all sizes are feeling the crushing weight of Amazon, Zappos, and other e-tailers. Many retailers are deploying retail analytics solutions—operating via Wi-Fi, Bluetooth, and even closed-circuit video surveillance cameras—that collect data from shoppers’ smartphones at retail locations, and these solutions are reversing the inequity that has existed where shoppers, and e-tailer competitors, both have had access to more and better Big Data than the retailers. Yet these systems may ‘work too well’ in that they can quickly obtain Personally Identifiable Information (PII) from shoppers—and, depending on the system, this can occur with no action at all by shoppers other than to walk into a store. This has regulators and consumer protection groups up in arms, and if proposed legislation passes, it might limit or eliminate retailers’ ability to use the most intrusive systems.”
Hype Vs Reality in Big Data