The big data projects are fizzling out, leaving behind projects that may be less spectacular, but more profitable.
The big data projects paradox
All the communication singing the praises of big data projects has done nothing to clear up the doubts that operations staff harbor in their daily activities. But it is a completely different story in practice: our contacts in those companies have told us just how difficult it is for them to reconcile such claims and the world-enlightening promises of data science with their own experiences inside the company and on the front lines.
“Our team of 10 data scientists has been working on data for two years. Millions have been plowed into a Hadoop cluster and heavy artillery. But everyone in the group wonders what they’ve been doing. They haven’t produced anything.”
This story (similar testimonials have been heard by many other companies) is in stark contrast to what has been publicly boasted in the media, from announcements of game-changing big data projects to certain organizations preaching how they led the way and just how modern and converted to data they are. Question marks are rightfully being raised about the trend towards big data, data science and analytics among business functions and IT professionals:
“OK, but bottom line, what will it bring to the company? What value will it actually create for the business functions and the company? What benefits come from exploiting data?”
McKinsey came to the same conclusion in an article published in July 2017 on the results of analytics and data projects in insurance companies. Projects delivering high impact were only reported by one in six respondents! Yet companies were investing as much as $80 million a year, and over half of the CEOs concerned considered data and analytics a top-five priority. 80% waste: that is currently the gap between fake news and the real world.
Why do big data projects fail?
Do big data projects really bring something to the company? Meaningful information or a new insight? A recommendation that nobody had thought of before? Support for the decision-making process? Data does not automatically generate value, leading big projects to fail. Companies often fail to ask these questions. The reasoning behind many projects runs in the opposite direction:
“Let’s build a huge system, and we’ll be damned if it doesn’t deliver something!”
The value of this “something” does not seem proportional to the hard work expended, as confirmed by this customer from the retail industry:
“Of course data scientists detect patterns. But if all they’ve got to say is that we need to stock up on stationery before the new school term begins or more six packs and nachos when there’s a football game on TV in the evening… then luckily we haven’t been waiting on their insights.”
Data can generate relevant conclusions… but, without bringing any value to the company.
Where to find big data projects profitability?
Producing value from a big data projects is important, but generating profitability is even better. IT services firms and software vendors have been surfing the bid data wave by explaining that data is synonymous with costly projects and prohibitively expensive. With high buy-in costs and hit-and-miss results (one in six cases, according to the McKinsey survey): is it all worth it?
We have become too used to announcements and claims. When AlphaGo beat the world Go champion, its raw performance was acclaimed and there was talk about the future of unsupervised learning, and so on. But the investment involved seemed to have been forgotten, such as the dozens of employees working on the project for seven years and unlimited computer resources. Not everyone is Google. Not everyone can afford to spend such vast sums on publicity. A representative from an automotive OEM once said:
Our manufactured parts reject rate for this part is 40%. I’m convinced that we can halve that number by tapping into our machine data. Although manufacturing part costs is about $4 each and any late deliveries could cost the company dearly in terms of penalties, is it really worth investing millions into machine learning at the risk of disrupting our chain?”
Good question. Big data projects can improve performance and reduce costs, albeit without generating a positive return on investment, especially if the investment is considerable.
Focus on real business issues
Gartner’s famous 2017 hype cycle put machine learning just after the first peak (the “Peak of Inflated Expectations”): the moment when the first results are disappointing in light of the hopes raised by the technology. The big data projects hype dies away and other subjects grab the media spotlight.
This is exactly the current atmosphere. Now is the time when true projects are launched and real use cases emerge. Our customers have been telling us:
“I’ve got a real business problem that need to be solved. It’s a real business case where data analytics can be profitable to the company. It’s not as spectacular as big data, and won’t revolutionize the world, but it will be useful for our company.”
Instate an iterative process
Companies can generate value from iterative processes in close liaison with the business function with a platform like the ForePaaS Platform. Because nobody can predict the best way of exploiting data in a given sector for big data projects. Only iterative-based work practices can find the path. The path is lined with a succession of specific benefits driven by feedback from internal and external customers and the actual benefits observed, not the result of a think tank’s brainstorming session.
When the circulation of data has been automated and become increasingly reliable, and when its daily benefits for guiding and supporting decisions have been substantiated, and when the data culture has been disseminated throughout the company, then… the team of 10 data scientists will be a major asset for achieving even greater progress.
For more articles on cloud infrastructure, data, analytics, machine learning, and data science, follow Paul Sinaï on Towards Data Science.
Get started with ForePaaS for FREE!
Discover how to make your journey towards successful ML/ Analytics – Painless
