Ness Point of View The Big Data Landscape

Ness Digital Engineering | ness.com

Perspective In January 2016, a number of analysts released their Big Data Landscape overview. Each analyst has their own taxonomy for organizing the players in the Big Data space, and their own method for categorizing the companies and technologies. What all these taxonomies have in common, despite their differences, is that they are larger and more complex than the Big Data Landscape overviews released at the beginning of 2015.

In theory, this should be viewed as a positive indication of the continued growth of the Big Data industry, and of the continued need and capacity for innovation. In practice, the continued proliferation of Big Data technologies is harmful, to the buyers, the vendors and the industry as a whole. The sheer number of contending technologies, products and companies creates a cacophony of competing claims that elicits a predictable response from buyers: caution.

In any mature high tech product category, there is always a 500 pound gorilla that leads the market, a couple of smaller competitors, and a number of niche players. For example, if you are looking for Cloud hosting, you can choose between the market leader (Amazon), a couple of reasonable sized competitors (Microsoft and Google), and a number of niche players (e.g., Heroku for Ruby). The Big Data market will one day look like this, once it matures and the winners are crowned. In the meanwhile, the ever growing list of players and technologies is a clear sign that the Big Data market is still immature.

The good news for consumers: winter is coming. Venture capital funding for new Big Data companies is drying up. Gone are the days when every new idea around Big Data got funded, companies are finding it much harder to raise Series A and Series B rounds needed in order to scale up their sales efforts, and companies that have already been funded are being ordered by investors to tighten their belts. After a feeding frenzy of investment in Big Data companies, investors are looking to double down on a few winners and drop the rest of their portfolio.

Next Steps If you are looking to launch a Big Data initiative, you probably do not have the luxury of waiting for the shakedown to occur before starting your project. In most business sectors, a large competitor has already declared success based on insights derived from some Big Data initiative, and the rest of the industry has to react now or risk being left behind.

In this volatile landscape, with too many products and technologies to choose from, you need to take advantage of the experience of others. The field is full of competing or overlapping products, each of which claims to be the right solution for Big Data. Left to evaluate this cacophony of conflicting voices is your organization. It’s hardly a fair battle – the only way to cut through the hype around a product is to try it yourself, and/or talk to someone you trust who is using it.

The best advice to a company contemplating a Big Data initiative for the first time is: Get help. Perhaps you can hire experts from outside your organization who have already built such a system. Perhaps you can build up a network of friends who have made the beginner’s mistakes and can share their insights with you. Another option is to partner with a company like Ness Software Engineering Services that has managed a broad range of Big Data projects and technologies, and has a proven track record of success.