Analyst: Hadoop ecosystem to be worth $812 million in 2016

IDC has published a report, Worldwide Hadoop-MapReduce Ecosystem Software 2012–2016 Forecast which estimates the current size of the Hadoop ecosystem and looks forward at the growth potential of the market. IDC says that in 2011, the Hadoop ecosystem generated revenues of around $77 million and it projects a compound growth rate of 60 per cent over the coming years to take that revenue to $812 million in 2016. It expects the Hadoop and MapReduce market to "develop along the lines established by the development of the Linux ecosystem", and believes that vendors in the market have already raised around $300 million in venture capital to fund the market's development.

Hadoop systems, based on the Apache Hadoop platform, allow for the creation of massive distributed systems which implement the MapReduce algorithm. MapReduce allows a task to be mapped out to numerous nodes, with each node taking on a segment of the problem. The results from each node are then merged and reduced down to a final result. This simple architecture has proved ideal for processing large amounts of data, so-called "big data". It has also fostered an entire ecosystem of software projects around it, providing services from getting data to the nodes through databases and file systems to orchestrating the management of the distributed network.

IDC believes that one of the key drivers in the Hadoop adoption will be the increased amount of data available from social media which companies will want to mine for "full value". The growth will be inhibited though by the scarcity of tools and staff with appropriate skills to harness Hadoop. Another problem could well be competition with closed source alternatives leading to lower licence fees; this could slow the growth of revenues for the open source vendors.

(djwm)